Abstract

Aim: This study evaluates the accuracy and reliability of ChatGPT’s responses to open-ended questions in otology and audiology, focusing on its potential use in training ear, nose, and throat (ENT) professionals. As artificial intelligence (AI) applications like ChatGPT become more accessible to healthcare professionals and the public, ensuring that the information provided is reliable, accurate, and reproducible is crucial, especially in the medical field.

Materials and Methods: In March 2024, 60 audiology-related questions, categorized as ‘general audiology,’ ‘hearing,’ and ‘balance,’ were posed twice using ChatGPT (version 4) on the same computer to assess reproducibility. The responses were recorded as the '1st' and '2nd' answers. Three ENT specialists independently evaluated the answers to ensure accuracy, with a third reviewer specializing in audiology assessing the agreement between the responses. Answers were categorized as 1 (completely correct), 2 (partially correct), 3 (mixed accuracy), or 4 (incorrect). Analyses were conducted separately for each subgroup.

Results: Statistically significant difference was found between the two responses in general audiology questions (p = 0.008) and across all responses collectively (p = 0.002), while no significant difference was observed in hearing and balance questions (p > 0.05). The second responses had higher accuracy rates, with 65%, 80%, and 70% accuracy for general audiology, hearing, and balance areas, respectively.

Conclusion: ChatGPT's second responses were more accurate and reliable, making it a valuable resource for clinicians despite occasional misleading answers. With continued advancements, AI is expected to become a more reliable tool in audiology.

Keywords: answer, artificial intelligence, audiology, Chat-GPT, head and neck surgery

Copyright and license

Copyright © 2026 The Author(s). This is an open-access article published by Bolu İzzet Baysal Training and Research Hospital under the terms of the Creative Commons Attribution License (CC BY) which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is properly cited.

How to cite

1.
Saygı Uysal G, Özdoğan A, Küçüktağ Z, Altan E. Evaluation of the accuracy of ChatGPT-generated information in the field of general audiology. Northwestern Med J. 2026;6(1):321-3. https://doi.org/10.54307/2026.NWMJ.182

References

  1. Munoz-Zuluaga C, Zhao Z, Wang F, Greenblatt MB, Yang HS. Assessing the accuracy and clinical utility of ChatGPT in laboratory medicine. Clin Chem. 2023; 69(8): 939-40. https://doi.org/10.1093/clinchem/hvad058
  2. The Lancet Digital Health. ChatGPT: friend or foe? Lancet Digit Health. 2023; 5(3): e102. https://doi.org/10.1016/S2589-7500(23)00023-7
  3. Van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: Five priorities for research. Nature. 2023; 614(7947): 224-6. https://doi.org/10.1038/d41586-023-00288-7
  4. Kuşcu O, Pamuk AE, Sütay Süslü N, Hosal S. Is ChatGPT accurate and reliable in answering questions regarding head and neck cancer? Front Oncol. 2023; 13: 1256459. https://doi.org/10.3389/fonc.2023.1256459
  5. Masters K. Artificial intelligence in medical education. Med Teach. 2019; 41(9): 976-80. https://doi.org/10.1080/0142159X.2019.1595557
  6. Qu RW, Qureshi U, Petersen G, Lee SC. Diagnostic and management applications of ChatGPT in structured otolaryngology clinical scenarios. OTO Open. 2023; 7(3): e67. https://doi.org/10.1002/oto2.67
  7. Chiesa-Estomba CM, Lechien JR, Vaira LA, et al. Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. Eur Arch Otorhinolaryngol. 2024; 281(4): 2081-6. https://doi.org/10.1007/s00405-023-08104-8
  8. Hoch CC, Wollenberg B, Lüers JC, et al. ChatGPT’s quiz skills in different otolaryngology subspecialties: An analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Otorhinolaryngol. 2023; 280(9): 4271-8. https://doi.org/10.1007/s00405-023-08051-4
  9. D’Amico RS, White TG, Shah HA, Langer DJ. I asked a ChatGPT to write an editorial about how we can incorporate chatbots into neurosurgical research and patient care…. Neurosurgery. 2023; 92(4): 663-4. https://doi.org/10.1227/neu.0000000000002414
  10. Amann J, Vayena E, Ormond KE, Frey D, Madai VI, Blasimme A. Expectations and attitudes towards medical artificial intelligence: A qualitative study in the field of stroke. PLoS One. 2023; 18(1): e0279088. https://doi.org/10.1371/journal.pone.0279088
  11. Else H. Abstracts written by ChatGPT fool scientists. Nature. 2023; 613(7944): 423. https://doi.org/10.1038/d41586-023-00056-7
  12. Haupt CE, Marks M. AI-generated medical advice-GPT and beyond. JAMA. 2023; 329(16): 1349-50. https://doi.org/10.1001/jama.2023.5321
  13. Knoedler L, Baecher H, Kauke-Navarro M, et al. Towards a reliable and rapid automated grading system in facial palsy patients: Facial palsy surgery meets computer science. J Clin Med. 2022; 11(17): 4998. https://doi.org/10.3390/jcm11174998
  14. Nielsen JPS, von Buchwald C, Grønhøj C. Validity of the large language model ChatGPT (GPT4) as a patient information source in otolaryngology by a variety of doctors in a tertiary otorhinolaryngology department. Acta Otolaryngol. 2023; 143(9): 779-82. https://doi.org/10.1080/00016489.2023.2254809
  15. Kim HY. A case report on ground-level alternobaric vertigo due to eustachian tube dysfunction with the assistance of conversational generative pre-trained transformer (ChatGPT). Cureus. 2023; 15(3): e36830. https://doi.org/10.7759/cureus.36830
  16. Ayoub NF, Lee YJ, Grimm D, Divi V. Head-to-head comparison of ChatGPT versus Google search for medical knowledge acquisition. Otolaryngol Head Neck Surg. 2024; 170(6): 1484-91. https://doi.org/10.1002/ohn.465
  17. Crowson MG, Dixon P, Mahmood R, et al. Predicting postoperative cochlear implant performance using supervised machine learning. Otol Neurotol. 2020; 41(8): e1013-23. https://doi.org/10.1097/MAO.0000000000002710
  18. Wang B, Zheng J, Yu JF, et al. Development of artificial intelligence for parathyroid recognition during endoscopic thyroid surgery. Laryngoscope. 2022; 132(12): 2516-23. https://doi.org/10.1002/lary.30173
  19. Lim SJ, Jeon ET, Baek N, et al. Prediction of hearing prognosis after intact canal wall mastoidectomy with tympanoplasty using artificial intelligence. Otolaryngol Head Neck Surg. 2023; 169(6): 1597-605. https://doi.org/10.1002/ohn.472
  20. Arambula AM, Bur AM. Ethical considerations in the advent of artificial intelligence in otolaryngology. Otolaryngol Head Neck Surg. 2020; 162(1): 38-9. https://doi.org/10.1177/0194599819889686
  21. Park I, Joshi AS, Javan R. Potential role of ChatGPT in clinical otolaryngology explained by ChatGPT. Am J Otolaryngol. 2023; 44(4): 103873. https://doi.org/10.1016/j.amjoto.2023.103873
  22. Brennan L, Balakumar R, Bennett W. The role of ChatGPT in enhancing ENT surgical training - a trainees’ perspective. J Laryngol Otol. 2024; 138(5): 480-6. https://doi.org/10.1017/S0022215123001354
  23. Levine DM, Tuwani R, Kompa B, et al. The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: An observational study. Lancet Digit Health. 2024; 6(8): e555-61. https://doi.org/10.1016/S2589-7500(24)00097-9
  24. Rao A, Kim J, Kamineni M, Pang M, Lie W, Succi MD. Evaluating ChatGPT as an adjunct for radiologic decision-making. medRxiv. 2023; 02.02.23285399. https://doi.org/10.1101/2023.02.02.23285399
  25. Bellinger JR, De La Chapa JS, Kwak MW, Ramos GA, Morrison D, Kesser BW. BPPV information on Google versus AI (ChatGPT). Otolaryngol Head Neck Surg. 2024; 170(6): 1504-11. https://doi.org/10.1002/ohn.506
  26. Bellinger JR, Kwak MW, Ramos GA, Mella JS, Mattos JL. Quantitative comparison of chatbots on common rhinology pathologies. Laryngoscope. 2024; 134(10): 4225-31. https://doi.org/10.1002/lary.31470
  27. Swanepoel D, Manchaiah V, Wasmann JW. The rise of AI chatbots in hearing health care. The Hearing Journal. 2023; 76(04): 26,30,32. https://doi.org/10.1097/01.HJ.0000927336.03567.3e
  28. Patel EA, Fleischer L, Filip P, et al. Comparative performance of ChatGPT 3.5 and GPT4 on rhinology standardized board examination questions. OTO Open. 2024; 8(2): e164. https://doi.org/10.1002/oto2.164
  29. Sireci F, Lorusso F, Immordino A, et al. ChatGPT as a new tool to select a biological for chronic rhino sinusitis with polyps, “Caution Advised” or “Distant Reality”? J Pers Med. 2024; 14(6): 563. https://doi.org/10.3390/jpm14060563
  30. Riestra-Ayora J, Vaduva C, Esteban-Sánchez J, et al. ChatGPT as an information tool in rhinology. Can we trust each other today? Eur Arch Otorhinolaryngol. 2024; 281(6): 3253-9. https://doi.org/10.1007/s00405-024-08581-5
  31. Workman AD, Rathi VK, Lerner DK, Palmer JN, Adappa ND, Cohen NA. Utility of a LangChain and OpenAI GPT-powered chatbot based on the international consensus statement on allergy and rhinology: Rhinosinusitis. Int Forum Allergy Rhinol. 2024; 14(6): 1101-9. https://doi.org/10.1002/alr.23310
  32. Goodman RS, Patrinely JR, Stone CA, et al. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open. 2023; 6(10): e2336483. https://doi.org/10.1001/jamanetworkopen.2023.36483
  33. Deiana G, Dettori M, Arghittu A, Azara A, Gabutti G, Castiglia P. Artificial intelligence and public health: Evaluating ChatGPT responses to vaccination myths and misconceptions. Vaccines (Basel). 2023; 11(7): 1217. https://doi.org/10.3390/vaccines11071217
  34. Patil NS, Huang RS, van der Pol CB, Larocque N. Comparative performance of ChatGPT and bard in a text-based radiology knowledge assessment. Can Assoc Radiol J. 2024; 75(2): 344-50. https://doi.org/10.1177/08465371231193716
  35. Jedrzejczak WW, Skarzynski PH, Raj-Koziak D, Sanfins MD, Hatzopoulos S, Kochanek K. ChatGPT for tinnitus information and support: Response accuracy and retest after three and six months. Brain Sci. 2024; 14(5): 465. https://doi.org/10.3390/brainsci14050465
  36. Karimov Z, Allahverdiyev I, Agayarov OY, Demir D, Almuradova E. ChatGPT vs UpToDate: Comparative study of usefulness and reliability of Chatbot in common clinical presentations of otorhinolaryngology-head and neck surgery. Eur Arch Otorhinolaryngol. 2024; 281(4): 2145-51. https://doi.org/10.1007/s00405-023-08423-w
  37. Zalzal HG, Cheng J, Shah RK. Evaluating the current ability of ChatGPT to assist in professional otolaryngology rducation. OTO Open. 2023; 7(4): e94. https://doi.org/10.1002/oto2.94