Abstract

Aim: To assess the readability of ChatGPT-4o and ChatGPT-5 responses to questions about ear care.

Materials and Methods: Ten questions about ear care from the World Health Organization’s ‘Questions and Answers’ section were each posed seven times to both ChatGPT-4o and ChatGPT-5. A total of 70 responses were obtained and analyzed for readability.

Results: For ChatGPT-4o, the mean readability scores were: Flesch Reading Ease 62.84 ± 8.32, Flesch–Kincaid Grade Level 8.13 ± 1.59, Simple Measure of Gobbledygook 7.35 ± 1.42, Gunning Fog Index 8.44 ± 1.79, Coleman–Liau Index 9.62 ± 1.71, and Automated Readability Index 8.70 ± 2.02. For ChatGPT-5, the corresponding scores were: Flesch Reading Ease 72.93 ± 7.69, Flesch–Kincaid Grade Level 5.75 ± 1.17, Simple Measure of Gobbledygook 6.35 ± 0.79, Gunning Fog Index 7.37 ± 1.22, Coleman–Liau Index 7.32 ± 1.52, and Automated Readability Index 5.78 ± 1.42. ChatGPT-5 demonstrated better readability than ChatGPT-4o across all readability indices (p<0.001).

Conclusion: The readability of the answers produced by ChatGPT-4o is “standard”, and that of ChatGPT-5 is “fairly easy”. ChatGPT-5 responses showed improved readability over ChatGPT-4o, aligning more closely with patient education readability standards and underscoring its potential as an accessible source of health information.

Keywords: artificial intelligence, ChatGPT health information, ear care, readability

Copyright and license

Copyright © 2026 The Author(s). This is an open-access article published by Bolu İzzet Baysal Training and Research Hospital under the terms of the Creative Commons Attribution License (CC BY) which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is properly cited.

How to cite

1.
Özyılmaz C, Karakuş MF. Readability assessment of ChatGPT-4 and ChatGPT-5 responses to ear care questions. Northwestern Med J. 2026;Early View:1-9. https://doi.org/10.54307/NWMJ.2026.256

References

  1. Koirala B, Manandhar S, Shah RK. Knowledge, attitude and practice of ear care among pre clinical medical students at Birat Medical College Teaching Hospital. Birat Journal of Health Sciences. 2020; 5(3): 1236-1240. https://doi.org/10.3126/bjhs.v5i3.33705
  2. Sağlam Ö, Saat B, Dursun E, et al. The prevalence of cerumen in young male population. KBB-Forum. 2013; 12(2): 32-5.
  3. Alateeq OM. Knowledge Attitude and practice of ear care in Saudi community. Baqai J Health Sci. 2017; 20(2): 11-4.
  4. Alrajhi MS, Alim BM, Aldokhayel SD, Zeitouni LM, Al Tawil LK, Alzahrani FA. Knowledge, attitudes, and practices pertaining to cotton-bud usages and the complications related to their misuse among outpatients in an ear, nose, and throat clinic. Journal of Nature and Science of Medicine. 2019; 2(4): 220-5.
  5. Tonsaker T, Bartlett G, Trpkov C. Health information on the Internet: gold mine or minefield? Can Fam Physician. 2014; 60(5): 407-8.
  6. Beheshti M, Toubal IE, Alaboud K, et al. Evaluating the reliability of chatgpt for health-related questions: a systematic review. Informatics. 2025; 12(1): 9. https://doi.org/10.3390/informatics12010009
  7. Johnson D, Goodman R, Patrinely J, et al. Assessing the accuracy and reliability of AI-generated medical responses: An evaluation of the Chat-GPT model. Research Square [Preprint]. 2023. https://doi.org/10.21203/rs.3.rs-2566942/v1
  8. Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023; 2(2): e0000198. https://doi.org/10.1371/journal.pdig.0000198
  9. Antaki F, Touma S, Milad D, El-Khoury J, Duval R. Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings. Ophthalmol Sci. 2023; 3(4): 100324. https://doi.org/10.1016/j.xops.2023.100324
  10. Huh S. Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study. J Educ Eval Health Prof. 2023; 20: 1. https://doi.org/10.3352/jeehp.2023.20.1
  11. Wang S, Mo C, Chen Y, Dai X, Wang H, Shen X. Exploring the performance of ChatGPT-4 in the Taiwan audiologist qualification examination: Preliminary observational study highlighting the potential of AI chatbots in hearing care. JMIR Med Educ. 2024; 10: e55595. https://doi.org/10.2196/55595
  12. Zalzal HG, Abraham A, Cheng J, Shah RK. Can ChatGPT help patients answer their otolaryngology questions? Laryngoscope Investig Otolaryngol. 2024; 9(1): e1193. https://doi.org/10.1002/lio2.1193
  13. Qu RW, Qureshi U, Petersen G, Lee SC. Diagnostic and management applications of ChatGPT in structured otolaryngology clinical scenarios. OTO Open. 2023; 7(3): e67. https://doi.org/10.1002/oto2.67
  14. Soon S, Perry B. Paging Dr. ChatGPT: safety, accuracy and readability of ChatGPT in ENT emergencies. Australian Journal of Otolaryngology. 2025; 8: 8. https://doi.org/10.21037/ajo-24-56
  15. Adams RC, Smith TP, Metts JK, Ross JW. Readability-its applicability to education of patients by pharmacy. Hosp Pharm. 1979; 14(11): 654-6.
  16. World Health Organization (WHO). Deafness and hearing loss: ear care. Available at: https://www.who.int/news-room/questions-and-answers/item/deafness-and-hearing-loss-ear-care (Accessed on June 4, 2025).
  17. Jindal P, MacDermid JC. Assessing reading levels of health information: uses and limitations of flesch formula. Educ Health (Abingdon). 2017; 30(1): 84-8. https://doi.org/10.4103/1357-6283.210517
  18. Flesch R. A new readability yardstick. J Appl Psychol. 1948; 32(3): 221-33. https://doi.org/10.1037/h0057532
  19. Gencer A. Readability analysis of ChatGPT’s responses on lung cancer. Sci Rep. 2024; 14(1): 17234. https://doi.org/10.1038/s41598-024-67293-2
  20. Hedman AS. Using the SMOG formula to revise a health-related document. Am J Health Educ. 2008; 39(1): 61-4. https://doi.org/10.1080/19325037.2008.10599016
  21. Azer SA, AlOlayan TI, AlGhamdi MA, AlSanea MA. Inflammatory bowel disease: An evaluation of health information on the internet. World J Gastroenterol. 2017; 23(9): 1676-96. https://doi.org/10.3748/wjg.v23.i9.1676
  22. Readability Scoring System. Available at: https://readabilityformulas.com/readability-scoring-system.php (Accessed on August 20, 2025).
  23. Baumann E, Czerwinski F, Rosset M, Seelig M, Suhr R. How do people in Germany seek health information? Insights from the first wave of HINTS Germany. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2020; 63(9): 1151-60. https://doi.org/10.1007/s00103-020-03192-x
  24. Kianian R, Carter M, Finkelshtein I, Eleswarapu SV, Kachroo N. Application of artificial intelligence to patient-targeted health information on kidney stone disease. J Ren Nutr. 2024; 34(2): 170-6. https://doi.org/10.1053/j.jrn.2023.10.002
  25. Fritsch SJ, Blankenheim A, Wahl A, et al. Attitudes and perception of artificial intelligence in healthcare: A cross-sectional survey among patients. Digit Health. 2022; 8: 20552076221116772. https://doi.org/10.1177/20552076221116772
  26. Høj S, Thomsen SF, Ulrik CS, Meteran H, Sigsgaard T, Meteran H. Evaluating the scientific reliability of ChatGPT as a source of information on asthma. J Allergy Clin Immunol Glob. 2024; 3(4): 100330. https://doi.org/10.1016/j.jacig.2024.100330
  27. Özcan F, Örücü Atar M, Köroğlu Ö, Yılmaz B. Assessment of the reliability and usability of ChatGPT in response to spinal cord injury questions. J Spinal Cord Med. 2025; 48(5): 852-7. https://doi.org/10.1080/10790268.2024.2361551
  28. Cavnar Helvaci B, Hepsen S, Candemir B, et al. Assessing the accuracy and reliability of ChatGPT’s medical responses about thyroid cancer. Int J Med Inform. 2024; 191: 105593. https://doi.org/10.1016/j.ijmedinf.2024.105593
  29. Onder CE, Koc G, Gokbulut P, Taskaldiran I, Kuskonmaz SM. Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy. Sci Rep. 2024; 14(1): 243. https://doi.org/10.1038/s41598-023-50884-w
  30. Long C, Lowe K, Santos A, et al. Evaluating ChatGPT-4 in otolaryngology-head and neck surgery board examination using the CVSA model. medRxiv [Preprint]. 2023. https://doi.org/10.1101/2023.05.30.23290758
  31. Rooney MK, Santiago G, Perni S, et al. Readability of Patient Education Materials From High-Impact Medical Journals: A 20-Year Analysis. J Patient Exp. 2021; 8: 2374373521998847. https://doi.org/10.1177/2374373521998847
  32. Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008; 53(10): 1310-15. https://doi.org/10.4187/respcare.08531310
  33. Momenaei B, Wakabayashi T, Shahlaee A, et al. Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases. Ophthalmol Retina. 2023; 7(10): 862-8. https://doi.org/10.1016/j.oret.2023.05.022
  34. Şimşek SD, Şensöğüt C. The use of readability formulas in polytrauma. Acta Medica Young Doctors. 2025; 1(1): 2-9. https://doi.org/10.5281/zenodo.17045168