Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Artificial intelligence chatbot interpretation of ophthalmic multimodal imaging cases

Eye (2024)Cite this article

Subjects

The integration of artificial intelligence (AI) within ophthalmology represents a paradigm shift that can redefine patient interactions and healthcare delivery [1]. The large language model ChatGPT-4 (OpenAI) has been recognized for its rapidly advancing proficiency in medical prompt analysis [2, 3]. While earlier iterations of this chatbot were confined to text-based input, our recent work has documented a remarkable expansion in its ophthalmic applications [4,5,6]. In particular, the latest iteration of the chatbot stands at the threshold of potentially revolutionizing the efficiency with which multimodal ophthalmic images are assessed.

Ophthalmologists heavily rely on the nuanced interpretation of multimodal imaging, a cornerstone in achieving diagnostic accuracy and improving patient outcomes [7]. Hence, we aimed to evaluate the chatbot’s performance on open-ended queries regarding multimodal ophthalmic cases, a crucial step in characterizing the real-world clinical utility of this burgeoning technology.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Data availability

All data generated or analyzed during this study are included in this published article and its supplementary information files.

References

  1. Li Z, Wang L, Wu X, Jiang J, Qiang W, Xie H, et al. Artificial intelligence in ophthalmology: the path to the real-world clinic. Cell Rep Med. 2023;4:101095.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Mihalache A, Huang RS, Popovic MM, Muni RH. ChatGPT-4: an assessment of an upgraded artificial intelligence chatbot in the United States Medical Licensing Examination. Med Teach. 2023: 1–7. Available at: https://www.tandfonline.com/doi/full/10.1080/0142159X.2023.2249588.

  3. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models Dagan A (ed). PLoS Digit Health. 2023;2:e0000198 https://dx.plos.org/10.1371/journal.pdig.0000198.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Mihalache A, Huang RS, Popovic MM, Patil NS, Shor R, Pandya BU, et al. Accuracy of an artificial intelligence chatbot’s interpretation of clinical ophthalmic images. JAMA Ophthalmol. https://doi.org/10.1001/jamaophthalmol.2024.0017.

  5. Mihalache A, Popovic MM, Muni RH. Performance of an artificial intelligence chatbot in ophthalmic knowledge assessment. JAMA Ophthalmol. 2023;141:589–97.

    Article  PubMed  Google Scholar 

  6. Mihalache A, Huang RS, Popovic MM, Muni RH. Performance of an upgraded artificial intelligence chatbot for ophthalmic knowledge assessment. JAMA Ophthalmol. 2023;141:798–800.

    Article  PubMed  Google Scholar 

  7. Kawali A, Pichi F, Avadhani K, Invernizzi A, Hashimoto Y, Mahendradas P. Multimodal imaging of the normal eye. Ocul Immunol Inflamm. 2017;25:726–36.

    Article  Google Scholar 

  8. Dave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023;6:1169595.

    Article  PubMed  PubMed Central  Google Scholar 

  9. OpenAI. GPT-4V(ision) system card. OpenAI. 2023. Available at: https://openai.com/research/gpt-4v-system-card Accessed 7 Jan 2024.

  10. Anton N, Doroftei B, Curteanu S, Catãlin L, Ilie OD, Târcoveanu F, et al. Comprehensive review on the use of artificial intelligence in ophthalmology and future research directions. Diagnostics. 2023;13:100.

    Article  Google Scholar 

Download references

Acknowledgements

RHM’s research is supported by the Silber TARGET Fund. AP is the cofounder of OCTCases, and AM and BUP are involved in the operation of the OCTCases website. Information reported in this manuscript has not been previously presented at a conference. Data was collected from the artificial intelligence chatbot ChatGPT-4 developed by OpenAI.

Funding

Silber TARGET Fund.

Author information

Authors and Affiliations

  1. Temerty School of Medicine, University of Toronto, Toronto, ON, Canada

    Andrew Mihalache, Ryan S. Huang & Bhadra U. Pandya

  2. Department of Ophthalmology and Vision Sciences, University of Toronto, Toronto, ON, Canada

    Miguel Cruz-Pimentel, Marko M. Popovic, Reut Shor, Austin Pereira & Rajeev H. Muni

  3. Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada

    Nikhil S. Patil

  4. Department of Ophthalmology, St. Michael’s Hospital/Unity Health Toronto, Toronto, ON, Canada

    Rajeev H. Muni

Authors
  1. Andrew Mihalache
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryan S. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miguel Cruz-Pimentel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nikhil S. Patil
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marko M. Popovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bhadra U. Pandya
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Reut Shor
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Austin Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Rajeev H. Muni
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AM was responsible for the conception of the study, reviewing the chatbot’s responses, analyzing data, interpreting results, and writing the manuscript. RSH was responsible for prompting the chatbot, reviewing the chatbot’s responses, and extracting data. MCP was responsible for reviewing the chatbot’s responses and revision of the manuscript. NSP was responsible for revision of the manuscript. MMP was responsible for the conception of the study and revision of the manuscript. BUP was responsible for revision of the manuscript. RS was responsible for revision of the manuscript. AP was responsible for supervision of the study and revision of the manuscript. RHM was responsible for supervision of the study and revision of the manuscript.

Corresponding author

Correspondence to Rajeev H. Muni.

Ethics declarations

Competing interests

AM: None; RSH: None; MCP: None; NSP: None; MMP: Financial support (to institution) – PSI Foundation, Fighting Blindness Canada; BUP: None; RS: None; AP: None; RHM: Consultant - Alcon, Apellis, AbbVie, Bayer, Bausch Health, Roche; Financial Support (to institution)- Alcon, AbbVie, Bayer, Novartis, Roche.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Figure Legends

eFigure 1. Sample case entry onto the chatbot.

41433_2024_3074_MOESM3_ESM.pdf

eFigure 2. Example response from the chatbot (left) compared to a model answer from OCTCases (right) for an open-ended question answered fully correctly.

41433_2024_3074_MOESM4_ESM.pdf

eFigure 3. Example response from the chatbot (left) compared to a model answer from OCTCases (right) for an open-ended question answered partially correctly.

41433_2024_3074_MOESM5_ESM.pdf

eFigure 4. Example response from the chatbot (left) compared to a model answer from OCTCases (right) for an open-ended question answered fully incorrectly.  

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mihalache, A., Huang, R.S., Cruz-Pimentel, M. et al. Artificial intelligence chatbot interpretation of ophthalmic multimodal imaging cases. Eye (2024). https://doi.org/10.1038/s41433-024-03074-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41433-024-03074-5

Search

Advanced search

Quick links