PDF herunterladen
CC BY 4.0 · Eur J Dent
DOI: 10.1055/s-0045-1809155
Original Article

Artificial Intelligence Chatbots as Sources of Implant Dentistry Information for the Public: Validity and Reliability Assessment

Tahani Mohammed Binaljadm
1   Department of Substitutive Dental Sciences (Prosthodontics), College of Dentistry, Taibah University, Al Madinah, Saudi Arabia
,
Ahmed Yaseen Alqutaibi 
1   Department of Substitutive Dental Sciences (Prosthodontics), College of Dentistry, Taibah University, Al Madinah, Saudi Arabia
2   Department of Prosthodontics, College of Dentistry, Ibb University, Ibb, Yemen
,
Esam Halboub
3   Department of Maxillofacial Surgery and Diagnostic Science, College of Dentistry, Jazan University, Jazan, Saudi Arabia
,
Muhammad Sohail Zafar 
4   Department of Clinical Sciences, College of Dentistry, Ajman University, Ajman, United Arab Emirates
5   Centre of Medical and Bio-allied Health Sciences Research, Ajman University, Ajman, United Arab Emirates
6   School of Dentistry, Jordan University, Amman, Jordan
,
Samah Saker
1   Department of Substitutive Dental Sciences (Prosthodontics), College of Dentistry, Taibah University, Al Madinah, Saudi Arabia
› Institutsangaben
› Weitere Informationen
Auch verfügbar auf
   Lizenzen und Reprints

Abstract

Objectives

This study assessed the reliability and validity of responses from three chatbot systems—OpenAI's GPT-3.5, Gemini, and Copilot—concerning frequently asked questions (FAQs) in implant dentistry posed by patients.

Materials and Methods

Twenty FAQs were prompted to three chatbots in three different times utilizing their respective application programming interfaces. The responses were assessed for validity (low and high threshold) and reliability by two prosthodontic consultants using a five-point Likert scale.

Statistical Analysis

The test of normality was utilized using the Shapiro–Wilk test. Differences between different chatbots regarding the quantitative variables in a given (fixed) time point and between the same chatbots in different time points were assessed using Friedman's two-way analysis of variance by ranks, followed by pairwise comparisons. All statistical analyses were conducted using the SPSS (Statistical Package for Social Sciences) Version 26.0 software program.

Results

GPT-3.5 provided the longest responses, while Gemini was the most concise. All chatbots advised consulting dental professionals more frequently. Validity was high under the low-threshold test but low under the high-threshold test, with Copilot scoring the highest. Reliability was high for all, with Gemini achieving perfect consistency.

Conclusion

Chatbots showed consistent and generally valid responses with some variability in accuracy and details. While the chatbots demonstrated a high degree of reliability, their validity—especially under high-threshold criterion—remains limited. Improvements in accuracy and comprehensiveness are necessary for more effective use in providing information about dental implants.

Keywords

artificial intelligence - dental education - ChatGPT - implantology - prosthodontics

Supplementary Material



Publikationsverlauf

Artikel online veröffentlicht:
20. Mai 2025

© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Thieme Medical and Scientific Publishers Pvt. Ltd.
A-12, 2nd Floor, Sector 2, Noida-201301 UP, India

 

  • References

  • 1 Alshadidi AAF, Alshahrani AA, Aldosari LIN. et al. Investigation on the application of artificial intelligence in prosthodontics. Appl Sci (Basel) 2023; 13 (08) 1-17
    Suche in Google Scholar
  • 2 Alqutaibi AY, Algabri RS, Elawady D, Ibrahim WI. Advancements in artificial intelligence algorithms for dental implant identification: a systematic review with meta-analysis. J Prosthet Dent 2023:S0022-3913(23)00783-7
  • 3 Alqutaibi AY, Hamadallah HH, Alassaf MS, Othman AA, Qazali AA, Alghauli MA. Artificial intelligence-driven automation of nasoalveolar molding device planning: a systematic review. J Prosthet Dent 2024:S0022-3913(24)00637-1
  • 4 Schleyer TK, Thyvalikakath TP, Spallek H, Torres-Urquidy MH, Hernandez P, Yuhaniak J. Clinical computing in general dentistry. J Am Med Inform Assoc 2006; 13 (03) 344-352
    PubMedSuche in Google Scholar
  • 5 Meyer P, Noblet V, Mazzara C, Lallement A. Survey on deep learning for radiotherapy. Comput Biol Med 2018; 98: 126-146
    PubMedSuche in Google Scholar
  • 6 Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst 2023; 47 (01) 1-5
    Suche in Google Scholar
  • 7 Teubner T, Flath CM, Weinhardt C, van der Aalst W, Hinz O. Welcome to the era of ChatGPT et al. The prospects of large language models. Bus Inf Syst Eng 2023; 65 (02) 95-101
    Suche in Google Scholar
  • 8 Rao A, Kim J, Kamineni M, Pang M, Lie W, Succi MD. Evaluating ChatGPT as an adjunct for radiologic decision-making. MedRxiv 2023; (ePub ahead of print)
    CrossrefPubMedSuche in Google Scholar
  • 9 Potapenko I, Boberg-Ans LC, Stormly Hansen M, Klefter ON, van Dijk EHC, Subhi Y. Artificial intelligence-based chatbot patient information on common retinal diseases using ChatGPT. Acta Ophthalmol 2023; 101 (07) 829-831
    PubMedSuche in Google Scholar
  • 10 Yeo YH, Samaan JS, Ng WH. et al. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol 2023; 29 (03) 721-732
    CrossrefPubMedSuche in Google Scholar
  • 11 Vaishya R, Misra A, Vaish A. ChatGPT: Is this version good for healthcare and research?. Diabetes Metab Syndr 2023; 17 (04) 102744
    CrossrefPubMedSuche in Google Scholar
  • 12 Kung TH, Cheatham M, Medenilla A. et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health 2023; 2 (02) e0000198
    CrossrefPubMedSuche in Google Scholar
  • 13 Radziwill NM, Benton MC. Evaluating quality of chatbots and intelligent conversational agents. arXiv 2017 (ePub ahead of print) https://doi.org/10.48550/arXiv.1704.04579
    Crossref
  • 14 Freire Y, Santamaría Laorden A, Orejas Pérez J, Gómez Sánchez M, Díaz-Flores García V, Suárez A. ChatGPT performance in prosthodontics: assessment of accuracy and repeatability in answer generation. J Prosthet Dent 2024; 131 (04) 659.e1-659.e6
    Suche in Google Scholar
  • 15 Mohammad-Rahimi H, Ourang SA, Pourhoseingholi MA, Dianat O, Dummer PMH, Nosrat A. Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics. Int Endod J 2024; 57 (03) 305-314
    PubMedSuche in Google Scholar
  • 16 Koga S, Martin NB, Dickson DW. Evaluating the performance of large language models: ChatGPT and Google Bard in generating differential diagnoses in clinicopathological conferences of neurodegenerative disorders. Brain Pathol 2024; 34 (03) e13207
    PubMedSuche in Google Scholar
  • 17 Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. Journal of Applied Learning and Teaching 2023; 6 (01) 364-389
    Suche in Google Scholar
  • 18 Lu Q, Qiu B, Ding L, Zhang K, Kocmi T, Tao D. Error analysis prompting enables human-like translation evaluation in large language models. arXiv 2023 (ePub ahead of print) https://doi.org/10.48550/arXiv.2303.13809
    Crossref
  • 19 Xu L, Sanders L, Li K, Chow JCL. Chatbot for health care and oncology applications using artificial intelligence and machine learning: systematic review. JMIR Cancer 2021; 7 (04) e27850
    PubMedSuche in Google Scholar
  • 20 Walker HL, Ghani S, Kuemmerli C. et al. Reliability of medical information provided by ChatGPT: assessment against clinical guidelines and patient information quality instrument. J Med Internet Res 2023; 25: e47479
    CrossrefPubMedSuche in Google Scholar
  • 21 Tsvetanov T. Dental Management of the Medically Compromised Patients. LAP LAMBERT Academic Publishing; 2016
    Suche in Google Scholar
  • 22 Othman Z, Abd Yusof NF, Malek MHA, Xi MLZXZ, Rohaizad AN, Amin MF. AI-powered dental consultation chatbot: enhancing public healthcare accessibility and awareness. J Advanced Research Applied Sciences Engineering Technology 2024; 14-27
    Suche in Google Scholar
  • 23 Thorat V, Rao P, Joshi N, Talreja P, Shetty AR. Role of artificial intelligence (AI) in patient education and communication in dentistry. Cureus 2024; 16 (05) e59799
    PubMedSuche in Google Scholar
  • 24 Coniglione F, Luciani F, Papa E, Leggeri A, Condo R, Agrestini C. Guidelines for the management of pregnant patients in dentistry. Ajmhs 2023; 63: 1-7
    Suche in Google Scholar
  • 25 Zhou X, Zhong Y, Pan Z, Zhang J, Pan J. Physiology of pregnancy and oral local anesthesia considerations. PeerJ 2023; 11: e15585
    PubMedSuche in Google Scholar
  • 26 Melo dos Santos G. Adaptive Human-Chatbot Interactions: Contextual Factors, Variability Design and Levels of Automation. 2023
  • 27 Sanmarchi F, Bucci A, Nuzzolese AG. et al. A step-by-step researcher's guide to the use of an AI-based transformer in epidemiology: an exploratory analysis of ChatGPT using the STROBE checklist for observational studies. J Public Health (Berl) 2023; 32 (09) 1-36
    Suche in Google Scholar
  • 28 Roganović J, Radenković M. Ethical Use of Artificial Intelligence in Dentistry, Ethics-Scientific Research, Ethical Issues, Artificial Intelligence and Education: Scientific Research, Ethical Issues, Artificial Intelligence and Education. 2023:83
  • 29 Rahim A, Khatoon R, Khan TA. et al. Artificial intelligence-powered dentistry: Probing the potential, challenges, and ethicality of artificial intelligence in dentistry. Digit Health 2024; 10: 20 552076241291345
    Suche in Google Scholar
  • 30 Roganović J, Radenković M, Miličić B. Responsible use of artificial intelligence in dentistry: survey on dentists' and final-year undergraduates' perspectives. Healthcare 2023; 11 (10) 1480
    CrossrefPubMedSuche in Google Scholar