The Use of Artificial Intelligence in Automation in the Fields of Gynaecology and Obstetrics – an Assessment of the State of Play

Abstract

The long-awaited progress in digitalisation is generating huge amounts of medical data every day, and manual analysis and targeted, patient-oriented evaluation of this data is becoming increasingly difficult or even infeasible. This state of affairs and the associated, increasingly complex requirements for individualised precision medicine underline the need for modern software solutions and algorithms across the entire healthcare system. The utilisation of state-of-the-art equipment and techniques in almost all areas of medicine over the past few years has now indeed enabled automation processes to enter – at least in part – into routine clinical practice. Such systems utilise a wide variety of artificial intelligence (AI) techniques, the majority of which have been developed to optimise medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection and classification and, as an emerging field of research, radiogenomics. Tasks handled by AI are completed significantly faster and more precisely, clearly demonstrated by now in the annual findings of the ImageNet Large-Scale Visual Recognition Challenge (ILSVCR), first conducted in 2015, with error rates well below those of humans. This review article will discuss the potential capabilities and currently available applications of AI in gynaecological-obstetric diagnostics. The article will focus, in particular, on automated techniques in prenatal sonographic diagnostics.

Key words

pregnancy - sonography - gynecology - malformation

Publikationsverlauf

Eingereicht: 22. April 2021

Angenommen: 01. Juni 2021

Artikel online veröffentlicht:
04. November 2021

• References/Literatur

• 1 Drukker L, Droste R, Chatelain P. et al. Expected-value bias in routine third-trimester growth scans. Ultrasound Obstet Gynecol 2020; 55: 375-382
• 2 Deng J, Dong W, Socher R. et al. ImageNet: A large-scale hierarchical image database. Paper presented at: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 20 – 25 June 2009. 2009
• 3 Russakovsky O, Deng J, Su H. et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis 2015; 115: 211-252
• 4 Turing AM. I – Computing Machinery and Intelligence. Mind 1950; LIX: 433-460
• 5 Deloitte. State of AI in the enterprise – 3rd ed. Deloitte, 2020. Accessed September 30, 2021 at: http://www2.deloitte.com/content/dam/Deloitte/de/Documents/technology-media-telecommunications/DELO-6418_State of AI 2020_KS4.pdf
• 6 Anthes E. Alexa, do I have COVID-19?. Nature 2020; 586: 22-25
• 7 Huang Z, Epps J, Joachim D. Investigation of Speech Landmark Patterns for Depression Detection. IEEE Transactions on Affective Computing 2019;
• 8 Bodalal Z, Trebeschi S, Nguyen-Kim TDL. et al. Radiogenomics: bridging imaging and genomics. Abdom Radiol (NY) 2019; 44: 1960-1984
• 9 Allen B, Dreyer K, McGinty GB. Integrating Artificial Intelligence Into Radiologic Practice: A Look to the Future. J Am Coll Radiol 2020; 17: 280-283
• 10 Purohit K. Growing Interest in Radiology Despite AI Fears. Acad Radiol 2019; 26: e75
• 11 Richardson ML, Garwood ER, Lee Y. et al. Noninterpretive Uses of Artificial Intelligence in Radiology. Acad Radiol 2021; 28: 1225-1235
• 12 Bennani-Baiti B, Baltzer PAT. Künstliche Intelligenz in der Mammadiagnostik. Radiologe 2020; 60: 56-63
• 13 Chan H-P, Samala RK, Hadjiiski LM. CAD and AI for breast cancer–recent development and challenges. Br J Radiol 2019; 93: 20190580
• 14 Fujita H. AI-based computer-aided diagnosis (AI-CAD): the latest review to read first. Radiol Phys Technol 2020; 13: 6-19
• 15 McKinney SM, Sieniek M, Godbole V. et al. International evaluation of an AI system for breast cancer screening. Nature 2020; 577: 89-94
• 16 Rodriguez-Ruiz A, Lång K, Gubern-Merida A. et al. Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study. Eur Radiol 2019; 29: 4825-4832
• 17 OʼConnell AM, Bartolotta TV, Orlando A. et al. Diagnostic Performance of An Artificial Intelligence System in Breast Ultrasound. J Ultrasound Med 2021;
• 18 Cho BJ, Choi YJ, Lee MJ. et al. Classification of cervical neoplasms on colposcopic photography using deep learning. Sci Rep 2020; 10: 13652
• 19 Shanthi PB, Faruqi F, Hareesha KS. et al. Deep Convolution Neural Network for Malignancy Detection and Classification in Microscopic Uterine Cervix Cell Images. Asian Pac J Cancer Prev 2019; 20: 3447-3456
• 20 Försch S, Klauschen F, Hufnagl P. et al. Künstliche Intelligenz in der Pathologie. Dtsch Arztebl 2021; 118: 199-204
• 21 Chang PJ. Moving Artificial Intelligence from Feasible to Real: Time to Drill for Gas and Build Roads. Radiology 2020; 294: 432-433
• 22 Tran D, Cooke S, Illingworth PJ. et al. Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Hum Reprod 2019; 34: 1011-1018
• 23 Zaninovic N, Rosenwaks Z. Artificial intelligence in human in vitro fertilization and embryology. Fertil Steril 2020; 114: 914-920
• 24 Bori L, Paya E, Alegre L. et al. Novel and conventional embryo parameters as input data for artificial neural networks: an artificial intelligence model applied for prediction of the implantation potential. Fertil Steril 2020; 114: 1232-1241
• 25 DEGUM. Pressemitteilungen. DEGUM, 2017. Updated 29.11.2017. Accessed September 30, 2021 at: http://www.degum.de/aktuelles/presse-medien/pressemitteilungen/im-detail/news/zu-viele-kindliche-fehlbildungen-bleiben-unentdeckt.html
• 26 Murugesu S, Galazis N, Jones BP. et al. Evaluating the use of telemedicine in gynaecological practice: a systematic review. BMJ Open 2020; 10: e039457
• 27 Benacerraf BR, Minton KK, Benson CB. et al. Proceedings: Beyond Ultrasound First Forum on Improving the Quality of Ultrasound Imaging in Obstetrics and Gynecology. J Ultrasound Med 2018; 37: 7-18
• 28 Timmerman D, Verrelst H, Bourne TH. et al. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol 1999; 13: 17-25
• 29 Froyman W, Timmerman D. Methods of Assessing Ovarian Masses: International Ovarian Tumor Analysis Approach. Obstet Gynecol Clin North Am 2019; 46: 625-641
• 30 Van Calster B, Van Hoorde K, Valentin L. et al. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ 2014; 349: g5920
• 31 Vázquez-Manjarrez SE, Rico-Rodriguez OC, Guzman-Martinez N. et al. Imaging and diagnostic approach of the adnexal mass: what the oncologist should know. Chin Clin Oncol 2020; 9: 69
• 32 Andreotti RF, Timmerman D, Strachowski LM. et al. O-RADS US Risk Stratification and Management System: A Consensus Guideline from the ACR Ovarian-Adnexal Reporting and Data System Committee. Radiology 2020; 294: 168-185
• 33 Christiansen F, Epstein EL, Smedberg E. et al. Ultrasound image analysis using deep neural networks for discriminating between benign and malignant ovarian tumors: comparison with expert subjective assessment. Ultrasound Obstet Gynecol 2021; 57: 155-163
• 34 Acharya UR, Mookiah MR, Vinitha Sree S. et al. Evolutionary algorithm-based classifier parameter tuning for automatic ovarian cancer tissue characterization and classification. Ultraschall Med 2014; 35: 237-245
• 35 Akazawa M, Hashimoto K. Artificial Intelligence in Ovarian Cancer Diagnosis. Anticancer Res 2020; 40: 4795-4800
• 36 Aramendia-Vidaurreta V, Cabeza R, Villanueva A. et al. Ultrasound Image Discrimination between Benign and Malignant Adnexal Masses Based on a Neural Network Approach. Ultrasound Med Biol 2016; 42: 742-752
• 37 Khazendar S, Sayasneh A, Al-Assam H. et al. Automated characterisation of ultrasound images of ovarian tumours: the diagnostic accuracy of a support vector machine and image processing with a local binary pattern operator. Facts Views Vis Obgyn 2015; 7: 7-15
• 38 Zhou J, Zeng ZY, Li L. Progress of Artificial Intelligence in Gynecological Malignant Tumors. Cancer Manag Res 2020; 12: 12823-12840
• 39 Al-Karawi D, Al-Assam H, Du H. et al. An Evaluation of the Effectiveness of Image-based Texture Features Extracted from Static B-mode Ultrasound Images in Distinguishing between Benign and Malignant Ovarian Masses. Ultrason Imaging 2021; 43: 124-138
• 40 Bakker MK, Bergman JEH, Krikov S. et al. Prenatal diagnosis and prevalence of critical congenital heart defects: an international retrospective cohort study. BMJ Open 2019; 9: e028139
• 41 van Nisselrooij AEL, Teunissen AKK, Clur SA. et al. Why are congenital heart defects being missed?. Ultrasound Obstet Gynecol 2020; 55: 747-757
• 42 Knackstedt C, Bekkers SC, Schummers G. et al. Fully Automated Versus Standard Tracking of Left Ventricular Ejection Fraction and Longitudinal Strain: The FAST-EFs Multicenter Study. J Am Coll Cardiol 2015; 66: 1456-1466
• 43 Tsang W, Salgo IS, Medvedofsky D. et al. Transthoracic 3D Echocardiographic Left Heart Chamber Quantification Using an Automated Adaptive Analytics Algorithm. JACC Cardiovasc Imaging 2016; 9: 769-782
• 44 Kusunose K. Steps to use artificial intelligence in echocardiography. J Echocardiogr 2021; 19: 21-27
• 45 Zhang J, Gajjala S, Agrawal P. et al. Fully Automated Echocardiogram Interpretation in Clinical Practice. Circulation 2018; 138: 1623-1635
• 46 Harari YN. Homo sapiens verliert die Kontrolle. Die Große Entkopplung. In: Homo Deus – Eine Geschichte von Morgen. 16. Aufl.. München: C. H. Beck; 2020
• 47 Gandhi S, Mosleh W, Shen J. et al. Automation, machine learning, and artificial intelligence in echocardiography: A brave new world. Echocardiography 2018; 35: 1402-1418
• 48 Arnaout R, Curran L, Zhao Y. et al. Expert-level prenatal detection of complex congenital heart disease from screening ultrasound using deep learning. medRxiv 2020;
• 49 Le TK, Truong V, Nguyen-Vo TH. et al. Application of machine learning in screening of congenital heart diseases using fetal echocardiography. J Am Coll Cardiol 2020; 75: 648
• 50 Dong J, Liu S, Liao Y. et al. A Generic Quality Control Framework for Fetal Ultrasound Cardiac Four-Chamber Planes. IEEE J Biomed Health Inform 2020; 24: 931-942
• 51 Hinton GE. To recognize shapes, first learn to generate images. Prog Brain Res 2007; 165: 535-547
• 52 Voelker R. Cardiac Ultrasound Uses Artificial Intelligence to Produce Images. JAMA 2020; 323: 1034
• 53 Yeo L, Romero R. Optical ultrasound simulation-based training in obstetric sonography. J Matern Fetal Neonatal Med 2020;
• 54 Steinhard J, Dammeme Debbih A, Laser KT. et al. Randomised controlled study on the use of systematic simulator-based training (OPUS Fetal Heart Trainer) for learning the standard heart planes in fetal echocardiography. Ultrasound Obstet Gynecol 2019; 54 (S1): 28-29
• 55 Day TG, Kainz B, Hajnal J. et al. Artificial intelligence, fetal echocardiography, and congenital heart disease. Prenat Diagn 2021; 41: 733-742
• 56 Garcia-Canadilla P, Sanchez-Martinez S, Crispi F. et al. Machine Learning in Fetal Cardiology: What to Expect. Fetal Diagn Ther 2020; 47: 363-372
• 57 Meiburger KM, Acharya UR, Molinari F. Automated localization and segmentation techniques for B-mode ultrasound images: A review. Comput Biol Med 2018; 92: 210-235
• 58 Rawat V, Jain A, Shrimali V. Automated Techniques for the Interpretation of Fetal Abnormalities: A Review. Appl Bionics Biomech 2018; 2018: 6452050
• 59 Yeo L, Luewan S, Romero R. Fetal Intelligent Navigation Echocardiography (FINE) Detects 98 % of Congenital Heart Disease. J Ultrasound Med 2018; 37: 2577-2593
• 60 Gembicki M, Hartge DR, Dracopoulos C. et al. Semiautomatic Fetal Intelligent Navigation Echocardiography Has the Potential to Aid Cardiac Evaluations Even in Less Experienced Hands. J Ultrasound Med 2020; 39: 301-309
• 61 Weichert J, Weichert A. A “holistic” sonographic view on congenital heart disease: How automatic reconstruction using fetal intelligent navigation echocardiography eases unveiling of abnormal cardiac anatomy part II-Left heart anomalies. Echocardiography 2021; 38: 777-789
• 62 DeVore GR, Klas B, Satou G. et al. Longitudinal Annular Systolic Displacement Compared to Global Strain in Normal Fetal Hearts and Those With Cardiac Abnormalities. J Ultrasound Med 2018; 37: 1159-1171
• 63 DeVore GR, Klas B, Satou G. et al. 24-segment sphericity index: a new technique to evaluate fetal cardiac diastolic shape. Ultrasound Obstet Gynecol 2018; 51: 650-658
• 64 DeVore GR, Polanco B, Satou G. et al. Two-Dimensional Speckle Tracking of the Fetal Heart: A Practical Step-by-Step Approach for the Fetal Sonologist. J Ultrasound Med 2016; 35: 1765-1781
• 65 Lee M, Won H. Novel technique for measurement of fetal right myocardial performance index using synchronised images of right ventricular inflow and outflow. Ultrasound Obstet Gynecol 2019; 54 (S1): 178-179
• 66 Leung V, Avnet H, Henry A. et al. Automation of the Fetal Right Myocardial Performance Index to Optimise Repeatability. Fetal Diagn Ther 2018; 44: 28-35
• 67 Rizzo G, Aiello E, Pietrolucci ME. et al. The feasibility of using 5D CNS software in obtaining standard fetal head measurements from volumes acquired by three-dimensional ultrasonography: comparison with two-dimensional ultrasound. J Matern Fetal Neonatal Med 2016; 29: 2217-2222
• 68 Welp A, Gembicki M, Rody A. et al. Validation of a semiautomated volumetric approach for fetal neurosonography using 5DCNS+ in clinical data from > 1100 consecutive pregnancies. Childs Nerv Syst 2020; 36: 2989-2995
• 69 Pluym ID, Afshar Y, Holliman K. et al. Accuracy of three-dimensional automated ultrasound imaging of biometric measurements of the fetal brain. Ultrasound Obstet Gynecol 2021; 57: 798-803
• 70 Ambroise Grandjean G, Hossu G, Bertholdt C. et al. Artificial intelligence assistance for fetal head biometry: Assessment of automated measurement software. Diagn Interv Imaging 2018; 99: 709-716
• 71 Huang R, Xie W, Alison Noble J. VP-Nets: Efficient automatic localization of key brain structures in 3D fetal neurosonography. Med Image Anal 2018; 47: 127-139
• 72 Xie HN, Wang N, He M. et al. Using deep-learning algorithms to classify fetal brain ultrasound images as normal or abnormal. Ultrasound Obstet Gynecol 2020; 56: 579-587
• 73 Cerrolaza JJ, Li Y, Biffi C. et al. Fetal Skull Reconstruction via Deep Convolutional Autoencoders. Annu Int Conf IEEE Eng Med Biol Soc 2018; 2018: 887-890
• 74 Ghesu FC, Georgescu B, Grbic S. et al. Towards intelligent robust detection of anatomical structures in incomplete volumetric data. Med Image Anal 2018; 48: 203-213
• 75 Cai Y, Droste R, Sharma H. et al. Spatio-temporal visual attention modelling of standard biometry plane-finding navigation. Medical Image Analysis 2020; 65: 101762
• 76 Baumgartner CF, Kamnitsas K, Matthew J. et al. SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound. IEEE Trans Med Imaging 2017; 36: 2204-2215
• 77 Yaqub M, Kelly B, Noble JA. et al. An AI system to support sonologists during fetal ultrasound anomaly screening. Ultrasound Obstet Gynecol 2018; 52 (S1): 9-10
• 78 Yaqub M, Sleep N, Syme S. et al. ScanNav^® audit: an AI-powered screening assistant for fetal anatomical ultrasound. Am J Obstet Gynecol 2021; 224 (Suppl.) S312
• 79 Sharma H, Drukker L, Chatelain P. et al. Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos. Med Image Anal 2021; 69: 101973
• 80 Droste R, Drukker L, Papageorghiou AT. et al. Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound. Med Image Comput Comput Assist Interv 2020; 12263: 583-592
• 81 Alsharid M, Sharma H, Drukker L. et al. Captioning Ultrasound Images Automatically. In: Shen D. et al., eds. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. (Lecture Notes in Computer Science, vol 11767; ). Cham: Springer; 2019
• 82 Lee W, Deter RL, Ebersole JD. et al. Birth weight prediction by three-dimensional ultrasonography: fractional limb volume. J Ultrasound Med 2001; 20: 1283-1292
• 83 Corrêa VM, Araujo Júnior E, Braga A. et al. Prediction of birth weight in twin pregnancies using fractional limb volumes by three-dimensional ultrasonography. J Matern Fetal Neonatal Med 2020; 33: 3652-3657
• 84 Gembicki M, Offerman DR, Weichert J. Semiautomatic Assessment of Fetal Fractional Limb Volume for Weight Prediction in Clinical Praxis: How Does It Perform in Routine Use?. J Ultrasound Med 2021;
• 85 Mack LM, Kim SY, Lee S. et al. Automated Fractional Limb Volume Measurements Improve the Precision of Birth Weight Predictions in Late Third-Trimester Fetuses. J Ultrasound Med 2017; 36: 1649-1655
• 86 Youssef A, Salsi G, Montaguti E. et al. Automated Measurement of the Angle of Progression in Labor: A Feasibility and Reliability Study. Fetal Diagn Ther 2017; 41: 293-299
• 87 Brocklehurst P, Field D, Greene K. et al. Computerised interpretation of fetal heart rate during labour (INFANT): a randomised controlled trial. Lancet 2017; 389: 1719-1729
• 88 Keith R. The INFANT study-a flawed design foreseen. Lancet 2017; 389: 1697-1698
• 89 Silver RM. Computerising the intrapartum continuous cardiotocography does not add to its predictive value: FOR: Computer analysis does not add to intrapartum continuous cardiotocography predictive value. BJOG 2019; 126: 1363
• 90 Gyllencreutz E, Lu K, Lindecrantz K. et al. Validation of a computerized algorithm to quantify fetal heart rate deceleration area. Acta Obstet Gynecol Scand 2018; 97: 1137-1147
• 91 Fung R, Villar J, Dashti A. et al. International Fetal and Newborn Growth Consortium for the 21st Century (INTERGROWTH-21st). Achieving accurate estimates of fetal gestational age and personalised predictions of fetal growth based on data from an international prospective cohort study: a population-based machine learning study. Lancet Digit Health 2020; 2: e368-e375
• 92 Lee KS, Ahn KH. Application of Artificial Intelligence in Early Diagnosis of Spontaneous Preterm Labor and Birth. Diagnostics (Basel) 2020; 10: 733
• 93 Maassen O, Fritsch S, Palm J. et al. Future Medical Artificial Intelligence Application Requirements and Expectations of Physicians in German University Hospitals: Web-Based Survey. J Med Internet Res 2021; 23: e26646
• 94 Littmann M, Selig K, Cohen-Lavi L. et al. Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nature Machine Intelligence 2020; 2: 18-24
• 95 Norgeot B, Glicksberg BS, Butte AJ. A call for deep-learning healthcare. Nat Med 2019; 25: 14-15
• 96 Borck C. Communicating the Modern Body: Fritz Kahnʼs Popular Images of Human Physiology as an Industrialized World. Canadian Journal of Communication 2007; 32: 495-520
• 97 Jachertz N. Populärmedizin: Der Mensch ist eine Maschine, die vom Menschen bedient wird. Dtsch Arztebl 2010; 107: A-391-393
• 98 Frey CB, Osborne MA. The future of employment: How susceptible are jobs to computerisation?. Technological Forecasting and Social Change 2017; 114: 254-280
• 99 Gartner H, Stüber H. Strukturwandel am Arbeitsmarkt seit den 70er Jahren: Arbeitsplatzverluste werden durch neue Arbeitsplätze immer wieder ausgeglichen. 16.7.2019. Nürnberg: Institut für Arbeitsmarkt- und Berufsforschung; 2019
• 100 Bartoli A, Quarello E, Voznyuk I. et al. Intelligence artificielle et imagerie en médecine fœtale: de quoi parle-t-on? [Artificial intelligence and fetal imaging: What are we talking about?]. Gynecol Obstet Fertil Senol 2019; 47: 765-768
• 101 Allen jr. B, Seltzer SE, Langlotz CP. et al. A Road Map for Translational Research on Artificial Intelligence in Medical Imaging: From the 2018 National Institutes of Health/RSNA/ACR/The Academy Workshop. J Am Coll Radiol 2019; 16 (9 Pt A): 1179-1189
• 102 Langlotz CP, Allen B, Erickson BJ. et al. A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781-791
• 103 Tolsgaard MG, Svendsen MBS, Thybo JK. et al. Does artificial intelligence for classifying ultrasound imaging generalize between different populations and contexts?. Ultrasound Obstet Gynecol 2021; 57: 342-343
• 104 Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J 2019; 6: 94-98