Yearbook of Medical Informatics, Table of Contents Yearb Med Inform 2010; 19(01): 34-42DOI: 10.1055/s-0038-1638686 Original Article Schattauer GmbH Imaging Informatics: Toward Capturing and Processing Semantic Information in Radiology Images D. L. Rubin 1 Department of Radiology, Stanford University, Stanford, CA,USA , S. Napel 1 Department of Radiology, Stanford University, Stanford, CA,USA › Author Affiliations Recommend Article Abstract Full Text PDF Download Keywords KeywordsImaging informatics ontologies - content based image retrieval - structured reporting References References 1 Ratib O. Imaging informatics: from image management to image navigation. Yearb Med Inform 2009; 167-72. 2 Branstetter BF. Basics of imaging informatics. Part 1. Radiology 2007; Jun; 243 (03) 656-67. 3 Armato 3rd SG, van Ginneken B. Anniversary paper: image processing and manipulation through the pages of Medical Physics. Med Phys 2008; Oct; 35 (10) 4488-500. 4 Andriole KP, Morin RL. Transforming medical imaging: the first SCAR TRIP conference a position paper from the SCAR TRIP subcommittee of the SCAR research and development committee. J Digit Imaging 2006; Mar; 19 (01) 6-16. 5 Mendelson DS, Bak PR, Menschik E, Siegel E. Informatics in radiology: image exchange: IHE and the evolution of image sharing. Radiographics 2008; Nov-Dec; 28 (07) 1817-33. 6 Bashyam V, Hsu W, Watt E, Bui AA, Kangarloo H, Taira RK. Problem-centric organization and visualization of patient imaging and clinical data. Radiographics 2009; Mar-Apr; 29 (02) 331-43. 7 Reiner BI, Siegel EL. The clinical imperative of medical imaging informatics. J Digit Imaging 2009; Aug; 22 (04) 345-7. 8 Thrall JH. Reinventing radiology in the digital age. Part II. New directions and new stakeholder value. Radiology 2005; Oct; 237 (01) 15-8. 9 Doi K. Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 2007; Jun-Jul; 31 (4-5): 198-211. 10 Sobel JL, Pearson ML, Gross K, Desmond KA, Harrison ER, Rubenstein LV. et al. Information content and clarity of radiologists’ reports for chest radiography. Acad Radiol 1996; Sep; 03 (09) 709-17. 11 Rubin DL. Informatics methods to enable patientcentered radiology. Acad Radiol 2009; May; 16 (05) 524-34. 12 Bodenreider O. Biomedical ontologies in action: role in knowledge management, data integration and decision support. Yearb Med Inform 2008; 67-79. 13 Cimino JJ. Review paper: coding systems in health care. Methods Inf Med 1996; Dec; 35 (4-5): 273-84. 14 Baker JA, Kornguth PJ, Lo JY, Williford ME, Floyd Jr CE. Breast cancer: prediction with artificial neural network based on BI-RADS standard-ized lexicon. Radiology 1995; Sep; 196 (03) 817-22. 15 Burnside ES, Davis J, Chhatwal J, Alagoz O, Lindstrom MJ, Geller BM. et al. Probabilistic computer model developed from clinical data in national mammography database format to classify mammographic f indings. Radiology 2009; Jun; 251 (03) 663-72. 16 Channin DS, Mongkolwat P, Kleper V, Rubin DL. Computing human image annotation. Conf Proc IEEE Eng Med Biol Soc 2009; 01: 7065-8. 17 Kahn Jr. CE, Langlotz CP, Burnside ES, Carrino JA, Channin DS, Hovsepian DM. et al. Toward best practices in radiology reporting. Radiology 2009; Sep; 252 (03) 852-6. 18 Sistrom CL, Langlotz CP. A framework for improving radiology reporting. J Am Coll Radiol 2005; Feb; 02 (02) 159-67. 19 D’Orsi CJ, Newell MS. BI-RADS decoded: detailed guidance on potentially confusing issues. Radiol Clin North Am 2007; Sep; 45 (05) 751-63 v. 20 Hansell DM, Bankier AA, MacMahon H, McLoud TC, Muller NL, Remy J. Fleischner Society: glossary of terms for thoracic imaging. Radiology 2008; Mar; 246 (03) 697-722. 21 Reporting terminology for brain arteriovenous malformation clinical and radiographic features for use in clinical trials. Stroke 2001; Jun; 32 (06) 1430-42. 22 Appel B. Nomenclature and classification of lumbar disc pathology. Neuroradiology 2001; Dec; 43 (12) 1124-5. 23 Goldberg SN, Grassi CJ, Cardella JF, Charboneau JW, Dodd 3rd GD, Dupuy DE. et al. Image-guided tumor ablation: standardization of terminology and reporting criteria. J Vasc Interv Radiol 2009; Jul; 20 (Suppl. 07) S377-90. 24 Brown DB, Gould JE, Gervais DA, Goldberg SN, Murthy R, Millward SF. et al. Transcatheter Therapy for Hepatic Malignancy: Standardization of Terminology and Reporting Criteria (Reprinted from J Vasc Interv Radiol, vol 18, pg 1469-1478, 2007). Journal of Vascular and Interventional Radiology 2009; Jul; 20 (07) S425-S34. 25 Zalis ME, Barish MA, Choi JR, Dachman AH, Fenlon HM, Ferrucci JT. et al. CT colonography reporting and data system: a consensus proposal. Radiology 2005; Jul; 236 (01) 3-9. 26 Horvath E, Majlis S, Rossi R, Franco C, Niedmann JP, Castro A. et al. An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management. J Clin Endocrinol Metab 2009; May; 94 (05) 1748-51. 27 Bosniak MA. The current radiological approach to renal cysts. Radiology 1986; Jan; 158 (01) 1-10. 28 Sahani DV, Kadavigere R, Saokar A, Fernandez-del Castillo C, Brugge WR, Hahn PF. Cystic pancreatic lesions: a simple imaging-based classification system for guiding management. Radiographics 2005; Nov-Dec; 25 (06) 1471-84. 29 Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM. et al. Gene ontology: tool for the unification of biology. The Gene Ontology Consortium Nat Genet 2000; May; 25 (01) 25-9. 30 Robinson PN, Kohler S, Bauer S, Seelow D, Horn D, Mundlos S. The Human Phenotype Ontology: a tool for annotating and analyzing human hereditary disease. Am J Hum Genet 2008; Nov; 83 (05) 610-5. 31 Rosse C, Mejino Jr JL. A reference ontology for biomedical informatics: the Foundational Model of Anatomy. J Biomed Inform 2003; Dec; 36 (06) 478-500. 32 Rubin DL. Creating and curating a terminology for radiology: ontology modeling and analysis. J Digit Imaging 2008; Dec; 21 (04) 355-62. 33 Langlotz CP. RadLex: a new method for indexing online educational materials. Radiographics 2006; Nov-Dec; 26 (06) 1595-7. 34 Kundu S, Itkin M, Gervais DA, Krishnamurthy VN, Wallace MJ, Cardella JF. et al. The IR RadLex project: an interventional radiology lexicon—a collaborative project of the Radiological Society of North America and the Society of Interventional Radiology. J Vasc Interv Radiol 2009; Apr; 20 (04) 433-5. 35 Marwede D, Daumke P, Marko K, Lobsien D, Schulz S, Kahn T. [RadLex German version: a radiological lexicon for indexing image and report information]. Rofo 2009; Jan; 181 (01) 38-44. 36 Mejino JL, Rubin DL, Brinkley JF. FMA-RadLex: An application ontology of radiological anatomy derived from the foundational model of anatomy reference ontology. AMIA Annu Symp Proc 2008; 465-9. 37 Kahn CE, Rubin DL. Automated Semantic Indexing of Figure Captions to Improve Radiology Image Retrieval. J Am Med Inform Assoc 2009; May-Jun; 16 (03) 380-6. 38 Marwede D, Schulz T, Kahn T. Indexing thoracic CT reports using a preliminary version of a standardized radiological lexicon (RadLex). J Digit Imaging 2008; Dec; 21 (04) 363-70. 39 Rubin DL, Rodriguez C, Shah P, Beaulieu C. iPad: Semantic annotation and markup of radiological images. AMIA Annu Symp Proc 2008; 626-30. 40 Sistrom CL, Dreyer KJ, Dang PP, Weilburg JB, Boland GW, Rosenthal DI. et al. Recommendations for additional imaging in radiology reports: multifactorial analysis of 5.9 million examinations. Radiology 2009; Nov; 253 (02) 453-61. 41 Cheng LT, Zheng J, Savova GK, Erickson BJ. Discerning Tumor Status from Unstructured MRI Reports-Completeness of Information in Existing Reports and Utility of Automated Natural Language Processing. J Digit Imaging. 2009 May 30. 42 Taira RK, Soderland SG, Jakobovits RM. Automatic structuring of radiology free-text reports. Radiographics 2001; Jan-Feb; 21 (01) 237-45. 43 Fujii H, Yamagishi H, Ando Y, Tsukamoto N, Kawaguchi O, Kasamatsu T. et al. Structuring of free-text diagnostic report. Stud Health Technol Inform 2007; 129 (Pt 1): 669-73. 44 Kushner DC, Lucey LL. Diagnostic radiology reporting and communication: the ACR guideline. J Am Coll Radiol 2005; Jan; 02 (01) 15-21. 45 Reiner BI, Knight N, Siegel EL. Radiology reporting, past, present, and future: the radiologist’s perspective. J Am Coll Radiol 2007; May; 04 (05) 313-9. 46 Hussein R, Engelmann U, Schroeter A, Meinzer HP. DICOM structured reporting: Part 1. Overview and characteristics. Radiographics 2004; MayJun; 24 (03) 891-6. 47 Kamauu AW, DuVall SL, Liimatta AP, Wiggins 3rd RH, Avrin DE. Translating the IHE Teaching File and Clinical Trial Export (TCE) profile document templates into functional DICOM structured report objects. J Digit Imaging 2008; Dec; 21 (04) 390-407. 48 Naik SS, Hanbidge A, Wilson SR. Radiology reports: examining radiologist and clinician preferences regarding style and content. AJR Am J Roentgenol 2001; Mar; 176 (03) 591-8. 49 Houssami N, Boyages J, Stuart K, Brennan M. Quality of breast imaging reports falls short of recommended standards. Breast 2007; Jun; 16 (03) 271-9. 50 Reiner B, Siegel E. Radiology reporting: returning to our image-centric roots. AJR Am J Roentgenol 2006; Nov; 187 (05) 1151-5. 51 Rosenthal DF, Bos JM, Sokolowski RA, Mayo JB, Quigley KA, Powell RA. et al. A voice-enabled, structured medical reporting system. J Am Med Inform Assoc 1997; Nov-Dec; 04 (06) 436-41. 52 Radiological Society of North America. CT Lung Nodule Structured Reporting Template. http://reportingwikirsnaorg/indexphp?title=Templates (Accessed 12/28/09). 53 Douglas PS, Hendel RC, Cummings JE, Dent JM, Hodgson JM, Hoffmann U. et al. ACCF/ACR/ AHA/ASE/ASNC/HRS/NASCI/RSNA/SAIP/ SCAI/SCCT/SCMR 2008 Health Policy Statement on Structured Reporting in Cardiovascular Imaging. J Am Coll Cardiol 2009; Jan 6; 53 (01) 76-90. 54 Rubin DL, Supekar K, Mongkolwat P, Kleper V, Channin DS. Annotation and Image Markup: Accessing and Interoperating with the Semantic Content in Medical Imaging. IEEE Intelligent Systems 2009; Jan-Feb; 24 (01) 57-65. 55 Channin DS, Mongkolwat P, Kleper V, Sepukar K, Rubin DL. The caBIG Annotation and Image Markup Project. J Digit Imaging. 2009 Mar 18. 56 Carneiro G, Chan AB, Moreno PJ, Vasconcelos N. Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Intell 2007; Mar; 29 (03) 394-410. 57 Yu F, Ip HH. Semantic content analysis and annotation of histological images. Comput Biol Med 2008; Jun; 38 (06) 635-49. 58 Mechouche A, Golbreich C, Morandi X, Gibaud B. Ontology-based annotation of brain MRI images. AMIA Annu Symp Proc 2008; 460-4. 59 Levy MA, Rubin DL. Tool support to enable evaluation of the clinical response to treatment. AMIA Annu Symp Proc 2008; 399-403. 60 Kahn Jr. CE, Rubin DL. Automated semantic indexing of figure captions to improve radiology image retrieval. JAm Med InformAssoc 2009; May-Jun; 16 (03) 380-6. 61 Hudson DL, Cohen ME. Multidimensional medical decision making. Conf Proc IEEE Eng Med Biol Soc 2009; 01: 3405-8. 62 Datta R, Joshi D, Li J, Wang JZ. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys. 2008 Apr;40(2):-. 63 Wang JZ, Wiederhold G, Firschein O, Wei SX. Content-based image indexing and searching using Daubechies’ wavelets. Int J on Digital Libraries 1997; 01 (04) 311-28. 64 Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C. et al. Automated storage and retrieval of thin-section CT images to assist diagnosis: system description and preliminary assessment. Radiology 2003; Jul; 228 (01) 265-70. 65 Giger ML, Huo Z, Vyborny CJ, Lan L, Bonta IR, Horsch K. et al. Intelligent CAD workstation for breast imaging using similarity to known lesions and multiple visual prompt aids. In: Sonka M, Fitzpatrick JM. editors. Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series. 2002: 768-73. 66 Zheng B, Mello-Thoms C, Wang XH, Abrams GS, Sumkin JH, Chough DM. et al. Interactive computer-aided diagnosis of breast masses: computerized selection of visually similar image sets from a reference library. Acad Radiol 2007; Aug; 14 (08) 917-27. 67 Greenspan H, Pinhas AT. Medical image categorization and retrieval for PACS using the GMM-KL framework. IEEE Trans Inf Technol Biomed 2007; Mar; 11 (02) 190-202. 68 Deselaers T, Muller H, Clough P, Ney H, Lehmann TM. The CLEF 2005 automatic medical image annotation task. Int J Comput Vis 2007; Aug; 74 (01) 51-8. 69 Lehmann TM, Guld MO, Deselaers T, Keysers D, Schubert H, Spitzer K. et al. Automatic categorization of medical images for content-based retrieval and data mining. Comput Med Imaging Graph 2005; Mar-Apr; 29 (2-3): 143-55. 70 Hersh W, Muller H, Kalpathy-Cramer J. The ImageCLEFmed Medical Image Retrieval Task Test Collection. J Digit Imaging 2009; Dec; 22 (06) 648-55. 71 Rahmani R, Goldman SA, Zhang H, Cholleti SR, Fritts JE. Localized content-based image retrieval. IEEE Trans Pattern Anal Mach Intell 2008; Nov; 30 (11) 1902-12. 72 Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A. et al. Automated Retrieval of CT Images of Liver Lesions Based on Image Similarity: Method and Preliminary Results. Radiology. 2010 (in press). 73 Avni. Addressing the ImageClef 2009 Challenge Using a Patch-based Visual Words Representation %U. http://www.clef-campaign.org/2009/working_notes/avni-paperCLEF2009.pdf. Working Notes CLEF2009 2009 74 Jurie F, Triggs B. Creating Efficient Codebooks for Visual Recognition. Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1 Volume 01: IEEE Computer Society. 2005; 604-10 %@ 0-7695-2334-X-01. 75 Nowak E, Jurie F, Triggs B. Sampling strategies for bag-of-features image classification. Computer Vision Eccv 2006, Pt 4, Proceedings 2006; 3954: 490-503. 76 Qiu G. Indexing chromatic and achromatic patterns for content-based colour image retrieval. Pattern Recognition 2002; Aug; 35 (08) 1675-86. 77 Ruiz ME. Combining image features, case descriptions and UMLS concepts to improve retrieval of medical images. AMIA Annu Symp Proc 2006; 674-8. 78 Zhenyu H, Yanjie Z, Tonghui L, Jianguo Z. editors. Combining text retrieval and content-based image retrieval for searching a large-scale medical image database in an integrated RIS/PACS environment. 2009. SPIE 79 Rutman AM, Kuo MD. Radiogenomics: creating a link between molecular diagnostics and diagnostic imaging. Eur J Radiol 2009; May; 70 (02) 232-41. 80 Kann MG. Advances in translational bioinformatics: computational approaches for the hunting of disease genes. Brief Bioinfor. 2009 Dec 10. 81 Blake JA, Bult CJ. Beyond the data deluge: data integration and bio-ontologies. J Biomed Inform 2006; Jun; 39 (03) 314-20. 82 Ananiadou S, Friedman C, Tsujii J. Introduction: named entity recognition in biomedicine. J Biomed Inform 2004; Dec; 37 (06) 393-5. 83 Shah NH, Jonquet C, Chiang AP, Butte AJ, Chen R, Musen MA. Ontology-driven indexing of public datasets for translational bioinformatics. BMC Bioinformatics 2009; 10 Suppl (Suppl. 02) S1. 84 Bug W, Astahkov V, Boline J, Fennema-Notestine C, Grethe JS, Gupta A. et al. Data federation in the Biomedical Informatics Research Network: tools for semantic annotation and query of distributed multiscale brain data. AMIA Annu Symp Proc 2008; 1220. 85 French L, Lane S, Law T, Xu L, Pavlidis P. Application and evaluation of automated semantic annotation of gene expression experiments. Bioinformatics 2009; Jun 15; 25 (12) 1543-9. 86 Roberts A, Gaizauskas R, Hepple M, Davis N, Demetriou G, Guo Y. et al. The CLEF corpus: semantic annotation of clinical text. AMIA Annu Symp Proc 2007; 625-9. 87 Larson SD, Martone ME. Ontologies for Neuroscience: What are they and What are they Good for?. Front Neurosci 2009; May; 03 (01) 60-7. 88 Taylor CF, Field D, Sansone SA, Aerts J, Apweiler R, Ashburner M. et al. Promoting coherent minimum reporting guidelines for biological and biomedical investigations: the MIBBI project. Nat Biotechnol 2008; Aug; 26 (08) 889-96. 89 Viti F, Merelli I, Caprera A, Lazzari B, Stella A, Milanesi L. Ontology-based, Tissue MicroArray oriented, image centered tissue bank. BMC Bioinformatics 2008; 09 Suppl (Suppl. 04) S4.
