Prospective revalidation of IOTA “two-step”, “alternative two-step” and “three-step” strategies for characterization of adnexal masses – An Indian study focussing the radiology context

Abstract

Objectives: The purpose of this study was to revalidate the diagnostic performance of IOTA “two step” (Simple Descriptors and Simple Rules), “ alternative two step” (Simple Rules Risk Calculation tool / SRrisk score), and three step (two step with subjective assessment) strategies, for characterization of adnexal masses as benign or malignant, using histopathology as gold standard. Materials and Methods: This prospective, study comprised of 100 patients with newly diagnosed adnexal masses, who underwent ultrasound evaluation first by a level I and then by a level III investigator (EFSUMB criteria). Initially, the level I investigator evaluated each adnexal mass, applying IOTA “two-step” strategy and simultaneously assigned a risk category, by applying the simple rules risk score (SRrisk score ) or performing the “alternative two step” strategy. Subsequently the inconclusive masses were evaluated by the level III investigator using “real time subjective assessment”, thereby performing the third step. Following histopathology diagnosis, the performance of each strategy was evaluated using diagnostic tests. Results: The sensitivity, specificity, PPV, NPV, and diagnostic accuracy of “two-step” strategy were 87.5%, 79.2%, 89.4%, 76%, and 84.7%, respectively; those of “alternative two-step” strategy were 91.5%, 75.6%, 84.4%, 86.1% and 88%; and those of “three-step” strategy were 98.2%, 93.3%, 94.7%, 97.7% and 96%, respectively. Conclusion: All IOTA strategies showed good diagnostic performance for characterization of adnexal masses and the “three-step” strategy performed best. We believe this is the first ever prospective re-validation and comparative evaluation of all three IOTA strategies by Indian Radiologists. Since ultrasound is the primary modality for evaluation of adnexal masses, based on the good results of our study, a recommendation for henceforth standard application, of the three-step IOTA strategy in routine Radiology practice appears justified. Although, IOTA strategies been proposed and validated mainly by Gynaecologists and Oncology surgeons, based on the results of our study, this paradigm can now be made to shift back to the arena of Radiology and Radiologists, the imaging experts.

Keywords

adnexal mass - IOTA two step strategy - IOTA alternative two step strategy - IOTA Simple rules risk (SRrisk) - IOTA three step strategy

Introduction

Ovarian malignancy is known to have a poor prognosis and high mortality, primarily due to its nonspecific initial presentation, resulting in delayed diagnosis at an advanced stage of disease.[1], [2] An early and accurate diagnosis resulting in timely intervention, undoubtedly remains critical in improving survival rates of patients with ovarian malignancy. As ultrasound (US) assessment is the most widely used primary imaging modality, the necessity for a standardized evaluation protocol in this context needs no emphasis.

Over the last two decades, following the assembly of the International Ovarian Tumor Analysis group (IOTA, a European work force), a standardized ultrasound evaluation protocol for characterization and stratification of ovarian masses has evolved and received wide acceptance. The IOTA group in the year 2000 initially proposed “terms, definitions and measurements” with respect to the ultrasound description of ovarian masses.[3] The same group subsequently proposed and evolved the US imaging approach through “simple descriptors (SD)” - “simple rules (SR)” combination, as a “two-step strategy”, “simple rules risk (SRrisk)” calculator tool (using the Simple rules), for assigning a risk of malignancy as an “alternative two-step strategy” and the “subjective assessment (SA)” by an expert examiner as the “third step” in the “three-step strategy”.[1], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12]

Since the IOTA assembly initially comprised mainly of Gynaecologists, Oncology surgeons and Statisticians, their proposed definitions, rules, and strategies have been more frequently reported in Gynaecology and Oncology literature.[4], [5], [6], [7], [8], [9], [10], [11] Furthermore, majority of the validation studies on the IOTA strategies are from the IOTA group themselves and/or their associates.[7], [8], [9], [10], [11], [12], [13], [14] Validation studies from India are sparse; there are two Indian studies validating only the SR, but none have so far evaluated and reported a prospective combined analysis on all three strategies, “two-step”, “alternative two-step,” and “three-step”, proposed by the IOTA group.[15], [16] Moreover, of the two Indian studies, only one has been performed by Radiologists and the other was a study by Gynaecologists.[15], [16]

The paucity of radiology literature especially on the applicability of IOTA strategies in routine radiology practice has prompted this study, especially since imaging and its interpretation is undeniably in the realm of Radiologists. The need for standard application of IOTA rules and strategies in routine radiology practice assumes greater significance, as in majority of the clinics and hospitals, pelvic ultrasound for gynaecology and obstetrics applications is performed and/or interpreted only by Radiologists. The inculcation of these rules and strategies would also serve as a “stepping stone primer” for the evolving O-RADS, proposed by the American College of Radiology, as the O-RADS, although a more voluminous and complex system, nevertheless, draws extensively from the IOTA algorithms.

Objectives

The primary purpose of this study was to revalidate and assess the diagnostic performance of IOTA’s ultrasound based, “two-step”, “alternative two-step,” and “three-step” strategies, in a tertiary care Indian hospital radiology department, for characterization of adnexal masses as benign or malignant, using histopathology as gold standard. Secondly, if found robust, the diagnostic performance in our study would serve as a background experience for proposing the applicability of IOTA strategies in routine radiology clinical practice.

Material and Methods

Study design

This single centre, prospective, Institutional board approved, observational study comprised of 100 consecutive patients with newly diagnosed adnexal masses, recruited from a period between April, 2018 to October, 2019, who finally had a histopathological confirmation either by biopsy or by pathology examination of a surgically resected specimen. A written informed consent was obtained from each participant in accordance with institutional research ethics guidelines. The inclusion criteria were as follows: females more than 18 years of age, with at least one adnexal lesion, in which biopsy and/or laparoscopic/laparotomy management was indicated, such that the tissue would be available for histopathological evaluation. Exclusion criteria were those patients not consenting to a transvaginal ultrasound, biopsy, or surgery or those in whom ultrasound features indicated a simple follicular or corpus luteum cyst, in which biopsy/laparoscopic surgery was contraindicated. A detailed tabulation of the patients’ demographic data, clinical history, examination, and results of investigations, including those of histopathology were done.

Lexicon used in the study

Majority of the authors, including those from IOTA group, have so far used the terminology of “ovarian mass/tumor” and “adnexal mass/tumor” synonymously.[1], [2] Very recently, the American College of Radiology has recommended the terminology of “ovarian–adnexal” in their evolving O-RADS system, which however incorporates IOTA rules and strategies.[17] We have used the terminology of “adnexal mass/tumor,” as majority of our readers would be currently more familiar with this terminology and it is more inclusive (includes both ovarian and tubal masses). In our future publications, we propose to use the ACR terminology.

Technical aspects

All patients underwent sequential ultrasound (US) evaluation by two independent investigators, using a Siemens Acuson S3000 equipment. The transducers used were, a curvilinear transducer of frequency 1-6 MHz and a transvaginal transducer with a frequency of 4-9 MHz. Both the investigators were radiologists: the first investigator (level I operator), Author SP, was a radiology resident, and the second investigator (level III operator), Author SBG, was a senior radiology faculty, with more than 25 years of experience in Gynaecology ultrasound. Both the investigators fulfilled the criteria of “level I” and “level III” training and experience, respectively, as recommended by the IOTA group and the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB).[18] Patients with an adnexal mass measuring less than 10 cm were assessed with transvaginal ultrasound, while those with a size greater than 10 cm were evaluated using transabdominal ultrasound. In patients with bilateral adnexal masses, the larger, more complex (on ultrasound) lesion was included in the study.[7]

Diagnostic work-flow

Initially, the level I operator performed the two-step analysis, followed by the alternative two-step analysis. The patients categorized as “inconclusive” in the two-step analysis were re-evaluated by the level III operator to complete the third step of the three-step strategy.

The two-step strategy was performed by the level I investigator using the sequential application of IOTA SD, followed by IOTA SR. The IOTA SD consist of six descriptors (four benign and two malignant descriptors) for making an easy or “instant” diagnosis. When none of the six descriptors were applicable or both benign and malignant descriptors were present, the diagnosis was considered “non-instant.”[14] The IOTA simple descriptors are tabulated in [Table 1]. When the diagnosis was not apparent on application of IOTA SD, IOTA SR was applied. The IOTA SR are tabulated in [Table 2a] and [b].[6], [14] SR included the assessment of five benign or “B-features,” which are illustrated (from the evaluated series) in [Figure 1] and five malignant or “M-features,” which are illustrated (from the evaluated series) in [Figure 2]. When one or more M-features were present, in the absence of a B feature, the mass was characterized as malignant—Rule 1. When one or more B-features were present, in the absence of an M feature, the mass was characterized as benign—Rule 2. When none of the features were applicable or when both B and M features were present, the adnexal mass was characterized as inconclusive—Rule 3.[7], [8]

Table 1

IOTA simple descriptors (SD) (Adapted from reference 14)

Benign descriptors	Malignant descriptors
Unilocular tumor with ground glass echogenicity in premenopausal age	Tumor with ascites and atleast moderate color Doppler blood
Unilocular tumor with mixed echogenicity and acoustic shadows in premenopausal age	flow in post menopausal women
Unilocular tumor anechoic tumor with regular walls and largest diameter of less than 100 mm	Clinical: Women aged >50 years and
Unilocular tumor with regular walls	Laboratory: Serum CA-125 >100 IU/ml

Table 2a

IOTA simple rules (SR) (Adapted from reference 6)

Benign features	Malignant features
B1 Unilocular cyst	M1 Irregular solid tumor
B2 Solid components <7 mm	M2 Ascites
B3 Acoustic shadows	M3 Atleast 4 papillary projections
B4 Smooth multilocular <100 mm	M4 Irregular multilocular solid >100 mm
B5 Color flow score 0/1	M5 Color flow score 4

Table 2b

IOTA simple rules (SR) (Adapted from reference 6)

Rule	Description	Interpretation
Rule 1	If one or more B-features are present with no M-features	Benign
Rule 2	If one or more M-features are present with no B-features	Malignant
Rule 3	If combination of both B- and M-features are present	Inconclusive

Figure 1 (A-F): Are ultrasound images of different patients with adnexal masses which fulfil the criteria for IOTA benign or B-features: (A) unilocular cystic tumour, consistent with B1; (B) cystic adnexal tumour with a small solid component, less than 7 mm in size, consistent with B2; (C) smooth solid appearing adnexal tumour with acoustic shadowing from an echogenic component, consistent with B3; (D) smooth multilocular cystic adnexal tumour, <100 mm in maximum diameter, consistent with B4; (E) colour Doppler image of adnexal tumour without vascularity, colour score of 1, consistent with B5 and (F) is a tabulation of benign or B-features.

Figure 2 (A-F): Are ultrasound images of different patients with adnexal masses which fulfil the criteria for IOTA malignant or M-features: (A) irregular solid adnexal tumour, consistent with M1; (B) ascites with smooth solid adnexal tumour which is >100 mm in largest diameter, consistent with M2; (C) large adnexal tumour with multiple papillary projections (>4 in number), consistent with M3; (D) irregular multilocular adnexal tumour which is >100 mm in largest diameter, consistent with M4; (E) adnexal tumour with profuse internal vascularity which has arterial flow pattern, that is colour score of 4, consistent with M5; and (F) is a tabulation of malignant or M-features.

An integral component of the IOTA’s SR was the assessment of vascularity of the adnexal tumor, using Color Doppler. Color Doppler assessment has been recommended by IOTA to be a “subjective semi- quantitative assessment of the amount of blood flow (area and color scale) within the septa, cyst walls, or solid tumor areas without including the Doppler spectrum” and was similarly obtained in the study. A score of 1, was assigned when no blood flow could be found in the lesion; a score of 2 was assigned when only a small amount of flow could be detected; when moderate flow was present, the score assigned was as 3, and when the adnexal tumor appeared highly vascular with marked blood flow, a score of 4 was assigned.[19] Representative images of the different color flow scores (from the evaluated series), are illustrated in [Figure 3].[6], [13], [20], [21]

Figure 3 (A-D): Are colour Doppler images of different patients with adnexal masses which fulfil the criteria for IOTA colour score: (A) shows an adnexal tumour, with no flow in septae, cyst walls or solid components, consistent with colour score 1; (B) shows an adnexal tumour, with minimal flow in (septae), cyst walls (or solid components), consistent with colour score 2; (C) shows adnexal tumour, with moderate flow in (septae, cyst walls or) solid components, consistent with colour score 3; (D) shows adnexal tumour, with marked flow in (septae, cyst walls or) solid components, consistent with a colour score of 4.

Based on the results of the two-step strategy, the adnexal tumors were categorized as either benign, malignant, or inconclusive. The level I investigator then performed the” SRrisk calculation”, (using SRrisk calculation tool 15) or “alternative two-step”, (the latter, a nomenclature assigned by Hidalgo et al.) and assigned a risk category to all the evaluated tumors.[13] The IOTA’s SRrisk calculator is based on the SR, as shown in [Table 3]. According to this IOTA model, the masses were stratified into five risk categories: “very low risk,” “low risk,” “intermediate,” “elevated risk,” and “very high risk.” The masses categorized into “very low risk” and “low risk” were considered benign. In our study, the masses categorized as “intermediate” were also considered along with those classified as “elevated risk” and “very high risk” in the malignant category. This was done to take into account the fact that some of the masses in “intermediate” category have been reported to have a risk of malignancy higher than 15%.[1], [20]

Table 3

IOTA simple rules risk calculator (SRrisk) model (Adapted from reference 1)

Features	Risk stratum	Estimated risk of malignancy
>2 B-features	Very low risk	0.01 to 0.29%
Only B1 or only 2 B-features	Low risk	0.19 to 3.1%
Only 1 B-feature (other than B1)	Intermediate risk	2.4 to 15.2%
2 sub-categories (both M- and B-features): Equal no. of M- and B-features	Elevated risk	2 sub-categories: 5.6 to 78.1%
More B-features than M-features		1.3 to 28.4%
More M-features than B-features	Very high risk	42 to>99.9%

Patients who were categorized as inconclusive by the level I investigator using the two-step strategy, were re-examined by the level III investigator using Subjective Assessment (SA), to provide a more definitive diagnosis.[9], [13], [14] The investigator performed SA of the adnexal mass according to three defined parameters called as ‘A,’ ‘B,’ and ‘C,’ as per IOTA’s protocol: A was to define whether the adnexal tumor is malignant or benign or borderline; B was to define the probability of malignancy on a numerical scale as: 1 = benign, 2 = probably benign, 3 = uncertain, 4 = probably malignant, and 5 = malignant; and C was to provide a “self-presumed” histological diagnosis.[9], [13], [14], [19] The parameters B and C are considered important for the purposes of assigning the malignancy risk category for stratifying appropriate levels of surgical expertise for management.[22], [23] Patients with tumors categorized as “borderline/uncertain” even after SA in the third step were considered and managed as malignant, until proven otherwise on histopathology. The final diagnosis for all the categorized adnexal masses (benign or malignant) was confirmed at histopathology either by biopsy and/or by examination of tumor sections following surgical resection.

Further management

After histopathological confirmation of benign/malignant nature of disease, all patients were further managed as per recommended protocols.[6], [13] Patients finally categorized as having a benign adnexal tumor and were clinically asymptomatic, were advised bi-annual ultrasound surveillance. Symptomatic patients in the group of benign tumors in whom surgery was indicated, underwent surgical resection.[13] Patients with an adnexal tumor finally diagnosed as malignant, underwent comprehensive oncological imaging assessment for preoperative evaluation and staging. The staging for malignant adnexal tumors was done as per International Federation of Gynaecology and Obstetrics classification.[6] The work-flow methodology is summarized in [Figure 4].

Figure 4: Schematic diagram showing methodology followed in the study.

Statistical analysis

Diagnostic performance of all three strategies for their ability to discriminate between benign and malignant adnexal tumors was evaluated using histopathology as the gold standard. The statistical analysis for two-step strategy was performed only for the tumors in which a categorization could be obtained (“applicable” tumors).[1], [13] For the alternative two step and three-step strategies, the analysis was done for all the tumors. Parameters used to record diagnostic performance were sensitivity, specificity, positive and negative predictive values, and diagnostic accuracy. The diagnostic performance of each of these three strategies was calculated independently and compared with each other.

Results

The study comprised of 100 ovarian tumors evaluated by ultrasound using the IOTA two-step, alternative two-step, and three-step strategies and on histopathology 43 of these were benign and 57 were malignant tumors. Majority of the patients were between 21–60 years and unilateral tumors were more prevalent, as shown in [Table 4].

Table 4

Clinical profile of the patients (n=100)

Age group	Tumors		Total
	Unilateral	Bilateral
18-20	8	1	9
21-40	37	3	40
41-60	36	5	41
>60	6	4	10

By performing the two-step analysis, the level I operator diagnosed 24 adnexal masses as benign, 45 as malignant, and 31 as inconclusive. The correlation of the two-step results with final histopathology diagnosis is as shown in [Table 5]. The illustrative clinical case examples for two-step analysis are shown in [Figures 5] and [6]. [Figure 5] is the two-step and alternative two-step analysis for a benign tumor and [Figure 6] is for a malignant tumor. Illustrative examples of three different cases found inconclusive by step 2 analysis and further examined by step 3 are shown in [Figure 7], [8], [9]. The latter three were conclusively and correctly diagnosed at the third step.

Table 5

Two-step strategy diagnoses & corresponding histo-pathology (N=100)

Age in years	Total number	Two-step diagnosis						Histopathology
		Benign		Inconclusive		Malignant		Benign		Malignant
		No.	%	No.	%	No.	No.	%	No.	%
18-20	9	5	55.6	4	44.4	-	-	7	77.8	2	22.2
21-40	40	12	30	13	32.5	15	18.8	17	42.5	23	57.5
41-60	41	7	17.1	10	24.4	24	58.5	19	46.3	22	53.7
>60	10	-	-	4	40	6	60	-	-	10	100

Figure 5 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) and final histopathology (F) in a 35 year old female with right sided pelvic pain. Grey-scale US (A): Cystic mass, with fat-fluid level (green arrow), acoustic shadowing (blue arrow) and small solid component (Rokitansky nodule) <7 mm (red arrow). Color Doppler (B): No vascularity seen, consistent with a color score of 1. Two-step analysis tabulated in (C) and (D): “benign” diagnosis. Alternative two-step analysis tabulated in (E): “Very low risk”. Histopathology (F) confirms the US diagnosis of mature cystic teratoma.

Figure 6 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) and final histopathology (F) in a 60 year old female with pelvic mass and abdominal distension. Grey-scale US (A): Large cystic mass, with multiple papillary projections (red arrows). Color Doppler (B): Central vascularity (green arrow) consistent with a color score 4. Two-step analysis tabulated in (C) and (D): “malignant” diagnosis. Alternative two-step analysis tabulated in (E): “Very high risk”. Histopathology (F) confirms the US diagnosis of mucinous adenocarcinoma.

Figure 7 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) and final histopathology (F) in a 24 year old female with pelvic pain and dyspepsia. Grey-scale US (A): Large (>100 mm) multilocular cystic mass, with nested solid component (red arrow). Color Doppler (B): Peripheral vascularity (green arrow), consistent with a color score of 2. Two-step analysis tabulated in (C) and (D): “benign” diagnosis. Alternative two-step analysis tabulated in (E): “Intermediate risk”. Mass was re-evaluated at third step, the diagnosis suggested was serous cystadenoma, confirmed at histopathology (F).

Figure 8 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) and final histopathology (F) in a 65 year old female with enlarging abdomino-pelvic mass. Grey-scale US (A): Unilocular cystic tumor showing homogeneous echoes (red arrow) and solid component (blue arrow), causing acoustic shadow (green arrow), with ascites (white arrow). Color Doppler (B): No vascularity seen, consistent with a color score of 1. Two-step analysis tabulated in (C) and (D): “inconclusive” diagnosis. Alternative two-step analysis tabulated in (E): “Elevated risk”. At the third step examination, diagnosis suggested was mucinous cystadenocarcinoma, confirmed on histopathology (F).

Figure 9 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) and final histopathology (F) in a 35 year old female with pelvic mass and menorrhagia. Grey-scale US (A): Smooth solid tumor with ascites (red arrow). Color Doppler (B): low grade internal vascularity, consistent with a color score of 2. Two-step analysis tabulated in (C) and (D): “inconclusive” diagnosis. Alternative two-step analysis tabulated in (E): “Elevated risk”. At third step, diagnosis suggested was granulosa cell tumor, confirmed on histopathology (F), blue arrows show Cal-Exner bodies.

Subsequent to performing the two-step analysis, the level I operator performed the alternative two-step analysis to assign a malignancy risk to the adnexal tumors. The correlation of the risk assigned by the alternative two-step with final histopathology diagnosis is as shown in [Table 6]. The results of the alternative two-step strategy for risk stratification in the adnexal tumors revealed: 16 as “very low risk,” 15 as “low risk,” 13 as “intermediate risk,” 23 as “elevated risk,” and 33 as “very high risk” for malignancy category. The illustrative clinical case examples for risk assigned to the tumors are shown in [Figures 5], [6], [7], [8], [9], [10], [11].

Table 6

Alternative two-step diagnosis and their corresponding histo-pathology (n=100)

Age in Years	Total no.	Alternative two-step risk category										Histopathology
		Very low		Low		Intermediate		Elevated		Very high		Benign		Malignant
		No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
18-20	9	4	44.4	2	22.2	3	33.3	-	-	-	-	7	77.8	2	22.2
21-40	40	7	17.5	5	12.5	5	12.5	15	37.5	8	20	17	42.5	23	57.5
41-60	41	5	12.2	8	19.5	3	7.3	6	14.6	19	46.3	19	46.3	22	53.7
>60	10	-	-	-	-	2	20	2	20	6	60	-	-	10	100

Figure 10 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) and final histopathology (F) in a 20 year old female with intermittent pain abdomen. Grey-scale US (A): Unilocular cystic tumor with papillary projections (red arrow). Color Doppler (B): No vascularity seen, consistent with a color score of 1 (green arrow). Two-step analysis tabulated in (C) & (D): “inconclusive” diagnosis. Alternative two-step analysis, tabulated in (E): “Elevated risk”. At third step, diagnosis suggested was mucinous cyst adenocarcinoma (malignant). Histopathology (F): Mucinous cystadenoma (benign).

Figure 11 (A-F): Are the Ultrasound (US) images (A & B), IOTA “two-step” & “alternative two-step” analysis (C-E) with final histopathology (F) in a 47 year old female with abdomino-pelvic mass & abdominal distension. Grey-scale US (A): Smooth solid tumor with ascites (red arrows). Color Doppler (B): Central vascularity seen, consistent with a color score of 3 (green arrow). Two-step analysis tabulated in (C) & (D): “inconclusive” diagnosis. Alternative two-step analysis, tabulated in (E): “Elevated risk”. At third step, diagnosis suggested was malignant germ cell tumor. Histopathology (F): Benign ovarian fibroma with Meig’s syndrome.

Subsequent to the two-step analysis and the alternative two-step analysis by the level I operator, 31 patients in the “inconclusive” category on two-step, were re-evaluated by the experienced operator, so as to fulfil the third step of the three-step strategy. Amongst 31 inconclusive adnexal tumors, 18 tumors were diagnosed as benign, 9 as malignant and 4 remained as “borderline”/“uncertain”. The correlation of the step three results with final histopathology diagnosis is as shown in [Table 7]. The diagnostic analysis by each of the individual three parameters of step 3 analysis is shown in [Table 8]. The illustrative case example of the application of the two-step (SD plus SR), alternative two-step (SRrisk score), and third step (SA) analyses are shown in [Figures 5], [6], [7], [8], [9], [10], [11]. For the patients illustrated in [Figures 5] and [6], the diagnosis arrived at the two-step was concordant with histopathology. For the patients illustrated in [Figures 7], [8], [9], the ultrasound diagnosis was “inconclusive” at two-step but conclusively diagnosed at the “third step” as the third-step diagnosis was found concordant with histopathology for benign/malignant category. The patients illustrated at [Figures 10] and [11] were “inconclusive” at two-step and were correctly diagnosed as malignant at third step; however, the “self-presumed histological diagnosis” was found discordant with the final histopathology. The final histopathology result and distribution of adnexal pathology are shown in [Table 9] and the correlation between the “self-presumed histological diagnosis” and the final histology are shown in [Table 10].

Table 7

Third-step diagnosis, corresponding histopathology, and evolution of diagnoses for tumors characterized as "inconclusive" on two-step strategy (n=31)

Age group	Inconclusive by two-step	Step 3	Histopathology
18-20	4	3 benign, 1 borderline/uncertain	4 benign
21-40	13	8 benign, 5 malignant	7 benign, 6 malignant
41-60	10	8 benign, 2 malignant	8 benign, 2 malignant
>60	4	2 malignant, 2 borderline/uncertain	3 malignant, 1 benign

Table 8

Diagnostic performance of IOTA third step (subjective assessment) (n=31)

Age group	Parameter A			Parameter B (probability of malignancy)					Parameter C (self-impression of examiner)	Final histology
	Benign	Borderline	Malignant	Benign	Probably benign	Uncertain	Probably malignant	Malignant
18-20	3	1	0	2	1	0	1	0	1 diagnosed as malignant mucinous cystadenocarcinoma	Mucinous cystadenoma
21-40	7	1	5	7	1	0	1	4	1 diagnosed as probably benign mucinous cystadenoma	Borderline mucinous cyst adenocarcinoma
41-60	8	0	2	6	2	0	0	2	All concordant	All concordant
>60	0	2	2	0	1	1	0	2	1 diagnosed as malignant germ cell tumor	Diagnosed as Fibroma with Meig’s syndrome
									1 diagnosed as probably benign serous tumor	Diagnosed as borderline serous CA

Table 9

Final histopathology of the assessed masses (n = 100)

Benign masses	Number	Malignant masses	Number
Endometrioma	8	Mucinous cystadenocarcinoma	14
Mature cystic teratoma	7	Metastasis or Krukenberg tumor	13
Mucinous cystadenoma	7	High grade serous tubo-ovarian malignancy	11
Serous cyst adenoma	6	Borderline mucinous or serous ovarian tumor	8
Hemorrhagic cyst	4	Immature teratoma	8
Sub-serosal fibroid	3	Dysgerminoma	2
Infective tubo-ovarian complex	3	Primary tubal carcinoma	1
Fibroma	2	Endometriotic carcinoma	1
Hydrosalpinx	2	Malignant Brenner tumor	1
Parovarian cyst	1	Malignant granulosa cell tumor	1

Table 10

Self-assessment diagnosis and its correlation with histology (n=31)

Category	Total number	Correctly diagnosed as benign	Correctly diagnosed as malignant	Incorrectly diagnosed
Epithelial ovarian tumors	14	7	5	2
Germ cell tumors	8	6	10	1
Stromal tumors	3	1	2	1
Metastasis	3	0	13	0
Endometriosis	1	8	0	0
Sub-serosal fibroid	1	3	0	0
Infective tubo-ovarian complex	1	3	0	0

The sensitivity, specificity, PPV, and NPV of all three strategies are shown in [Table 11] and in [Figure 12]. The sensitivity, specificity, PPV, and NPV for the two-step strategy was 87.5%, 79.2%, 89.4%, and 76%, respectively; for alternative two-step strategy, it was 91.5%, 75.6%, 84.4%, and 86.1%; and for three-step strategy, it was 98.2%, 93.3%, 94.7%, and 97.7%, respectively. Diagnostic accuracy was 84.7% for two-step strategy, 88% for alternative two-step strategy, and 96% for three-step strategy.

Table 11

Diagnostic performance of IOTA strategies

Diagnostic parameters	Sensitivity	Specificity	PPV	NPV	Diagnostic accuracy
Two-step strategy	87.5%	79.2%	89.4%	76%	84.7%
Alternative two-step strategy	91.5%	75.6%	84.4%	86.1%	88%
Three-step strategy	98.2%	93.3%	94.7%	97.7%	96%

Figure 12: Schematic diagram depicting a summary of progressive improvement of diagnostic performance after application of IOTA “two-step”, “alternative two-step” and “three-step” strategies.

Discussion

An early and accurate characterization of adnexal masses affects timely intervention and undoubtedly remains critical in improving survival rates of patients with ovarian malignancy. As ultrasound assessment is the most widely used primary imaging modality, a standardized evaluation protocol plays a significant role in obtaining optimal management results. As early as 1991, Sassone et al. from New York, were the pioneers for suggesting a morphology-based ultrasound scoring system for adnexal masses.[24] With ongoing technological advances and the advent of Color Doppler, the additional utility of the latter technique over conventional grey scale evaluation was explored and reported by Valentin et al. from Malmo.[25], [26] Although imaging is the primary domain of Radiologists, paradoxically in the field of adnexal mass examination, ultrasound evaluation scoring systems have been led by Gynaecologists and Oncology surgeons, since their very inception.

The IOTA group has over last two decades performed commendable work toward standardizing ultrasound evaluation in adnexal tumors. The latter which was founded by Dr Dirk Timmerman, Lil Valentin, and Tom Bourne, initially comprised mainly of Gynaecologists, Oncology surgeons, and Statisticians from nine European countries, who have been working extensively for the last two decades. The coordinating centre is KU Leuven, Leuven, Belgium.[20] In the year 2000, IOTA published their “consensus statement on terms, definitions and measurements,” to describe the sonographic features of adnexal masses.[3] Subsequently, the group proposed a “Risk of malignancy index (RMI)” and two logistic regression models, LR1 and LR2, which were later surpassed by their own “Simple Rules” published in 2008.[6], [9]

The IOTA models and strategies are based on application of IOTA’s “Simple Descriptors” (SD) and “Simple Rules” (SR), which are based on pre-defined morphological US characteristics favouring either benignity or malignancy. The IOTA surmise being that most ovarian/adnexal masses are accurately classifiable as benign or malignant, by an “operator”/“sonographer” with adequate level I training, by performing the first and second step examinations and the remainder by an “operator with level III training”/“experienced clinician” at the third step.[9] The primary “two-step” strategy comprises of applying the SD in the first step and SR in the second step to distinguish ovarian tumors as benign or malignant. The “modified/alternative two-step” strategy, or “simple rules risk calculation,” (SRrisk), is aimed at effectively segregating patients into specific malignancy risk categories for assigning the appropriate level of clinical/surgical management based on the ultrasound diagnosis.[1], [11], [13] The “three-step strategy” is a subsequent “real-time subjective assessment” (.SA) for diagnosis, which is performed by an experienced operator, solely for those tumors classified as “inconclusive” at the “second step”.[9], [11], [13] At the third step, the goal is to try and achieve an as accurate as possible preoperative diagnosis, for assigning the most appropriate level of clinical management. This is imperative in patient care interest, since benign tumors will anyway have a good outcome with surgery at the hands of general Gynaecologists; however, malignant ones will certainly have a better prognosis at specialized oncology centres.[12] SRrisk calculation/estimation or “ alternative two step”, which is based on SR has been proposed as a next best alternative to the third-step strategy.

The IOTA SR have been vigorously validated and published by their associates.[6], [7], [22], [23], [27], [28] Additionally, there are two Indian studies which have reported on the “IOTA Simple Rules”.[15], [16] Although the American College of Radiology has proposed O-RADS for ovarian-adnexal masses, a very recent paper by Patel-Lippmann et al. from the USA, which compares “IOTA simple rules with the Society of Radiologists in Ultrasound guidelines-for detection of malignancy in adnexal cysts,” has reported that the IOTA simple rules were more accurate.[17], [29] The diagnostic performance of the IOTA two-step strategy in the present study was comparable to previous studies from India and the other validation studies. For “two-step” strategy, using IOTA simple rules, the sensitivity, specificity, PPV, and NPV in our study were 87.5%, 79.2%, 89.4%, and 76%, respectively, which was comparable to the results obtained by the IOTA group study, which were 95%, 91%, 80.9%, and 97.6%, respectively, and also that by most external validators.[2], [9], [27], [28] The results for diagnostic accuracy of SR obtained by one of the previous Indian studies were 92.8%, 92.9%, 70.2%, and 98.6%, respectively.[16] A comparative overview of the diagnostic accuracy in a few of previous studies, with those in the present one, is summarized in [Table 12].

Table 12

Comparison of performance of IOTA SR in our study and in previous studies

Authors	Country	Year	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Diagnostic Accuracy (%)	Inconclusive (%)
Timmerman[6]	Multiple, EU	2008	95	91	80.9	97.6	91.9	23.9
Timmerman[7]	Multiple, EU	2010	92	96	87.4	97.4	94.8	22.5
Sayasneh[2]	Multiple, UK	2013	87	98	93.8	95.7	95.3	16.1
Alcazar[26]	2 centers, Spain	2013	88	97	95.3	98.3	96.3	20.6
Nunes[27]	Single center, UK	2014	96	89	87.1	96.7	91.9	21.8
Garg[15]	Single center, India	2017	91.7	84.8	68.8	96.6	88.9	10
Shetty[16]	Single center, India	2019	92.8	92.9	70.2	98.6	91.4	10.7
Grover et al, [present study]	Single center, India	2019-20	87.5	79.2	89.4	76	84.7	28

As the clinical outcome of any patient with an adnexal tumor, depends a great deal on the expertise available for surgical management, therefore its mere classification as benign or malignant, fails to completely fulfil the range of diagnostic information required for effective management stratification. To overcome this information gap, the IOTA group had, based on simple rules, proposed a “Simple rules risk calculation tool 15”, (SRrisk) which was published in 2016.[1], [20] The “alternative two-step” strategy is the application of SRrisk calculator for assigning the levels of surgical expertize individualized for each patient, so as to achieve the best prognosis.[1], [11], [13] The application of the SRrisk calculator tool has been named as the “alternative two step” by Hidalgo et al., a nomenclature which has been used in the present study as well.[13] The external validation studies on this alternative two-step assessment of the performance of the SRrisk tool, have been published infrequently, one only in 2019 by Hidalgo et al. from Spain.[13] The latter study was a retrospective validation of the SRR tool. To the best of our knowledge, ours is among the few prospective external validation studies for the alternative two-step analysis, that too from a radiology group, based in a tertiary care Indian hospital. For IOTA SRrisk tool, the sensitivity, specificity, and NPV in our prospective study were 91.5%, 75.6%, and 86.1%, respectively, which was comparable to the results obtained by Hidalgo et al., which was 98.8%, 83.2%, and 98.8%, respectively, in a retrospective study. Moreover, the PPV of IOTA SRrisk, in our study was superior, achieving a value of 84.7% compared to a value of 31.3% obtained in the study by Hidalgo et al.[13]

The IOTA group has concluded in a number of their major papers that a “subjective analysis” based on pattern recognition, when performed by a level III operator with years of experience, provides the most accurate diagnosis.[12], [28] For the IOTA third step, performed as a SA by an expert examiner, the sensitivity and specificity in our study was 98.2% and 93.3%, respectively, which was comparable to the results obtained by the IOTA group, which was 92% and 92%, respectively, and by external validators, Hidalgo et al., which was 95.1% and 97.7%, respectively.[13] In addition, the excellent diagnostic accuracy of the three-step strategy in our study is in agreement with the results of a meta-analysis of studies by Meys et al. 2016, where in “subjective evaluation by expert examiners” was found to yield the best results, compared to all other ultrasound strategies.[30] That “Expert operators” are likely to provide an accurate histological diagnosis, after years of experience has also been sufficiently validated in our study. The expert was able to predict an accurate histological diagnosis in 87.6% of the inconclusive cases in our study, as shown in [Table 10], and in [Figure 7], [8], [9]. The clinical relevance of the level III operator providing a “presumed histological diagnosis” is for the purposes of planning the ideal levels of surgical expertize as explained above. The diagnostic performance of IOTA three-step in our study and comparison of the same with previous studies are shown in [Table 13].

Table 13

Comparison of performance of IOTA three-step strategy our study and in previous studies

Author	Year	Sample size	Sensitivity (%)	Specificity (%)	Comment
Ameye[9]	2012	1938	92.2	92.3	Based on histology and expert assessment
Sayasneh[2]	2013	301	93.0	92.0	Based on histology and nonexpert assessment
Testa[11]	2014	2403	92.5	87.6	Based on histology and expert assessment
Alcazar[14]	2016	666	94.3	94.9	Based on histology, follow up, and nonexpert assessment
Hidalgo[13]	2019	283	95.1	97.7	Based on histology, follow up, and nonexpert assessment
Present study	2019-20	100	98.2	93.3	Based on histology and expert assessment

The strength of our study was that the diagnostic performance obtained for all three IOTA strategies using “two-step,” “alternative two-step,” and the third step was as good as that reported in all previous studies. Another strength of our study was that the diagnostic performance of all the three ultrasound-based IOTA strategies have been evaluated by radiologists, which is a group of specialists comprehensively trained for image interpretation.

The limitations of our study were localization to a single centre, a not-so-large cohort of cases, and single operators at both ends of the levels of EFSUMB expertize, which limited the prediction of inter-observer variability. In addition, it was found, that the composition of the study population was slightly skewed toward a larger number of malignant tumors, which is understandable as our Institute is a tertiary care centre. A similar distribution has been observed by other investigators as well.[12] The applicability of the ORADS to our study population could also be considered a sort of limitation; however, as the ORADS was still evolving at the initiation of our study, it could not have been incorporated into the study protocol.

MRI is also considered a good modality for differentiating adnexal masses and ADNEX MRI scores have been proposed toward this goal.[31], [32], [33] However, MRI suffers from limitations such as prolonged examination time, contraindications in patients with metallic implants, contraindication for contrast administration in those with compromised renal functions, besides high cost and lack of wide availability in resource poor countries. We are, therefore, in agreement with other investigators who believe that MRI should be considered the preferred modality mainly for local staging in ovarian cancer but not for primary diagnosis, wherein ultrasound remains the recommended tool. Our recommendation is that only for the few patients categorized as “borderline”/“uncertain” after the third step (in IOTA strategy), MRI may be recommended for problem solving and characterization of the adnexal tumor at the stage of primary diagnosis. A similar view has been shared in a very recent study by Pereira et al., as well.[33]

Conclusions

To the best of our knowledge, ours is the first external validation study for IOTA’s “two-step”, “alternative two-step,” and “three-step” strategies from a tertiary care hospital in India. While two step and alternative two step strategies showed a good diagnostic accuracy of 84.7% and 88%, respectively, the three step strategy demonstrated an excellent diagnostic accuracy of 96%. Since ultrasound is the primary modality in the evaluation of adnexal masses, our recommendation for future alignment towards standard application of the IOTA three-step strategy in routine Radiology practice, appears sufficiently justified. Especially, more so, since in majority of hospitals and radiology clinics, pelvic ultrasound for gynaecology and obstetrics applications is performed/interpreted only by trained radiologists and the requisite expertise for EFSUMB levels of ultrasound operators is naturally fulfilled. Although ultrasound evaluation and scoring systems for adnexal masses have since their very inception been proposed and validated mainly by Gynaecologists and Oncology surgeons, this rather strange paradigm can be made to shift toward being more radiology-centric and back into the arena of Radiology and Radiologists, the imaging experts.

Recommendations

The standardization of ultrasound reports for adnexal masses is an unfulfilled clinical necessity, which is long over-due not only to our gynae-oncology surgeon colleagues but also to our patients. Therefore, based on the results of our study and on the previous literature, we propose that the Radiology departments in hospitals, or Radiology clinics, may adopt protocols whereby, patients with adnexal masses are primarily examined by a level I operator, who performs the two-step and alternative two-step by applying the SR for characterization and also assigns a malignancy risk using the SRrisk calculator tool. The final report, however, should incorporate the SA of the expert operator as the third step. Since, all three strategies have shown good diagnostic performance in our Radiology setting and also because radiologists are comprehensively trained in image interpretation, the IOTA strategies should be more enthusiastically adopted by Radiologists. We believe that the inculcation of these rules and strategies will also serve as an advantageous “stepping stone primer,” for the upcoming O-RADS, as the O-RADS, although a more elaborate and complex system, nevertheless, draws extensively from the IOTA algorithms.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form, the patients have given their consent for their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Conflict of Interest

There are no conflicts of interest.

• References

• 1 Timmerman D, Van Calster B, Testa A, Savelli L, Fischerova D, Froyman W. et al. Predicting the risk of malignancy in adnexal masses based on the simple rules from the International Ovarian Tumor Analysis group. Am J Obstet Gynecol 2016; 214: 424-37
• 2 Sayasneh A, Kaijser J, Preisler J, Johnson S, Stalder C, Husicka R. et al. A multicenter prospective external validation of the diagnostic performance of IOTA simple descriptors and rules to characterize ovarian masses. Gynecol Oncol 2013; 130: 140-6
• 3 Timmerman D, Valentin L, Bourne T, Collins W, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: A consensus opinion from the International Ovarian Tumor Analysis (IOTA) group. Ultrasound Obstet Gynecol 2000; 16: 500-5
• 4 Timmerman D, Testa A, Bourne T, Ferrazzi E, Ameye L, Konstantinovic M. et al. Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: A multicenter study by the international ovarian tumor analysis group. J Clin Oncol 2005; 23: 8794-801
• 5 Timmerman D, Van Calster B, Jurkovic D, Valentin L, Testa A, Bernard J. et al. Inclusion of CA-125 does not improve mathematical models developed to distinguish between benign and malignant adnexal tumors. J Clin Oncol 2007; 25: 4194-200
• 6 Timmerman D, Testa A, Bourne T, Ameye L, Jurkovic D, Van Holsbeke C. et al. Simple ultrasound-based rules for the diagnosis of ovarian cancer. Ultrasound Obstet Gynecol 2008; 31: 681-90
• 7 Timmerman D, Van Calster B, Testa A, Guerriero S, Fischerova D, Lissoni A. et al. Ovarian cancer prediction in adnexal masses using ultrasound-based logistic regression models: A temporal and external validation study by the IOTA group. Ultrasound Obstet Gynecol 2010; 36: 226-34
• 8 Timmerman D, Ameye L, Fischerova D, Epstein E, Melis G, Guerriero S. et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: Prospective validation by IOTA group. BMJ 2010; 341: c6839
• 9 Ameye L, Timmerman D, Valentin L, Paladini D, Zhang J, Van Holsbeke C. et al. Clinically oriented three-step strategy for assessment of adnexal pathology. Ultrasound Obstet Gynecol 2012; 40: 582-91
• 10 Kaijser J, Bourne T, Valentin L, Sayasneh A, Van Holsbeke C, Vergote I. et al. Improving strategies for diagnosing ovarian cancer: A summary of the International Ovarian Tumor Analysis (IOTA) studies. Ultrasound Obstet Gynecol 2012; 41: 9-20
• 11 Testa A, Kaijser J, Wynants L, Fischerova D, Van Holsbeke C, Franchi D. et al. Strategies to diagnose ovarian cancer: New evidence from phase 3 of the multicentre international IOTA study. Br J Cancer 2014; 111: 680-8
• 12 Alcázar JL. Ultrasound-based IOTA simple rules allow accurate malignancy risk estimation for adnexal masses. BMJ Evid Based Med 2016; 21: 197
• 13 Hidalgo J, Ros F, Aubá M, Errasti T, Olartecoechea B, Ruiz-Zambrana Á. et al. Prospective external validation of IOTA three - step strategy for characterizing and classifying adnexal masses and retrospective assessment of alternative two - step strategy using simple-rules risk. Ultrasound Obstet Gynecol 2019; 53: 693-700
• 14 Alcázar J, Pascual M, Graupera B, Aubá M, Errasti T, Olartecoechea B. et al. External validation of IOTA simple descriptors and simple rules for classifying adnexal masses. Ultrasound Obstet Gynecol 2016; 48: 397-402
• 15 Garg S. Evaluation of IOTA simple ultrasound rules to distinguish benign and malignant ovarian tumours. J Clin Diagn Res 2017; 11: TC06-9
• 16 Shetty J, Saradha A, Pandey D, Bhat R, Kumar P, Bharatnur S. IOTA simple ultrasound rules for triage of adnexal mass: Experience from South India. J Obstet Gynecol India 2019; 69: 356-62
• 17 Andreotti R, Timmerman D, Strachowski L, Froyman W, Benacerraf B, Bennett G. et al. O-RADS US risk stratification and management system: A consensus guideline from the ACR ovarian-adnexal reporting and data system committee. Radiology 2020; 294: 168-85
• 18 [Internet]. Efsumb.org. 2020. Available from: http://www.efsumb.org/guidelines/2009-04-14apx1.pdf. [Last cited on 2020 Apr 19]
• 19 Bourne T, Valentin L, Timmerman D. A multicentre study to examine the short and long term outcomes of the conservative management of benign-looking adnexal masses and the pre-operative characterisation of ovarian tumours [Internet]. Clinical trials.gov 2020. Available from: https://clinicaltrials.gov/ProvidedDocs/32/NCT01698632/Prot_000.pdf. [Last cited on 2020 Apr 19]
• 20 IOTA. Simple Rules and SR risk calculator to diagnose ovarian cancer. Iota Group; [Internet]. Iotagroup.org. 2020. Available from: https://www.iotagroup.org/iota-models-software/iota-simple-rules-and-srrisk-calculator-diagnose-ovarian-cancer. [Last cited on 2020 Apr 19]
• 21 Pineda L, Salcedo E, Vilhena C, Juez L, Alcázar J. Interobserver agreement in assigning IOTA color score to adnexal masses using three-dimensional volumes or digital videoclips: Potential implications for training. Ultrasound Obstet Gynecol 2014; 44: 361-4
• 22 Kaijser J, Bourne T, Valentin L, Sayasneh A, Van Holsbeke C, Vergote I. et al. Improving strategies for diagnosing ovarian cancer: a summary of the International Ovarian Tumor Analysis (IOTA) studies. Ultrasound in Obstetrics & Gynecology 2012; 41 (01) 9-20
• 23 Kaijser J. Towards an evidence-based approach for diagnosis and management of adnexal masses: findings of the International Ovarian Tumour Analysis (IOTA) studies. Facts Views Vis Obgyn 2015; 7 (01) 42-59
• 24 Sassone AM, Timor-Tritsch IE, Artnew A, Westhoff C, Warren WB. Transvaginal sonographic characterization of ovarian disease: Evaluation of a new scoring system to predict ovarian malignancy. Obstet Gynecol 1991; 78: 70-6
• 25 Valentin L, Sladkevicius P, Marsàl K. Limited contribution of Doppler velocimetry to the differential diagnosis of extrauterine pelvic tumors. Obstet Gynecol 1994; 83: 425-33
• 26 Valentin L. Gray scale sonography, subjective evaluation of the color Doppler image and measurement of blood flow velocity for distinguishing benign and malignant tumors of suspected adnexal origin. Eur J Obstet Gynecol Reprod Biol 1997; 72: 63-72
• 27 Alcazar J, Pascual M, Olartecoechea B, Aubá M, Graupera B, Hereter L. et al. OC14.08: IOTA simple rules for discriminating between benign and malignant adnexal masses: A prospective external validation. Ultrasound Obstet Gynecol 2013; 42 (01) 30-30
• 28 Nunes N, Ambler G, Foo X, Naftalin J, Widschwendter M, Jurkovic D. Use of IOTA simple rules for diagnosis of ovarian cancer: Meta-analysis. Ultrasound Obstet Gynecol 2014; 44: 503-14
• 29 Patel-Lippmann K, Sadowski E, Robbins J, Paroder V, Barroilhet L, Maddox E. et al. Comparison of International ovarian tumor analysis simple rules to society of radiologists in ultrasound guidelines for detection of malignancy in adnexal cysts. AJR 2020; 214: 694-700
• 30 Meys E, Kaijser J, Kruitwagen R, Slangen B, Van Calster B, Aertgeerts B. et al. Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis. Eur J Cancer 2016; 58: 17-29
• 31 Thomassin-Naggara I, Aubert E, Rockall A, Jalaguier-Coudray A, Rouzier R, Daraï E. et al. Adnexal masses: Development and preliminary validation of an mr imaging scoring system. Radiology 2013; 267: 432-43
• 32 Pereira PN, Sarian LO, Yoshida A, Araújo KG, Barros RH, Baião AC. et al. Accuracy of the ADNEX MR scoring system based on a simplified MRI protocol for the assessment of adnexal masses. Diagn Interv Radiol 2018; 24: 63-71
• 33 Pereira P, Sarian L, Yoshida A, Araújo K, Silva A, de Oliveira Barros R. et al. Improving the performance of IOTA simple rules: Sonographic assessment of adnexal masses with resource-effective use of a magnetic resonance scoring (ADNEX MR scoring system). Abdom Radiol 2020; 45: 3218-29

Publication History

Received: 20 August 2020

Accepted: 17 August 2020

Article published online:
19 July 2021