Rofo 2021; 193(08): 919-927
DOI: 10.1055/a-1346-0028
Technical Innovations

Automatic, log file-based process analysis of a clinical 1.5T MR scanner: a proof-of-concept study

Article in several languages: English | deutsch
1   Department of Radiology and Nuclear Medicine, University Hospital Schleswig-Holstein, Campus Lübeck, Germany
,
Tobias Boppel
2   Department of Neuroradiology, University Hospital Schleswig-Holstein, Campus Lübeck, Lübeck, Germany
,
Victoria Sieber
1   Department of Radiology and Nuclear Medicine, University Hospital Schleswig-Holstein, Campus Lübeck, Germany
,
Joachim D. Schmidt
3   Philips GmbH Innovative Technologies, Research Laboratories, Hamburg, Germany
,
Jörn Borgert
3   Philips GmbH Innovative Technologies, Research Laboratories, Hamburg, Germany
,
Peter Schramm
2   Department of Neuroradiology, University Hospital Schleswig-Holstein, Campus Lübeck, Lübeck, Germany
,
Jörg Barkhausen
1   Department of Radiology and Nuclear Medicine, University Hospital Schleswig-Holstein, Campus Lübeck, Germany
,
3   Philips GmbH Innovative Technologies, Research Laboratories, Hamburg, Germany
› Author Affiliations
 

Abstract

Purpose In light of the steadily increasing need for economical efficacy and capacity utilization it was the aim of this proof-of-concept work to implement an automated logfile-based analysis tool for MRI scanner utilization and to establish a process analysis. As a primary step, analyses of scanner and protocol utilization, parametrization of protocol processes, their durations, age dependency, and scan efficacy were to be tested.

Materials and Methods Logfiles were continuously extracted from a 1.5 T MR scanner (Philips Achieva) and automatically explored for relevant scan parameters. Parameters were extracted into a database and logically combined to protocol parameters. Visualization was achieved using PowerBI (Microsoft, USA). Data aggregation comprised a day-based and protocol-based strategy. In addition, age- and regional-based testing was performed. The frequency of protocol usage was evaluated and those protocols with frequent usage compared regarding efficacy to those rarely used.

Results After successful technical implementation, 3659 MR exams were available for further analysis. Out of a plethora of parameters, those relevant to the understanding of the scan process were identified. The initial results mirror the daily scanner usage and allow identifying, e. g., shortened scanner usage on Fridays or longer examination times in children. A scan efficacy of 69.6 ± 17.6 % excluding preparation process was identified as a parameter with high potential to be optimized in daily routine.

Conclusion The logfile-based analysis of MR scanner processes was successfully introduced and holds the promise to be extended into a comprehensive analytic tool for the analysis and optimization of scanner processes. In combination with other variables from the departmental or institutional infrastructure or patient-specific information such tool may be developed into a intelligent steering tool.

Key Points:

  • The automated log file analysis of MR-scanner processes was successfully introduced

  • The log file-analysis allows for a detailed analysis of scanner processes

  • From a log file-analysis, there is potential benefit to users, applications specialists and developers

Citation Format

  • Frydrychowicz A, Boppel T, Sieber V et al. Automatic, log file-based process analysis of a clinical 1.5T MR scanner: a proof-of-concept study. Fortschr Röntgenstr 2021; 193: 919 – 927


#

Introduction

Modern medicine is characterized by the ever-increasing need for resource management. In this context, concepts such as efficiency increase and, in the imaging disciplines, scanner utilization are parameters confronting clinics and practices alike. In MRI, reducing the number of sequences is an obvious but limited opportunity to increase efficiency. The extent to which alternative sequences or acceleration techniques, such as parallel imaging [1] or process chain restructuring [2], are efficient or cost-effective in scheduling, patient preparation, and management can be established only using process metrics and analysis.

In addition to the time stamps of the image data available in the DICOM header, so-called log files which continuously record numerous states and changes of the scanner are available. Using MRI as an example, this includes features such as scanner states, exam protocols and sequences used, coils employed, patient table movement, and error messages. The corresponding time points of the changes are also logged. These data provide, for example, required information to the service department during maintenance or troubleshooting, but are not utilized by the user outside of the servicing process. Users and manufacturers could benefit from these data, e. g., for process optimization concepts or individually-optimized consulting strategies.

The multi-layered nature of these data provides insight into their wide-ranging utility. In addition to typical or necessary settings by the medical-technical assistant, the duration of aborted or repeated sequences and planning times can also be identified, leading to more precise representation of operating times than via the time stamps of stored image data. In particular evaluations, only information derived from such a database can be evaluated, such as the calculation of the changeover time as the difference in times from the end of one examination to the start of the subsequent examination or idle times between planning steps and diagnostic sequences. This allows further analysis of scanner, protocol or sequence usage and change-over frequencies to be performed and conclusions to be drawn about weaknesses in protocols or process delays.

This makes the data especially suitable for process analysis and optimization by the user or by application specialists, whether as a building block of process optimization chains [2] [3] [4] or as a stand-alone information medium, thus providing the ability to identify and simplify complex or time-consuming protocols, for example. Such an analysis also allows frequently repeated sequences to be identified as needing improvement or training. In addition, such a concept has the potential to be used for decision making, e. g., before investing in acceleration technologies [5].

Therefore, the aim of this work was to present, in a feasibility study, the design and development of a PowerBI-based (Microsoft, USA) evaluation strategy of automatically-generated log files at the level of scanner and protocol usage. For this purpose, data from an MR scanner used in routine inpatient care were evaluated, and workday procedures and protocol usage frequencies were comparatively analyzed. To this end, we present day-aggregated data supporting general scanner usage analysis, and protocol-aggregated data, enabling insight into protocol-level execution. The age-dependence of individual parameters of the MRI scan are analyzed as an example.


#

Materials and Methods

Study planning

Prior to methodological implementation, the consent of relevant employees and staff council for this project was obtained, since evaluation of the performance of individual employees can potentially be achieved by comparing analysis results with duty rosters or the entries in the RIS (radiology information system). Prior to the start of the project approval was also obtained from the Data Protection Office and the Ethics Committee. Since only pseudonymized data were assessed retrospectively and after anonymization, the Ethics Committee waived de-novo consent into the study.


#

Infrastructure and IT concept

The analysis was performed on data from a routine clinical MRI scanner (1.5T; Achieva, v3.2.2, Philips, Best, NL) installed in 2006, which is used workdays between 8:00a. m. and 8:00 p. m., Fridays until 6:00 p. m. with an option to extend to 8:00 p. m. in case of emergency. Weekends, holidays, and all-day scanner downtime due to maintenance or technical faults were excluded from the analysis. Thus, only days were considered on which data were available from at least 6 hours of scanning operation. Examinations of a scientific nature (studies, protocol validation or optimization) as well as clinical examinations < 15 and > 75 min were excluded to eliminate premature scan interruptions or situations such as emergencies.

The multi-step, automated and daily data processing initially involved nightly transfer of the log files to the analysis server, as shown schematically in [Fig. 1]. The central element of the analysis is the subsequent automatic extraction of the relevant information of the examination sequences (time stamps and events) and the possibility to derive or define a large number of resulting parameters. As an example, the table preparation time (“TableSetupTime”) is defined as the interval between the first table movement of an examination until the last table movement before the start of the first sequence, typically an overview scan (survey). The parameters evaluated in this work are illustrated in [Fig. 2] and their definitions are provided in the Appendix. Respectively prepared and aggregated data were fed into a database (Microsoft, SQLExpress) and then visualized using PowerBI or, alternatively, exported in tabular form for evaluation, e. g., using other statistics programs.

Zoom Image
Fig. 1 Schematic of the daily updated, automated data processing chain of the log file analysis.
Zoom Image
Fig. 2 Schematic visualization of the process analysis and its terminology. Depicted time lines become more detailed in descending order. Dashed lines depict magnified illustrations of respective sections within the exam. For definitions of used terms please refer to the appendix.

The installation of the system required only the integration of the analysis server (commercial desktop computer with network card) into the secure hospital network. The log files were provided by means of an encrypted data connection between the MRI scanners and the analysis computer set up by the manufacturer for this purpose. No other hardware or software modifications to the MRI scanner were required. The system does not allow access to image data or patient records, nor does it allow connection to the public Internet.

In addition to the parameters listed in [Fig. 2], which were defined on the basis of the log file entries, Effgesamt (English: Efftotal) and EffScan were calculated as examples of derived characteristics which can be used to assess efficiency by reflecting the proportion of diagnostic scan time (DiagnosticScanTime) in the total examination time (ExamDuration) or the scan time adjusted for pre- and post-processing procedures (ScanDuration).

An aggregation of the parameters at the daily level and log level was available in each case. Usage analysis employed frequency of protocols used from a predefined pool of approximately 140 protocols stored in the scanner. On the one hand, the 5 most frequently used protocols were searched for, and on the other hand, an analysis determined how many n protocols were used at least n times. In addition, the anatomical location of the examination protocols was categorized as craniocervical, cardiothoracic, abdominopelvic, extremity, strictly vascular, and spinal. The “Other” category contained logged protocols anonymized during the export from the scanner. Analysis of age relationships was based on age-related results shown in [Fig. 3].

Zoom Image
Fig. 3 Age distribution of the patient collective. Boxplots depict the number of exams in each age group which is also given as absolute number in each box. Black bar graphs indicate the average exam duration for every age group with error indicator indicating the standard deviation. Asterisk (*) indicates statistical significance in comparison to all other groups, + indicates significance in comparison to age groups 12–30, 31–40, and 51–60, o indicates significance in comparison to age group > 80.

#

Statistics

Data are shown as mean ± standard deviation with indications of minimum and maximum [Min; Max] as well as median. SPSS 26 (IBM, USA) was used for statistical evaluation. A Mann-Whitney U test was used to compare independent random samples (e. g. frequently used protocols vs. other protocols), since Shapiro-Wilk and Kolmogorov-Smirnov tests rejected a normal distribution. In comparing non-normally distributed data with grouped data such as age cohorts, pairwise testing was used using a Kruskal-Wallis test, which included a Bonferroni correction for repeated testing.


#
#

Results

Cohort

During the observation period 09/2018–02/2020, all examinations using the scanner were continuously submitted for evaluation. After exclusion due to low duration of use, data from 357 working days were considered. Data were available from n = 3936 studies; n = 277 were excluded: Duration < 15 min n = 68, duration > 75 min n = 44, scientific studies, etc. n = 165. This resulted in n = 3659 studies included in the statistical assessment. Overall patient age was 56.3 ± 21.8 years (median 60.5 years), including 50.4 % women, 49.6 % men with a weight of 76.0 ± 20.7 kg (median 76 kg). The grouped age distribution is shown in [Fig. 3].

The examinations were distributed among the following regions: abdominopelvic (1534; 41.9 %), extremity (736; 20.1 %), spinal (510; 13.9 %), cardiothoracic (370; 10.1 %), and craniocervical (292; 8.0 %); see also [Fig. 4]. Purely vascular examinations were less frequent (108; 3.0 %). The distribution of protocols also showed that of the 140 protocols predefined on the scanner, 30 protocols were used at least 30 times during the assessment period and 63 protocols were used more than 10 times.

Zoom Image
Fig. 4 Regional distribution of exam protocols and comparison of protocol frequency with the average ExamDuration. Except for the abdominopelvic and craniocervical regions and extremity protocols, all differences revealed statistical significance.

#

Daily-aggregated analysis

Daily-aggregated data provide an overview regarding scanner usage. On average, 11.1 ± 2.2 (median 11) examinations that met the inclusion criteria were performed per day. The scanner was operated for an average of 10.4 ± 1.5 hours per day (median 10.8). Table A1 shows an overview of the aggregated key figures of the scanning operation. The daily and total averages behave in the same way. The operating times on Fridays were lower (see [Fig. 5]), corresponding to the shorter operating times, but showed a wider spread with optional emergency operation. While the nominal operation time Monday to Friday allows a total scan time of 14 hrs, the actual operating time (OperationTime) is below this value, which is particularly due to excluded examinations.

Zoom Image
Fig. 5 Analysis of daily averaged processes during the average ScanDuration per day in relationship to the OperationTime (grey boxplots) and the possible usage time of the scanner (grey hatched bar).

#

Analysis on the protocol level

Analysis on the protocol level allows evaluation of parameters characteristic of one or more protocols. For this purpose, the five most frequently used protocols were analyzed according to the process parameters presented in [Fig. 2] and compared with the entirety of the examinations and the other protocols. Table A2 provides an overview of the related values. While the mean and median values of the examination parameters varied only slightly in the context of the scanner processes, these differences showing a tendency towards better optimized Top 5 protocols demonstrated statistical significance in some cases. The sometimes considerable variance of the values within the respective protocols, expressed by minimum and maximum measured values, should be emphasized.

With respect to the age distribution, the trend of slightly longer ExamDurations in the 0–3 and 4–10 year olds compared to the age groups up to 80 years shown in [Fig. 3] can also be appreciated in the ScanDuration and TotalDiagnosticScanTime. However, the differences in the other parameters showed less obvious trends. On average an additional scan was performed for the 0–3 and 4–10 year-olds, and one scan was repeated.

Considering protocol regions, [Fig. 4] shows a clearly extended ExamDuration for cardiothoracic and for the protocols subsumed under Other. A higher number of scans and repeated scans could be shown for scans of both regions. As can be seen in [Fig. 6], these protocols mainly show longer planning times and IdleTimes. This observation is especially true for the vascular protocols, where the ratio of actual scan time to planning time and IdleTime is even less favorable. The reason for this is probably the waiting time between time-resolved MR angiography of the lower leg and static MR angiography of the pelvic and leg arteries in order to filter the contrast medium renally.

Zoom Image
Fig. 6 Visualization of scan processes during the ScanDuration averaged over individual imaging regions. For color-coding please refer to [Fig. 4].

Efficiency analyses are of particular interest for process optimization. While EffScan averaged 90.6 ± 7.4 %, the Effgesamt of 69.6 ± 17.6 % indicates the potential for optimization in the area of positioning and scan preparation. Table A2 provides an overview of the efficiencies for the frequent and the less common protocols. There was a marked spread between the minimum and maximum values. With respect to age categories, Effgesamt in the Kruskal-Wallis comparison showed no significance of the small differences. In contrast, the differences in Effscan between age groups revealed the differences with statistical significance as depicted in [Fig. 7]. The fact that the differences between the reduced EffScan in the group of 71–80 year olds and the other values were not statistically significant can be explained by the wide range of results in the groups of older patients.

Zoom Image
Fig. 7 Depiction of the Scan-efficacy EffScan for each age group. Asterisk (*) indicates differences with statistical significance of age group 21–30. Also, the comparison of age groups 31–40 and > 80 years revealed statistically significant differences.

The scan efficiency related to the regional assignment of the protocols is shown in [Fig. 8]. The Kruskal-Wallis test demonstrated statistical significance of the Effgesamt and Effscan scan efficiencies except for the abdominopelvic/craniocervical comparison.

Zoom Image
Fig. 8 Depiction of regional distribution of scan protocols and Scan-efficacy (Effscan). All comparisons but the comparison abdominopelvic vs. craniocervical revealed statistical significance.

#
#

Discussion

We successfully introduced an automatic log file-based process analysis using data from a clinical 1.5 T MR scanner. The analysis and work with the data is made possible by an automatically-generated daily evaluation and visualization of the results set up in PowerBI, which was reproduced here according to scientific-statistical criteria using SPSS employing the identical data. Daily aggregated and protocol-related usage analyses were presented as part of the detailed process analysis in the area of scanner usage.

Log files have previously been used elsewhere to assign protocols to body regions [6] and to identify prolonged examinations and to monitor optimization approaches in which, for example, protocols were shortened [7] [8]. No further analyses of these data were carried out, which may be due to their non-automated generation. The log file analysis can also be expanded from a process analysis to a controlling instrument if the information available from the log files were to be combined with billing codes from corresponding systems [9]. In this study, however, the focus was on process analysis and the identification of processes to be optimized. In this context, the data can also be used as a basis for process optimization procedures [2], such as measures already introduced to improve MRI procedures based on the Lean Six Sigma program [10] [11] [12] [13] which seeks to identify and avoid inefficient work steps and reduce variability in the remaining working procedures. Initial verification approaches to data from other scanners show that this methodology is suitable for making changes, e. g., to check their effect on the sequence concept as a before/after analysis using various parameters [5]. While the relevance of the selected parameters presented in this feasibility study or of the concept of efficiency remains unanswered, further corroborating studies are necessary to define the significance of individual parameters based on relevant issues and to put them to use.

Our concept stands out from other usage analyses, which are limited to reading the time stamp of the DICOM header of archived image data. The multidimensionality of data achieved only through log file analysis represents both an opportunity and a challenge. On the one hand, the data can be summarized in many dimensions, as demonstrated here as an example on the time axis for scanner usage analysis using day-aggregated statements and on the protocol axis for detailed sequence analysis of the examination. This also includes the option, not yet shown, to evaluate patients or sequences based on their characteristics. In particular, sequence-oriented analysis, e. g., allowing conclusions to be drawn about the frequency of changes to sequences is very promising for process optimization, problem identification, or employee training. If it can be shown that a sequence is frequently adapted and the IdleTime or examination time is prolonged, problem-centered remedial action can be taken or retraining can be provided. The database structure allows identification of even individual processes or process steps at any time, which is not possible with the evaluation of the time stamps of the DICOM data alone.

At the same time, these possibilities pose a challenge to the form of the presentation and use of results. A purely exploratory approach runs the risk of lack of focus. From the authors’ experience, predefined evaluations tailored to specific issues, such as process analysis, are useful, as they benefit from the constantly updating data within the database structure, thus allowing a daily updated evaluation or one which can be limited to freely-selectable time frames, thereby providing a comparative assessment [5].

Such a system could fully demonstrate its strengths in combination with information from the hospital information system (HIS) and with the integration of infrastructural environment variables [14]. Information about the patient, e. g., mobility, the need for support, and the ability to communicate would enable an expanded data structure that could also be used to achieve process projections. If these variables and the daily data situation are combined with concepts of artificial intelligence, projections of the expected process flow or self-learning scheduling appear possible.

The fundamental basis of a feasibility study makes it subject to numerous limitations. An SPSS re-evaluation supports the scientific requirement of assessment and validating statistics, resulting in a detour not necessary for practical purposes. However, it is indirectly able to validate the results that can be more easily and directly visualized using PowerBI. In principle, PowerBI is also able to integrate statistical calculations. Likewise, prior manufacturer-specific development can be seen as a limitation. Transferring the concept to scanners from other manufacturers can be implemented in principle, but requires adaptation and successful handling of proprietary data formats. The data used here for demonstration purposes from a single scanner in routine clinical practice provides a picture of specific scanner utilization that cannot be generalized. Filters were used to narrow the data, resulting in a simplified representation within the feasibility study that does not reflect the requirements or conditions of, for example, a practice or optimized outpatient clinic operation or other clinics.

In summary, based on the log file analysis of one MRI scanner, we present a concept that forms the basis for further development steps, promising new possibilities in process analysis that can be tailored to the individual user or issue. In this way, questions from the point of view of the operator, application specialist but also the development department can be answered while optimally generating added value at every level.


#
#

Conflict of Interest

Joachim D. Schmidt, Jörn Borgert, and Thomas Amthor are employees of Philips GmbH.

Ergänzendes Material/Supplementary Material


Correspondence

Prof. Alex Frydrychowicz
Department of Radiology and Nuclear Medicine, University Hospital Schleswig-Holstein, Campus Lübeck
Ratzeburger Allee 160
23562 Lübeck
Germany   
Phone: +49/4 51 50 01 70 12   

Publication History

Received: 31 August 2020

Accepted: 22 December 2020

Article published online:
03 February 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany


Zoom Image
Fig. 1 Schematic of the daily updated, automated data processing chain of the log file analysis.
Zoom Image
Fig. 2 Schematic visualization of the process analysis and its terminology. Depicted time lines become more detailed in descending order. Dashed lines depict magnified illustrations of respective sections within the exam. For definitions of used terms please refer to the appendix.
Zoom Image
Fig. 3 Age distribution of the patient collective. Boxplots depict the number of exams in each age group which is also given as absolute number in each box. Black bar graphs indicate the average exam duration for every age group with error indicator indicating the standard deviation. Asterisk (*) indicates statistical significance in comparison to all other groups, + indicates significance in comparison to age groups 12–30, 31–40, and 51–60, o indicates significance in comparison to age group > 80.
Zoom Image
Fig. 4 Regional distribution of exam protocols and comparison of protocol frequency with the average ExamDuration. Except for the abdominopelvic and craniocervical regions and extremity protocols, all differences revealed statistical significance.
Zoom Image
Fig. 5 Analysis of daily averaged processes during the average ScanDuration per day in relationship to the OperationTime (grey boxplots) and the possible usage time of the scanner (grey hatched bar).
Zoom Image
Fig. 6 Visualization of scan processes during the ScanDuration averaged over individual imaging regions. For color-coding please refer to [Fig. 4].
Zoom Image
Fig. 7 Depiction of the Scan-efficacy EffScan for each age group. Asterisk (*) indicates differences with statistical significance of age group 21–30. Also, the comparison of age groups 31–40 and > 80 years revealed statistically significant differences.
Zoom Image
Fig. 8 Depiction of regional distribution of scan protocols and Scan-efficacy (Effscan). All comparisons but the comparison abdominopelvic vs. craniocervical revealed statistical significance.
Zoom Image
Abb. 1 Schematische Darstellung der automatischen, tagesaktuellen Datenverarbeitung der Logfile-Analyse.
Zoom Image
Abb. 2 Schema der Begrifflichkeiten der Logfile-basierten Prozessanalyse an einem MRT. Die hier gezeigten Zeitpfade werden von der ersten zur dritten Zeile zunehmend detaillierter und entsprechen den durch gestrichelte Linien angezeigten Bereichen. Definitionen finden sich im Anhang.
Zoom Image
Abb. 3 Altersverteilung im untersuchten Kollektiv. Begleitend ist beispielhaft die altersabhängige Gesamtuntersuchungszeit (Exam Duration) in Minuten aufgezeigt. * zeigt statistische Signifikanz gegenüber allen anderen Altersgruppen an, + zeigt statistische Signifikanz gegen Altersgruppen 12–30, 31–40 und 51–60, o zeigt statistische Signifikanz gegenüber Altersgruppe > 80 an.
Zoom Image
Abb. 4 Regionale Verteilung der Untersuchungsprotokolle und Vergleich der Häufigkeitsunterschiede mit den durchschnittlichen Untersuchungszeiten. Mit Ausnahme der Vergleiche unter den Lokalisationen abdominopelvin, Extremitäten und kraniozervikal zeigten alle Unterschiede statistische Signifikanz.
Zoom Image
Abb. 5 Analyse der innerhalb der Scan Duration ablaufenden Prozesse in Bezug zur tatsächlichen Betriebszeit (Operation Time; graue Boxplots) und der maximal möglichen täglichen Betriebsdauer des Scanners, jeweils als Mittelwerte der Tagessummen für die verschiedenen Wochentage.
Zoom Image
Abb. 6 Darstellung der innerhalb der Scan Duration ablaufenden Prozesse bezogen auf die 7 definierten Regionen. Die Farbkodierung entspricht [Abb. 4].
Zoom Image
Abb. 7 Darstellung der Scaneffizienz EffScan bezogen auf die Altersgruppen. Unterschiede mit statistischer Signifikanz ergaben sich im Vergleich der Gruppe 21–30 Jahre zu den mit * markierten Werten und zwischen den 31–40 und den > 80-jährigen Patienten.
Zoom Image
Abb. 8 Häufigkeit und Effizienz EffScan der Protokollregionen. Mit Ausnahme des Vergleichs abdominopelvin/kraniozervikal weisen alle Unterschiede zwischen den Scaneffizienzen statistisches Signifikanzniveau auf.