Automated detection of changes in patient exposure in digital projection radiography using exposure index from DICOM header metadata

Abstract

Purpose. Automated collection of image data from DICOM headers enables monitoring of patient dose and image quality parameters. Manual monitoring is time consuming, owing to the large number of exposure scenarios, thus automated methods for monitoring needs to be investigated. The aim of the present work was to develop and optimise such a method. Material and methods. Exposure index values from digital systems in projection radiography were collected over a period of five years, representing data from 1.2 million projection images. The exposure index values were converted to detector dose and an automated method for detection of sustained level shifts in the resulting detector dose time series was applied using the statistical analysis tool R. The method combined handling of outliers, filtering and estimation of variation in combination with two different statistical rank tests for level shift detection. A set of 304 time series representing central body parts was selected and the level shift detection method was optimised using level shifts identified by ocular evaluation as the gold standard. Results. Two hundred and eighty-one level changes were identified that were deemed in need of further investigation. The majority of these changes were abrupt. The sensitivity and specificity of the optimised and automated detection method concerning the ocular evaluation were 0.870 and 0.997, respectively, for detected abrupt changes. Conclusions. An automated analysis of exposure index values, with the purpose of detecting changes in exposure, can be performed using the R software in combination with a DICOM header metadata repository containing the exposure index values from the images. The routine described has good sensitivity and acceptable specificity for a wide range of central body part projections and can be optimised for more specialised purposes.

Quality control in digital projection radiography has become increasingly challenging, owing to the design of modern devices and the rapid development of information technology infrastructures. The large dynamic exposure range of the detector system makes changes in exposure levels difficult to detect in clinical images. The exposure index (EI) is a relative value describing the dose to the detector, and has been proposed as an indicator of the general dose level for the actual projection, given that the exposure settings of the projection do not change over time [1]. Moreover, the EI enables detection of projections with low exposure levels that lead to degrading noise [2]. As different equipment and projections generate different patterns of EI values, there is a need to monitor a large number of scenarios for changes, generally the number of projections multiplied by the number of detector systems in use. For any large radiology institution this amounts to several thousand sets of individual EI data sets that can be interpreted as time series, as the data is organised in chronological order and evenly distributed over time. As indicated by other authors [3], there is a need for rational acquisition and storage of data, which in turn facilitates automated monitoring. The EI value is available in the digital DICOM (Digital Imaging and Communications in Medicine) image for flat panel detectors and imaging plates, and can be utilised rationally using open source software [4].

The purpose of the present work was to investigate whether changes in EI values for digital projection radiography equipment could be detected in an automated workflow, and to develop and optimise an automated level shift detection routine for use in different types of clinical scenarios. The focus was on the detection of sustained changes in patient exposure level for central body parts.

Material and methods

Automated collection of EI values

Exposure index values were obtained from January 2005 to May 2010 from an automated DICOM metadata collection workflow that gathered all metadata from the regional radiology departments in the county of Dalarna in Sweden, populated by 277 000 inhabitants. The collection was fully automated and the EI values were stored in a database [4]. Normal quality control in accordance with national regulations was performed annually during the period by experienced medical physicists. The EI values were grouped according to projection and sorted in chronological order for each examination and detector, forming 3365 time series containing approximately 1.2 million EI values.

All the time series representing examinations covering neck, thorax, abdomen, pelvis and hips were selected as the test group, which resulted in 615 time series containing EI values from 540 000 images. For reasons of calculation time constraints, half the selected time series were randomly chosen for inclusion and these EI values was converted to relative detector dose [2]. The conversion was performed to simplify the ocular evaluation, as the appearance of the EI time series otherwise would be influenced by the different mathematical transformation made by the vendors of the equipment.

Ocular evaluation

The time series included in the study were evaluated by two medical physicists with the instruction to indicate level changes in the detector dose time series. Before the evaluation began a benchmarking session was held, in which different types of changes were discussed and a consensus on what should be regarded as interesting from a patient dosimetry or image quality point of view was reached. The time series were reviewed by ocular evaluation from a plot of the EI values converted to detector dose versus the chronological order number of each image in each time series, aided by zooming and panning. The locations of the level shifts detected by ocular evaluation were inserted in a results table in the metadata repository and later used in the calculation of sensitivity and specificity.

During the evaluation the changes were classified as abrupt or diffuse, the latter being defined as a tendency towards gradual changes in detector dose value over time. Changes that occurred at the beginning or in the end of the time series were excluded from further evaluation in order to eliminate technical errors in the analysis.

Development and optimisation of the automated level shift detection routine

The set of time series was analysed using an automated detection routine, based on an algorithm combining filtering and statistical analysis. The routine was optimised by testing all combinations of selected statistical processing variables with the ocular evaluation as the gold standard. Sensitivity and specificity of the routine were calculated for each combination of statistical processing variables and the average sensitivity and specificity for all the evaluated time series were then calculated for each set of processing variable combinations. The most favourable combination of average sensitivity and specificity was selected, with the constraint that the average specificity for all the time series should be at least 0.995, as this would give a number of false positives assessed as manageable in the clinical workflow. The comparison was made by calculating true positives from overlapping registrations of the ocular evaluation and the automated analysis with a window of ±20 detector dose values around every level shift detected by ocular evaluation.

The ability to detect level shifts was compared for two different rank test methods. As the detector dose or EI was not generally normally distributed, care had to be taken to choose analysis methods that were valid under this presumption. In the first stage the analysis package Robust filter including a median test was used [5,10]. In the second stage a standard one-sided Wilcoxon test [10] was used instead of the median test.

Analysis using Robust filter and the median test. The Robust filter package [5], available as a packet in the R statistical tool, was used to perform the analysis in three steps using a moving window in a detector dose time series. Firstly, when analysing the last point in the analysis window somewhere in the time series, the detector dose value could be regarded as an outlier and action could be taken to prevent or moderate the influence of this value on further analysis. Secondly, if the value was accepted, a filter was applied that estimated the signal level in the analysis window. Finally, the median test was applied where the levels of the detector dose values in the rightmost part of the analysis window were compared with the ones in the left part of the same window. When comparing, the variation within the window was taken into consideration, discriminating level changes that were small as compared with the variation. A change in signal level was detected by the median test when more than half of the residuals in the rightmost part of the window were larger or smaller than a constant times the estimate of the scale [6].

Optimisation of statistical processing variables and procedure. The possible combinations of the selected processing variable values [5] were tested in loops using the statistical tool R. The number of variables used was limited by calculation time and was chosen from a practical point of view, taking the clinical characteristics of the time series into consideration. Two different widths on the analysis window were chosen, 20 and 40, and the approximation of the signal level was made with all the possible choices in the package: median filtering, repeated median regression, least trimmed squares regression and least median of squares regression [5]. Two different methods for scale estimation were chosen for the Robust filter and median test: median absolute deviation [5] and the Q_n scale estimator [6]. Detection of outliers was based on the estimated variation in the analysis window and an outlier was recognised when the detector dose value was outside the signal level ± a multiple of the estimated scale, σ. The alternatives used were to replace outliers outside the signal level ± 3σ with the current level estimate or to shrink outliers outside the signal level ± 2σ towards the estimated signal level, replacing the outlier with the signal level ± 2σ [5]. The alternatives 1σ and 2σ was used as a rule for shift detection and the regulation of the analysis window with regard to adaptability to more certain changes was tested in two variables containing two values each [5].

Analysis using Robust filter and Wilcoxon rank sum test. As indicated by other authors [7], the Wilcoxon rank sum test can be used to increase the resistance of the statistical analysis, i.e. to decrease the number of false positives, when detecting level changes in a time series. The Wilcoxon rank sum test was therefore added to the analysis, replacing the abovementioned median test. The detector dose time series were processed with the optimised statistical processing variables resulting from the first stage where the median test was used. It was assumed that the statistical processing variables with the best performance for the median test method would also give the best performance for the Wilcoxon rank sum test. The Wilcoxon test was performed, using the scale estimator Q_n [8] in combination with a constant, on the previously derived median filtered time series. Using a moving window technique the samples tested in the Wilcoxon test were moving adjacent windows in the time series of 20 detector dose values each (Figures 1 and 2).

Figure 1. Detector dose time series and median filtered signal with analysis windows A and B. Excerpt from lumbar spine frontal (Fuji XG-1).

Figure 2. Detector dose time series with the signal level in analysis window B, L_B, decreased by a factor kσ, where k is a constant and σ is the mean value of the variation in analysis window B as estimated by Q_n.

An upward change was indicated if the filtered values in the first window (window A, Figure 2), according to the Wilcoxon test, were considered smaller than the values in the second window (window B, Figure 2) decreased by a constant multiplied by the mean value of Q_n in the second window.

A downward change in the time series was tested accordingly, adding a constant multiplied by the mean value of Q_n in the second window to the values in the second window and testing whether the values in the first window were larger than the values in the second.

The output from the one-sided Wilcoxon test was the p-value as a function of image # (Figure 3). Together with an optimal p-value, indicating level shift, the constant deciding the desired influence of the scale estimate, Q_n, on the detection threshold had to be determined in the second stage of the optimisation process. The Wilcoxon p-values selected were 10⁻¹, 10⁻², 10⁻³, 10⁻⁵ and the scale influence was tested using the values 0.5, 0.7, 1.0, 1.5 as multiples of the mean value of Q_n.

Figure 3. Variation of Wilcoxon p-value in a detector dose time series with an abrupt change around image #80 in lumbar spine time series.

Results

Ocular evaluation

The evaluation group contained 304 time series. A total of 281 level shifts, in 146 time series, were detected by ocular evaluation. One hundred and fifty-eight time series were considered to be without level shifts. Two hundred and four of the level shifts were considered abrupt, 50 diffuse and 28 were excluded as being in the vicinity of the end or beginning of the time series. The sizes of the changes were in the range of 5–100%, the larger ones generally being the result of dose optimisation projects. Taking the different characteristics of the time series into consideration, changes below 5–10% of the general level of the detector dose time series were difficult to ascertain.

Analysis using Robust filter and the median test

The optimisation consisted of 256 statistical processing variable combinations for each time series. Two hundred and eighty-six time series, of the original 304, were successfully processed; time series with fever than 50 detector dose values were excluded from the optimisation.

The Robust filter and median test gave optimal result for a window width of 20 detector dose values in combination with the median filter, the Q_n scale estimator, the 2σ rule for detecting and shrinking outliers and a 1σ rule for detection of changes. For this combination, the average sensitivity/specificity was 0.700/0.995 over all the time series. The adaptive features of the package were of relatively little importance to the outcome.

Analysis using Robust filter and Wilcoxon rank sum test. For the Robust filter and Wilcoxon test a p-value of 0.1 in combination with the constant 0.7 multiplied by the mean value of the scale estimator Q_n gave the highest specificity, 0.997, with slightly improved sensitivity, 0.730, as compared with the median test. When diffuse changes and changes at the beginning or end of the time series were excluded, the sensitivity/specificity increased to 0.870/0.997

Discussion

Exposure index values have been questioned as indicators for patient exposure [11]. One of the problems concerns a lack of standardisation in the actual values; different vendors use different principles to calculate the EI value from the detector dose [2]. Another problem is a general shortage of tools to make the vast amounts of data comprehensible. The present study shows that the behaviour of EI time series is predictable to an extent where an automated detection of level change is possible, assuming that the time series are organised in projections for each detector system. Under these circumstances an ocular evaluation is possible from a conceptual point of view, but unfortunately not from a practical one as the different exposure scenarios are too numerous. The methodology used has been suggested for intensive care monitoring where the datasets share the same characteristics as the datasets in the present study [12].

Radiographic methodology and validity

The objective of the present study was to cover different aspects of the clinical workflow, disregarding the fact that a mix of Automatic Exposure Control (AEC) and manual exposure settings were in use for individual projections. As reported by other authors [13], the entrance dose for different projections reveals characteristics related to the radiographic methodology, for example the usage of AEC or manual exposure settings. In the present study many projections showed detector dose values with stable levels and small variations, typically large projections of the chest or abdomen with AEC. Other projections, with a mix of AEC and manual exposure settings, exhibited more random behaviour. Using the same manual exposure parameters for all patients on a digital detector generates EI values that reflect the thickness of the patient's body, not the optimised detector dose [14]. The general extent of the problems in connection to changes in radiographic methodology, deliberate or not, is relatively unknown according to evaluations from the Swedish Radiation Safety Authority [15]. The result of the present study, and others [13], indicate that radiographic methodology evolves over time for a new installation and that a relatively long learning period (years) might be anticipated before exposure indicators stabilises on reasonable levels with acceptable variations over time. In the present study, the detector dose values also exhibited level shifts after the initial learning period.

On introduction in the clinical environment, there should be a simple possibility to tailor the analysis for projections that behave in a way that decreases overall sensitivity and specificity. Other possibilities could be to limit the analysis to projections with AEC, if this is the purpose of the monitoring, or to group the time series on the basis of whether or not AEC is in use. This information is normally only available in the image header for flat panel detectors, not for imaging plates [16].

In the present study, the Wilcoxon test increased the performance of the level shift detection routine as compared to the median test. This is believed to be a general finding supported by other authors [9]. It must be pointed out that the optimal configuration of the statistical processing variables may be dependent on e.g. examination type. Thus, the result obtained in the present work may only be valid for EI time series covering central body parts. Furthermore, taking local variations in radiographic methodology into consideration, the statistical processing variables might need further tuning before implementations outside the included institutions. However, the main finding of the present work is that it is feasible to employ an automated level shift detection routine and that it can reach a high level of performance after optimisation.

Statistical considerations

The optimisation resulted in a preference for the median filter to establish the signal level. Compared with averaging filters, the median has high resistance to outliers [6]. Moreover, the results indicate that trends are not the problem, as the median filter deteriorates in trend periods as compared to the other filter methods tested [12].

Outliers represent large deviations from normal exposure or the inability of the systems to calculate an EI value that is consistent with the detector dose [11]. A decision should be made as to whether the objective of the analysis is to include as much influence of the outliers as possible or to detect deviations from the normal exposure level. In the latter case, it might be an alternative to remove the outliers completely. Another challenge regarding the relatively abundant number of outliers is the need for a robust method to estimate the local variation in the time series as the scale estimate has an impact on the detection threshold.

The Wilcoxon test and the median test are both non-parametric methods, well suited for analysis of variables with unknown distribution [10]. The number of detector dose values in the adjacent test windows affects the delay of the change in p-value: the larger the window size, the longer the delay before the rank sum of the test reaches the critical level, represented by the p-value from the Wilcoxon test. On the other hand, the specificity of the test decreases when smaller test groups are used [6]. In selecting the size of the test groups, the clinical consideration should concern how many patients could be examined with non-optimal exposure in contrast to the workload involved in handling a large number of false positive signals from the test. Different ways of determining the optimal size of the test group has been proposed [7].

As the one-sided Wilcoxon test option was used it was possible to separate an increase from a decrease. This fact could be of use if correlations to other parameters, such as the dose-area product, in the whole metadata set were to be investigated with the purpose of further enhancing sensitivity and specificity.

Online implementation

For implementing the automated analysis online, it is simplified if the EI values are stored in a database. Methods for the collection of DICOM header metadata have been described [4,17,18]. The collection method used in this work is implemented using dcm4che [19,20], a collection of open source applications and utilities implemented in the Java programming language. Alternatively, the time series can be collected using modality performed procedure step [16] or other standards. With a general solution for DICOM metadata storage several other parameters could be of concern, such as dose-area product from projection radiography systems, exposure parameters from computed tomography systems or average glandular dose from mammography units. As the statistical processing variables are flexible, adaptation to the specific data set can be performed.

The R software is an open source statistical tool with a large number of packages of different focus, such as the Robust filter package, and it is easily interfaced with the MySQL database used in the present work. Extraction of data, transfer and calculations can be automated and the desired monitoring of about 700 projection scenarios, to ensure stable exposure conditions for exposure of central body parts, can be achieved using modest hardware.

In conclusion, the results of the present study show that an automated analysis of EI values, with the purpose of detecting changes in patient exposure, can be performed using the R software in combination with a DICOM header metadata repository and that the routine described has good sensitivity and acceptable specificity for central body parts. The Wilcoxon rank sum test improves sensitivity and specificity as compared with the median test. The sensitivity and specificity can be expected to be higher for changes that can be considered abrupt and the resulting p-value from the Wilcoxon test makes it possible to adapt the sensitivity and specificity to the general behaviour of the time series at hand. The criteria for change detection can be altered using different thresholds for the Wilcoxon p-value, as well as different scale constants.

The workflow described in the present work has been implemented in the clinical production, and a follow-up study with the objective of further optimising the procedure is planned.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper