Clinical Decision Support System (CDSS) in primary care: from pragmatic use to the best approach to assess their benefit/risk profile in clinical practice

Abstract

Clinical Decision Support Systems (CDSSs) are computer-based tools intended to support physicians in clinical decision making. MilleDSS is an illustrative example for the Italian context. It is featured by four domains of GP-software interaction, such as clinical management and follow-up evaluation, prescribing appropriateness and clinical risk, prevention strategies and medical computerized stewardship on scientific update and training. MilleDSS registered 23,222 accesses in early September 2021. In specific, the sections on prevention and training were viewed 19,440 and 21,797 times, respectively.

The Medical Device Regulation (MDR: (EU) 2017/745) indicates that clinical evidence needs to be provided for any software intended to medical purpose. Clinical research on CDSS effectiveness will be therefore conducted through epidemiological studies. In theory, this generation of evidence would follow the pyramid of evidence as per medications approval but, given the large use and constant update of CDSS for daily clinical practice, attentions should be posed on the most cost-effective study.

Keywords:

• CDSS
• computer-based tool
• primary care
• clinical research

Clinical Decision Support Systems (CDSSs) are software-based tools intended to support physicians in clinical decision making. They are commonly administered using electronic healthcare records (EHR) interacting with other clinical workflows, which have been made easier by the growing adoption of EHR over the last decades¹. In essence, several patient’s characteristics are matched to a computer-based clinical knowledge so notifying physicians on patient-specific actions to be taken^2,3. There are several CDSS intended to primary care physicians, given that General Practitioners (GPs) have a multidimensional approach to their patients, who are usually over the sixties, suffer from concurrent chronic diseases and are exposed to polypharmacy. In this context, being compliant with evidence-based clinical practice guidelines (CPGs) may be heterogenous and unsatisfactory, so leading to suboptimal disease management. CDSSs operating on long-term follow-up are what mainly fits the GPs’ needs. However, the need of being compliant to Medical Device Regulation (MDR) raises several questions for public health authorities, clinicians, and researchers. Therefore, this manuscript aims to discuss an Italian experience regarding CDSSs and how to assess their evidence-based implementation according to the current MDR.

Indeed, there is a number of functions provided by CDSS for GPs, including diagnostics, prevention, clinical management and risk, prescribing appropriateness and much more^2,4,5. All of them can alert physicians through reminders, data report on patient’s conditions, computerized guidelines and further questionnaire templates. It has been demonstrated that the use of CDSS enhance the adherence to official clinical guidelines, application of prevention and public health strategies, improve patient’s safety with reduction of prescribing inappropriateness and/or errors, favor cost containment with reduction of order duplications and suggesting cheaper medications, facilitate administrative functions using auto-fills, generally improve any workflow, and ease the identification of subjects potentially eligible for clinical studies⁵. On the other way round, CDSSs are not free of risk. Certain phenomena, such as alert fatigue due to too many alerts might impair the GP’s interest⁶. Of note, 49–96% of alerts are known to be disregarded⁷. Similarly, excessive trust on the automatic system or GPs’ distrust for the computerized contents, which might be insufficiently updated, could compromise the efficacy of the CDSSs leading to misinterpretation and/or medication errors. Finally a fragmentation of the workflow due to contents external to EHRs (i.e. multi-item assessment to be filled out, such as Rockwood frailty evaluation)⁸ might be excessively demanding, and therefore reduce the access to CDSS-related utilities^4,9. That being said, any safety concern should be weighed against CDSS-related benefits. The benefit/risk profile should be proven for Medical Device Software (MDSW) as public health authorities require for pharmacological therapies. As for each type of an intervention, in addition to efficacy and safety, benefit/risk profile needs to consider other aspects, such as costs and patient’s preference. In this context, the selection of outcomes is a critical point, and it depends on the selected perspective, namely on the perspective of GP or patient. As example, a CDSS might be tested by evaluating the incidence rate of medication/prescribing errors and the occurrence of adverse drug reactions.

For this reason, MDSW, as CDSSs embedded in GPs’ software, need to demonstrate appropriate clinical utility to fulfil the requirements set out in Regulation (EU) 2017/745, the Medical Device Regulation (MDR). In other words, when CDSS is part of a software for which the manufacturer claims a specific medical purpose, a Clinical Evidence Report (CER) must be part of any conformity assessment. To produce a CER, three main sources of data could be included: (i) studies published in medical literatures concerning similar CDSSs with which to perform systematic reviews and meta-analyses; (ii) still unpublished internal data gathered by experimental studies; (iii) evidence stemming from post-marketing real-world investigations. Nevertheless, these resources are featured by pros and cons given the existing use and diffusion of CDSS, which also impose ethical considerations.

In this respect, we would like to discuss the Italian experience of MilleDSS (Genomedics Srl - Florence, Italy), a series of CDSSs embedded in Millewin, which is the software with highest number of users among Italian GPs. Of note, the use of software is mandatory for Italian GPs as in other countries. Namely, more than 17,000 GPs have a Millewin (produced by Millenium SRL, Dedalus Group) subscription, so having the opportunity to access the MilleDSS functions.

MilleDSS is featured by four domains, including clinical management and follow-up evaluation (composed by three sub-domains: timely controls, expirations, target to reach), prescribing appropriateness and clinical risk evaluation, prevention strategies (composed by predictive scores and supports for primary and secondary prevention) and medical computerized stewardship on scientific update and courses (continuously updated). As an example, there are reminders for a timely assessment of measurements, such as creatinine (kidney functions), TSH (Thyroid-Stimulating Hormone), or BMI according to their last available registration. Similarly, MilleDSS contains functions aimed to evaluate prescribing appropriateness. In essence, the system is able to alert GPs in case of reimbursement eligibility for certain drugs, such as Proton Pump Inhibitors (PPIs) and statins according to individual patient risk, as well as the potential presence of drug-drug interactions for elderly patients as per Beers and START&STOPP criteria¹⁰. Finally, prevention strategies are implemented via CDSS through prediction scores developed and validated in the Italian primary care setting. For example, they automatically calculate the risk of osteoporotic fractures over 5 or 10 years of follow-up (i.e. FRActure-HS index)¹¹, the risk of TEV events (TEV-HS Score)¹² or of COVID-19-related complication (HS-CoVId) over 3 or 1 month from the baseline assessment, respectively¹³.

Figure 1 shows an example of dashboard, as viewed by GPs, depicting the four MilleDSS domains on 8th September 2021, MilleDSS registered 23,222 accesses. Among them, the sections on prevention and training were visualized 19,440 and 21,797 times, respectively, so demonstrating a very high complacency by GPs. This is a further proof of the necessity, among GPs, to adopt CDSS in their practice given the growing complexity of their patients and the related workloads.

Figure 1. Example of MilleDSS dashboard as viewed by general practitioners in their software Millewin. Follow-up, prescribing appropriateness, and prevention domains are visualized. As an example, clicking on the reminder concerning the Asthma Control Test, an informative balloon provides additional details.

The fact that MilleDSS must fulfil MDR regulation and thoroughly provide clinical evidence on its effectiveness, raises several questions for public health authorities, clinicians and researchers. CDSSs are indeed already adopted in primary care and their clinical benefit should be proven following the pyramid of evidence. On the top of this, there are Randomized Clinical Trials (RCTs) which surely remain the methodological standard for new CDSSs. Nevertheless, software developers and companies are questioned by GPs to update and improve CDSSs almost on daily bases. As stated above, a relevant number of GPs access CDSS and the fact that other add-ons are continuously implemented, the demonstration of their clinical utility via RCTs should require relevant efforts, time and resources. Obviously, frequent repetitions of data analysis might be required according to the specific clinical contexts, namely those requiring “slow” (e.g. guidelines on diabetes management) or “quick” amendments (e.g. Sars-CoV-2 pharmacotherapy).

One or more control groups are (no DSS “as usual care”) needed while the use of software is mandatory for GPs, so introducing ethical concerns on “non-updated” software to provide healthcare. Then, RCT cannot investigate the multi-dimensional use of CDSS on co-morbid patients and/or those treated with polypharmacy. For those reason, an alternative strategy might be represented by quasi-experimental studies, such as the Interrupted Time Series (ITS) analyses, static group comparison, pre/post-test with non-equivalent control group^14,15. For example, the benefit of a predictive/prognostic score implemented in a CDSS, can be tested by evaluating proportions, and relative trends, of referrals to specialistic service, as well as the rate of patients prescribed with specific diagnostic procedures pre and post score implementation. Quasi-experimental studies are featured by pros and cons as well. The main advantage is certainly due to the presence of clinical data sources with which to perform retrospective and rapid studies with long follow-up. Moreover, they would allow quick reruns in agreement with the ongoing update of CDSS. To complement this analysis, GPs’ accesses to the system could be evaluated with this same design so monitoring the fatigue-related alerts⁶. For what concerns the ITS-related limitations, several time points (at least ten) are needed to avoid unpowered analyses; a comparison control group observed in the same time points is uneasily available; the unmeasured confounders might be carefully considered given the non-experimental nature of this design^16,17.

In conclusion, CDSS is a growing mainstay for primary care and other clinical settings. Currently, clinical evidence of CDSS effectiveness is an urgent matter, and further requirements will arrive soon from healthcare authorities. The clinical research on CDSS is therefore questioned on the best study type given the specific aim of the reminders/alerts intended to GPs along with the several ethical concerns. Although the scientific and regulatory contexts might appear similar to those applied for medications approval, some different approaches to providing clinical evidence on MDSW should be addressed soon.

Transparency

Declaration of funding

The study was funded by Genomedics Srl which has the ownership and copyright of MilleDSS technology.

Declaration of financial/other relationships

Cricelli I., Marconi E. and Lapi F. received a personal fee by Genomedics Srl for data acquisition and informatics consultation. Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Author contributions

Cricelli I. and Lapi F. conceptualized the study. Cricelli I., Marconi E. and Lapi F. wrote the manuscript. Cricelli I. is responsible for the integrity of the work; he is the guarantor. All aspects of the study were led by the authors.

Acknowledgements

None.