Evaluation of Sleep Expert–a computer-aided decision support system for sleep disorders

Abstract

Sleep Expert–a medical decision support system–is a prototype program, with knowledge based on the International Classification of Sleep Disorders (1990). The goal of this project was to evaluate Sleep Expert. In the evaluation project the knowledge of the program was first validated. Three physicians, experts in sleep disorders, were asked to choose 10 typical patient cases with sleep disorders, and to write a description. They also made a diagnosis for each case. Next, each expert made a diagnosis of the cases supplied by the other experts. They were not given the original diagnosis. The ‘right diagnosis’ (so-called majority agreement) was determined from the three diagnoses. Then the diagnosis of each expert was compared with the ‘right diagnosis’. Two physicians, not experts in sleep disorders, were asked to make a diagnosis by using Sleep Expert. Compared to the ‘right diagnosis’ the diagnoses of each user (non-expert physician) were correct to 63 and 70% of cases, which is quite a good result, although it does not reach the level of the expert physicians (≥ 87%). The functionality of Sleep Expert was studied by using a limited inquiry. On the basis of the user inquiry Sleep Expert provided a useful clinical tool for non-experts.

Key Words:

• Evaluation
• Validation
• Decision support system
• Sleep