Assessing the performance of a waste characterization expert system

Scientists at the Mississippi State University Diagnostic Instrumentation and Analysis Laboratory and the Idaho National Engineering and Environmental Laboratory (INEEL) have developed an expert system for a noninvasive characterization of containerized radiological waste. The characterization of the containers is necessary for determining their proper disposition. Three prototypes were developed, with each using a different method of handling uncertainty - a fuzzy system, a Bayesian network system, and a neural network system. The performance of each expert system was assessed to determine how well it modeled the decisions made by the INEEL domain expert. The prototype systems were also analyzed to measure the agreement in their decisions, the domain expert's decisions, and the decisions made by two additional experts. The neural network prototype was further analyzed to determine how consistent it was in its assessments. This paper describes the analysis of the performance of the three expert system prototypes.