Evaluation of Manufacturing Expert Systems: Framework and Model

ABSTRACT

A great deal of attention has been given to the use of expert systems in solving manufacturing problems by software development experts as well as application experts. However, the evaluation of expert systems in manufacturing applications has received very little attention in the literature. Whether or not an expert system is justified for a particular manufacturing environment depends on the expert system's effectiveness on both the manufacturing problem and the problem environment.

An expert system may be evaluated according to the effectiveness of its ability to solve problems (interpret, predict, diagnose, design, plan, monitor, debug, repair, instruct, and control), interface with users, and handle knowledge (learn, represent, induce, and retrieve). It may also be evaluated on other aspects such as expandability, responsiveness, or simply its costs. However, it is difficult to measure all the costs and benefits of an manufacturing expert system and to perform a traditional economic analysis. Hence, we propose the use of a decision theoretic technique.

This paper provides a model to evaluate expert systems for manufacturing problems. This model can be implemented within the framework of an Analytic Hierarchy Process developed by Saaty [23]. It is applicable for evaluating expert systems developed for manufacturing problem areas such as process planning, assembly, quality control, scheduling, production automation, maintenance, machine tools selection, equipment layout, and material handling.