Adaptive Default Hierarchy Formation

Abstract

Autonomous systems are likely to be required to face situations that cannot be foreseen by their designers. The potential for perpetually novel situations places a premium on mechanisms that allow for automatic adaptation in a general setting. The term reinforcement learning problems (Mendel and McLaren, 1970) generally describes problems where a control system must adapt based on performance-only feedback. This paper considers the learning classifier system (LCS) as an approach to reinforcement learning problems. An LCS is a type of adaptive expert system that uses a knowledge base of production rules in a low-level syntax that can be manipulated by a genetic algorithm (GA) (Holland. 1975; Goldberg, 1989) Genetic algorithms comprise a class of computerized search procedures that are based on the mechanics of natural genetics (Goldberg, 1989; Holland. 1975). An important feature of the LCS paradigm is the possible adaptive formation of default hierarchies (layered sets of default and exception rules) )Holland et al., 1986). This paper examines the problem of default hierarchy formation under the conventional bid-competition method of LCS conflict resolution, and suggests the necessity auction and a separate priority factor as modifications to this method. Simulations show the utility of this method. Final discussion presents conclusions and suggests avenues for further research