A bottom-up complement to the logic-based top-down approach to the story arrangement test

Abstract

Psychometric AI is a type of AI distinguished by the pursuit of intelligent systems able to excel on psychometrically validated human-level tests of cognitive abilities. We seek to build a system that solves a specific sub-test within Psychometric AI: the story arrangment test. Items in this test confront the test-taker with a set of jumbed snapshots (whether diagrammatic or otherwise) which must be ordered to tell a coherent story. We propose a dual-process system that combines bottom-up non- or sub-symbolic processing (e.g. neural network-based modelling) with top-down symbolic processing (e.g. deductive reasoning over declarative information represented as formulae in a logical system) for solving these tests of cognitive ability. The top-down process provides the benefits of a traceable proof, but requires a large amount of pre-existing knowledge. The bottom-up technique sacrifices provability and certainty on some problems for speed, but always yields some level of an answer to a given problem. This demonstrates a natural marriage between the two: the bottom-up approach seems especially powerful when used as a form of pre-processing in conjunction with a logic-based approach, because the latter approach would only need to consider a small number of possible orderings of snapshots.

Keywords:

• artificial intelligence
• tests
• psychometrics
• stories
• narrative
• picture arrangement

Notes

1. The actual opening is as follows:

• As Gregor Samsa awoke one morning from uneasy dreams he found himself transformed in his bed into a gigantic insect. He was lying on his hard, as it were armor-plated, back and when he lifted his head a little he could see a dome-like brown belly divided into stiff arched segments on top of which the bed quilt could hardly keep in position and was about to slide off completely. His numerous legs, which were pitifully thin compared to the rest of his bulk, waved helplessly before his eyes. (Kafka 1948, p. 67)

2. Our sense of snapshot is analogous to the use of the term in recursion theory, where the computation of a Turing machine (and equivalents) is often said to be a sequence of ‘snapshots’. E.g. see (Davis, Sigal, and Weyuker 1994).

3. It should be noted that the WAIS requires that only one feasible solution for each PA problem be presented, though several different valid possibilities may exist.

4. Following an approach to AI described, e.g. in (Nilsson 1991; Bringsjord and Ferrucci 1998; Bringsjord 2008b).

5. Admittedly, these are also problems a human competitor would face as well.

6. We do not opine that these are insurmountable drawbacks, but at any rate they must be faced.

7. Personal communication with Chuck Stewart, a longtime computer vision expert in computer science at RPI. A similar verdict has been provided by other computer vision experts we have consulted. Furthermore, if this were not true, then Yahoo!'s Word Verification Puzzle, used to prevent creation of spam email accounts by programs, would not be as successful as it consistently is year after year.

8. An electronic version of this article, in which colours can be clearly seen, is available at http://kryten.mm.rpi.edu/story_arrangement_AAAIFS_07_051507.pdf.