Herbert A. Simon

Born 1916, Milwaukee, Wis.; Nobel Prize winner in economics and leader in AI and cognitive psychology who in 1956, with Allen Newell and J. C (Cliff) Shaw, first articulated a model of human and computer problem-solving based on heuristic search, and invented list processing languages to implement it.

Education: PhD, political science, University of Chicago, 1943.

Professional Experience: Int'l City Managers' Assoc., Chicago, 1936-1939; University of California, Berkeley, 1939-1942; Illinois Institute of Technology, 1942-1949; Carnegie Institute of Technology (later renamed Carnegie Mellon University): Graduate School of Industrial Administration, 1949-1965, Richard King Mellon University Professor of Computer Science and Psychology, 1965-present.

Honors and Awards: Award for Distinguished Scientific Contributions, American Psychological Association, 1969; ACM A.M. Turing Award (with Allen Newell), 1975; Nobel Prize in economics, 1978; ORSA/TIMS John von Neumann Award, 1984; National Medal of Science, 1986; member, National Academy of Sciences; fellow, ACM, 1994.

Simon's research career has been focused on the nature of intelligence, especially as exhibited in problem-solving and decision-making. As this topic cuts across many fields, it is difficult to identify his contributions with particular disciplines. His major awards in four areas psychology, economics, management science, and computer science attest to his breadth of interest and expertise. Prior to his early contacts with computers, he studied the informational and computational limits on the rationality of human decision makers, especially in organizations. His initial contributions to computer science were made in collaboration with Allen Newell and Cliff Shaw in the development of the first heuristic programs, the first list-processing languages (a series of IPLs [Information Processing Languages]), and the establishment of the field of artificial intelligence, which they preferred to call "complex information processing." Simon and Newell continued their collaboration through most of the 1960s, and remained in close association at Carnegie Mellon (although not engaged in joint research projects) until Newell's death in 1992. Their joint work on problem-solving was reported in Human Problem Solving (1972), which contains some of the first examples of the use of production systems to model human thought.

In the 1960s and subsequently, Simon's main research effort was aimed at extending the boundaries of artificial intelligence, with particular concern for the simulation of human thought processes, moving from the well-structured tasks addressed in early AI programs to wider ranges of tasks that call on substantial bodies of knowledge and that are relatively loosely structured. Among the important early programs were the General Problem Solver (GPS), jointly with Newell, and the Elementary Perceiver and Memorizer (EPAM), with Edward A. Feigenbaum, Lee W. Gregg, and Howard B. Richman.

The work on GPS led to a sustained program of research, with graduate students and colleagues, on human problem-solving, involving laboratory experimentation paired with computer modeling of human performance in various problem environments. One line of work, with William G. Chase, explored the role of knowledge-based recognition processes based in expert performance (using chess as the principal domain of study). Recognition processes, modeled by production systems, have provided one important basis for the expert systems now common in AI applications.

The work on EPAM has also continued to the present time, producing a system that simulates human behavior over a wide range of the perceptual, learning, and concept induction tasks that have been studied in the psychological laboratory, thereby constituting a theory of unmatched generality in this domain. Among the important phenomena implemented by EPAM's discrimination (sorting) mechanisms is learning by chunking--the assembly, through learning, of small units into larger entities, performed recursively to produce structures of arbitrary size and complexity. The discrimination net of EPAM was a progenitor of the Rete nets used to access productions in modern production system languages. Many of the papers reporting the research on problem-solving and on EPAM have been collected in the two volumes of Models of Thought.

A third line of research, with Kenneth Kotovsky and subsequently with Pat Langley, Gary Bradshaw, and Jan Zytkow, has built systems for discovering lawful patterns in empirical data, initially in sequences of letters or numbers (the Thurstone letter series completion tasks). In the past decade and a half, this research has expanded into the computational study of the processes of scientific discovery, producing such well-known discovery programs as BACON, DALTON, KEKADA, and LIVE. This research, too, has moved from highly structured but simple "toy" tasks to important discovery tasks taken from the history of science.

Simon has been deeply concerned with problems of knowledge representation and their relation to language understanding processes. The dissertation of his student Laurent Siklóssy demonstrated a model that learned language by matching linguistic sentences with their semantic interpretations. This and related work was collected in Simon and Siklóssy (eds.), 1972. With John R. Hayes, Simon built the UNDERSTAND program, which reads task descriptions in natural language and constructs representations of the task suitable as inputs to GPS. The research on representation has led to inquiries into computational procedures for reasoning from diagrams, and for automatic generation of representations.

In this research, the construction of intelligent systems has been closely linked with the development of a computational theory of human intelligence. The two chess programs Simon has constructed with colleagues (the NSS program with Newell and Shaw [1958], and the MATER program with George W. Baylor) contrast with more powerful programs like DeepThought and HighTech, the former modeling highly selective humanoid search rather than relying on massive computing power to compensate for lack of chess knowledge.

Simon has also made a series of contributions to "pure" computer science, not directly related to psychology. His Heuristic Compiler (1961, in Simon and Siklóssy 1972) was apparently the first system with (simple) capabilities for automatic programming. He has applied theories and techniques that originated in other disciplines to research problems in computer science. Methods of decomposing large systems into hierarchies of subsystems in order to simplify their analysis were developed by Simon and Ando (1961) for application to problems in economics, then transported by Courtois into computer science for the analysis of operating systems and other applications. A formal approach to causal ordering developed by Simon in the 1950s has been shown by Iwasaki and Simon to have broad applicability to research in qualitative physics, and generally, to the automatic induction of causal structures within AI.

At a theoretical level, Simon (1983) has examined the computational equivalence and inequivalence of different knowledge representations. He has shown the disadvantages of modal logic, compared with ordinary predicate logic and standard mathematics, as representation languages in AI, and has examined the computational differences among natural language, logical, diagrammatic, and schema-like representations as tools of inference. With J. Kadane, he proved an important theorem on optimal all-or-none search. He has discussed problems of parallel architecture and programming in terms of our knowledge of hierarchical organization (1969), and has addressed other foundational issues in computational AI.

The rapid development of computer science has demanded of its early practitioners a considerable investment of effort in institution building. Alan Perlis, Allen Newell, and Herbert Simon played the central roles in bringing computers to the campus of what was then Carnegie Institute of Technology, and building there the strong programs and faculties in computation that now exist not only in the School of Computer Science, but also in the Psychology Department, the Engineering Research Design Center, the Philosophy Department, and elsewhere on the campus. Simon has also been active in science policy at the national level, primarily through the council and committees of the National Academy of Sciences and its associated National Research Council, and (1968-1972) the President's Science Advisory Committee.

BIBLIOGRAPHY

Biographical

Baars, Bernard J., The Cognitive Revolution in Psychology, Guilford Press, New York, 1986.

McCorduck, Pamela, Machines Who Think, W.H. Freeman & Co., San Francisco, 1979.

Newell, Allen, "Simon, Herbert A.," in Ralston, Anthony, and Edwin D. Reilly, Encyclopedia of Computer Science and Engineering, Van Nostrand Reinhold Co., New York, 1983, pp. 1324-1325.

Simon, Herbert A., Models of My Life, Basic Books, New York, 1991.

Simon, Herbert A., and Allen Newell, "Information Processing Language V on the IBM 650," Ann. Hist. Comp., Vol. 8, No. 1, Jan. 1986, pp. 47-49.

Significant Publications

Newell, Allen, J.C. Shaw, and H.A. Simon, "Empirical Explorations of the Logic Theory Machine," Proc. Western Joint Computer Conf., 1957, pp. 218-239.

Newell, Allen, J.C. Shaw, and H.A. Simon, "Chess-Playing Programs, and the Problem of Complexity," IBM J Research and Development, Vol. 2, No. 4, 1958, pp. 320-335.

Newell, Allen, and Herbert A. Simon, Human Problem Solving, Prentice-Hall, Englewood Cliffs, NJ., 1972.

Simon, Herbert A., The Sciences of the Artificial, MIT Press, Cambridge, Mass., 2nd ed., 1981.

Simon, Herbert A., Models of Thought, Yale Univ. Press, New Haven, Conn. Vol. 1, 1979; Vol. 2, 1989.

Simon, Herbert A., "Search and Reasoning in Problem Solving," Artificial Intelligence, Vol. 1, 1983, pp. 7-29.

Simon, Herbert A., and Laurent Siklóssy, eds., Representation and Meaning, Prentice-Hall, Englewood Cliffs, NJ., 1972.

UPDATES

Herb Simon died February 9, 2001 (MRW, 2012).

Portrait added (MRW, 2013).