From the general problem-solver to the society of mind: cognitivist insights from the early AI era

One of the first developed AI systems, at the end of the 1950s, is the pioneering work of Herbert Simon, John Clifford Shaw, and Allen Newell on the General Problem Solver (GPS). GPS was a system able to demonstrate simple logic theorems and its decision strategies were explicitly inspired by human verbal protocols[1] (Newell, Shaw & Simon, 1959). The underlying idea of this approach was that the computer system had to approximate the decision operations described by humans in their verbal descriptions as closely as possible. In this way, when the program ran on the computer, it would be possible to identify its problems, compare them with the description of the human verbalization, and modify them to improve its performance. In particular, the GPS system was able to implement a key mechanism in human problem solving: the well-known “means- ends analysis” (or M-E heuristics). The M-E heuristics implemented in GPS works as follows: the problem solver makes a comparison between the current situation and a goal situation; then, it computes and evaluate the “distance” between these two states and tries to find, in memory, suitable operators able to reduce such difference. Once a suitable operator is found, it is then applied to change the current situation. The process is repeated until the goal is gradually attained via a process of progressive distance reduction. There are, however, generally no guarantees that the process will succeed. This kind of heuristic was also used to solve, in the decades to come, problems in a number of domains. In order to be executed, in fact, it “only” required an explicit domain representation of the problem to solve (a problem space), operators to move through the space, and information about which operators were relevant for reducing which differences.[2] GPS can be arguably considered the first cognitively inspired AI system ever developed.

A decade after the development of GPS, a Ph.D. student of Herbert Simon[3] at Carnegie Mellon University (then still named Carnegie Institute of Technology) - Ross Quillian - developed another influential idea in the context of AI of cognitive inspiration; he invented the Semantic Networks: a psychologically plausible model of human semantic memory implemented in a computer system. The idea (Quillian, 1968) was that human memory is associative in nature and that concepts are represented as sort of nodes in graphs and are activated through a mechanism of “spreading activation”, implemented through a marker passing algorithm, allowing the propagation of information through the network to determine the strength of the relationships between concepts. In this setting, the higher the activation of a node in the network, the more contextually relevant that node/concept was assumed to be for the task in focus. Interestingly enough, the research on Semantic Networks paved the way for both the development of the first graph-like, knowledge-based systems and formalisms (which make use of so-called symbolic representations) as well as the improvement of the so-called connectionist or sub-symbolic systems, since the concept of “spreading activation” has been very influential in the context of the “connectionist” investigations (see Cordeschi, 2002: 235, on this point). Before proceeding further with our examples of early cognitively inspired AI systems, it is necessary to briefly introduce the above-mentioned basic notions of “symbolic representations” (and paradigm) and “connectionist or sub-symbolic representations” (and paradigm), since they have been, and still are, really crucial modelling methods in both the past and present AI and cognitive modelling communities. In particular, the notion of “symbolic representation” constitutes a core assumption of the so-called “symbolic paradigm” in AI and cognitive science (which will be better clarified in more detail later in the book). In short, according to this view, intelligence in natural and artificial systems is associated with the capability of storing and manipulating the information in terms of abstract “symbols” (representing, in many cases, some mental proxy associated with external physical objects) and on the capability of executing mental operations and calculations over such symbols. This view was (is) severely criticized by the so-called “connectionist or sub-symbolic paradigm”, according to which the organization of the “mental content” in natural and artificial systems is not based on any symbolic structure but is, on the other hand, (1) distributed in nature and (2) based on parallel models of computations (these are the two core assumptions of the “connectionist representations”), in a way that is more similar to the biological organization and processing mechanisms of neurons and synapses in our brain. From a modelling perspective, this approach has led to the development of the Artificial Neural Networks, or ANNs (partially inspired by the biological neural structure of our brain), and self-organizing systems. We will discuss later the impact of “neural” or brain-inspired methods in early (and modern) AI research.[4] For the moment it is probably worth mentioning that, from a historical point of view, the “symbolic paradigm” represented the mainstream assumption in the context ofboth early AI and cognitive modelling research.

A confirmation of what was just discussed is provided by the next example of a cognitively inspired AI framework, which we are going to investigate: the notion of Frames (still a symbolic representational framework) operated by Marvin Minsky almost a decade after Quillian’s proposal (Minsky, 1975). With this proposal, Minsky intended to attack another well-known “symbolic approach” developed back then: the “logicist”' position a la McCarthy for the representation of knowledge in artificial systems. In particular, Minsky argued that such a proposal was not able to deal with the flexibility of the commonsense reasoning that is so evident in human beings. Frames, on the other hand, were proposed for endowing AI systems with commonsense knowledge (including default knowledge) about the external world.[5] [6] The type of knowledge organization proposed in the Frames enabled the first AI systems to extend their automated reasoning abilities from classical deduction to more complicated forms of commonsense and defeasible reasoning (going from induction to abduction). In this case, the idea of the Frames was directly inspired by the work of the psychologist Eleanor Rosch (Rosch, 1975) about the organization of conceptual information in humans known as the “prototype theory”[7] as well as by the memory “schemas” proposed by the cognitive psychologist Bartlett (Bartlett, 1958). A simple example and use case, done by Minsky himself, of a frame data structure is the following: let us imagine opening a door inside a house we are not familiar with. In this case, we typically expect to find a room that more or less is characterized by features that we have already seen in other rooms we have been in. Such features are referred to as a body of knowledge organized in the form of prototypes (i.e., the typical room). The data structures that reflect this flexible way of using knowledge, which is typical of human beings, can be described as “frame systems”. Therefore, the “room frame” is a characterized by different types of information that includes - listed in appropriate “slots” - the typical features of a room, such as a certain number of doors, walls, windows, and so on. There could be various kinds of rooms - dining rooms, bedrooms, etc. — each constituting, in turn, a frame with more specific features, again listed in appropriate slots. This kind of representation also allows for individual differences in conceptualization; e.g., Francesca’s dining room might be quite different from Paola’s in various details, but it will always be part of one and the same kind of room frame. The proposal of the frames as data structures for commonsense reasoning was not completely successful from a computational point of view (since frame systems did not scale well) but was very influential for the development of research in the context of commonsense reasoning.

In those years, a proposal very much aligned with Minsky’s was put forth by Roger Schank and his “conceptual dependency” theory (Schank, 1972). Schank aimed at explaining natural-language understanding phenomena via psychologically plausible computational processes. He proposed identifying a small set of “semantic primitives”, the use of which would have made it possible to construct the representation of meaning for any English verb. In his original programs, a sentence was analyzed by making explicit its representation in terms of semantic primitives. Such primitives were considered common to all natural languages and constituted a sort of interlingua. This interlingua was then used to build the first machine translation systems (e.g., MARGIE, see Shank & Nash-Webber, 1975). When Schank passed from constructing programs translating single sentences to ones aimed at translating entire stories, he realized that it was necessary to take commonsense into account. In this respect, a relevant problem concerned the knowledge needed to derive meaningful inferences from the union of different sentences in a story, so as to make explicit the implicit beliefs and expectations assumed in the context of a story. To tackle this and other problems, Schank and Abelson (1977) endowed their program - SAM (Script Applier Mechanism) - with “scripts”. Scripts are a data structure for representing knowledge of common sequences of events (e.g., the sequence of events used to go out for dinner) and are used in natural-language processing systems as way to enable intelligent answers to questions about simple stories. A classic example used to explain the notion of a “script” (which is also tightly connected with the notion of a “Frame”) is the so called “restaurant situation”. Let us consider a situation of an agent going out to a restaurant for dinner. A script representing the restaurant situation is a data structure that would record the typical events associated with this scenario; e.g., entering the restaurant, asking for a table, sitting down, consulting a menu, eating the food, paying the check, etc. This kind of representational structure enabled early AI systems to answer questions about simple stories. For example, let us consider a story like this: “Mary went to a restaurant and ordered salmon. When she was paying, she noticed that she was late for her next appointment.” In this case, computerized systems were able to answer a question such as, “Did Mary eat dinner last night?” in a positive way (as we do). It is worth noticing that this information is not explicitly provided in the story. Answering these types of questions is possible through the use of a “script” of the restaurant situation.

The capability of understanding natural-language instructions was also a crucial feature of Terry Winograd’s famous robotic system known as SHRDLU (named for the alphabetic symbols composing a row of keyboards in that era). In SHRDLU (Winograd, 1972), interactions with humans focused on a simulated blocks world that humans could view on a graphics display and to which the system had direct access. Users drove the conversation via written text by typing sentences, including commands like, “Find a block that is taller than the one you are holding and put it into the box” and “Is there anything that is bigger than every pyramid but not as wide as the thing that supports it?”. As reported in Langley (2017),

These inputs required not only the ability to parse quite complex structures and extract their meanings but also to draw inferences about relationships and execute multistep activities. The innovative system handled simple anaphora, disambiguated word senses, and had basic memory for its previous interactions.

SHRDLU was, therefore, an important advancement because it integrated sentence level understanding, reasoning about domain content, execution of multistep activities, and natural interaction with human users. At that time, there was no other artificial system able to show the same range of capabilities, and it offered a proof of concept that such an integrated intelligent system was possible. This accomplishment, of course, relied on some important simplifications: SHRDLU operated in a narrow and well-defined domain and had complete access to the entire state of the simulated environment. Nevertheless, it was an impressive achievement, which fostered further work on intelligent agents. To a certain extent, the integrated abilities exhibited by SHRDLU were the inspiration also for the subsequent work of Allen Newell and his colleagues at Carnegie Mellon University, concerning the development of the first integrated cognitive architecture for general intelligence: SOAR (Newell, Laird, & Rosenbloom, 1982).[8] [9]

At the very time that SOAR was first being developed (by now we were already in the mid-1980s), another relevant proposal in the context of cognitively inspired AI was made, once again, by Marvin Minsky, who introduced the evocative idea of the “Society of Mind” (Minsky, 1986, 2007) as a way to conceptualize, analyze, and design intelligent behaviour. This idea relies on the importance of considering, in natural and artificial agents, problem-solving activities “in layers” of interconnected micro-faculties (i.e., as a “society” of processes). In particular, Minsky suggested that the capability of dealing with commonsense knowledge11 is the grounding element of these layers of growing thinking capabilities. Such an approach has been historically impactful — not from an engineering perspective (since much more detail would have been needed in the Minsky proposal to specify how the processes can and should interact in an efficient computer implementation) - but mainly for the idea of considering, from a methodological and modelling perspective, the classical problem-solving activity (which was already modelled in systems like GPS or SOAR) through this sort of layered conceptual view involving a multistep reasoning process. As we will see in the following sections, this layered approach influenced, under completely different assumptions, another protagonist of the AI story from the previous century: Rodney Brooks.[10]

This list of examples of early cognitively inspired AI systems reviewed so far is, of course, not exhaustive. However, all these early systems shared a common “view” about the study of intelligence in artificial systems. More precisely, all these systems adhere - at different levels - to the so-called “cognitivist tradition”[11] of AI, also known as GOFAI (Good Old Fashioned AI).

Such early view is successfully synthesized by Pat Langley (Langley, 2012), who said, “(Early) AI aimed at understanding and reproducing in computational systems the full range of intelligent behaviour observed by humans” (Langley, 2012).

Langley identifies the following set of features that characterize the early AI period and the main cognitivist modelling assumptions:

the role of symbolic representations as a building block upon which operate a set of manipulation operations to let intelligent behaviour emerge; the importance of a general cognitively inspired approach to the study of the mind and intelligence (what Pat Langley calls a “system view”); the main focus on the so-called “high level cognition” (the systems for natural language processing,[12] for example, underwent a big development in this early period);

the adoption of heuristics (we will return on this concept later) as a method for problem solving;

the intrinsic interdisciplinary and exploratory nature of the research.

We will analyze in more details these aspects of the cognitivist tradition (and its differences from emergentist perspectives) in the next few sections of the chapter.

However, from a historical perspective, it is worth mentioning that this approach to the study of the artificial did not come out ex-abrupto. It borrowed its original inspiration, even if grounded on different assumptions, from the methodological apparatus developed by scholars in cybernetics (Cordeschi, 1991). The origins of cybernetics, in fact, are usually traced back to the middle of the 1940s, with the release of the 1948 book by Norbert Wiener entitled Cybernetics: Or Control and Communication in the Animal and the Machine. An underlying idea of cybernetics was one about building mechanical models to simulate the adaptive behaviour of natural systems. As indicated in Cordeschi (Cordeschi, 2002): “The fundamental insight of cybernetics was in the proposal of a unified study of organisms and machines”. In this perspective, the computational simulation of biological processes was assumed to play a central epistemological role in the development and refinement of theories about the elements characterizing the nature of intelligent behaviour in natural and artificial systems. Such kind of simulative approach, as mentioned, was inherited by the early AI research that used computer programs to reproduce performances, which, if observed in human beings, would be regarded as “intelligent”. The adoption of such a perspective was crucial in AI, for the development of both intelligent solutions inspired by human processes and heuristics (Newell & Simon, 1976; Gigerenzer & Todd, 1999) and for the realization of computational models of cognition built with the aim of providing a deeper understanding of human thinking, as originally suggested in the manifesto of Information Processing Psychology (IPP) (Newell & Simon, 1972). These two sides of the cognitivist tradition are nowadays still alive. They correspond, roughly, to the research areas known as “cognitively inspired AI” (or “cognitive systems”) and “cognitive modelling” (or “computational cognitive science”), respectively.

  • [1] This technique is also known as the “thinking aloud protocol” in the psychological literature(Ericsson & Simon, 1980) and consists of recording the verbal explanations provided by peoplewhile executing a given laboratory task.
  • [2] As we will see in more detail in the following sections, the ingredients required for the execution of this kind of heuristic strategy - essentially based on a “search space” approach to problemsolving - explicitly supported the so-called “symbolic approach” for the study, analysis, execution, and replication of intelligent behaviour in artificial systems.
  • [3] Herbert Simon is arguably one of the most important scientists of the last century. His influence,indeed, went well beyond his original training in cognitive psychology. Simon was awarded aNobel Prize in Economics for his studies on “bounded rationality”, which showed - differingfrom the classical decision models of the time - how humans are not optimal decision makers.This field of study has led to the development of an entirely new discipline that is nowadaysknown as “behavioural economics”. In addition, he was one of the founding fathers and mainprotagonist of the field of AI; along with people like Marvin Minsky, John McCarthy, AllenNewell, Nathaniel Rochester, and many others, he was an active participant in the DartmouthWorkshop. As a result of his “bounded rationality” theory in decision making, he was, one ofthe first scholars to point out, in both cognitive psychology and AI, the role played by heuristicsas decisional shortcuts to solve complex problems. The application of the heuristic approachin the context of AI was one of the reasons behind him winning, in 1975, the Turing Award,together with Allen Newell. The particular meanings attributed to the term “heuristics” in theAI research, will be explained later in this chapter.
  • [4] For the sake of completeness, it is also worth mentioning that within the cognitive modellingand AI communities another paradigm has been historically proposed relying on so-called“analog” or “diagrammatic” representations. In particular, according to the supporters of thisschool of thought, mental representations take the form of “pictures” in the mind. There aremany different examples of analog representations proposed, one of the most famous corresponding to the “mental models” by Johnson-Laird (1983, 2006). A general underlying assumption of this class of representation is that “spatial cognition” abilities (represented via these“picture-like” schemas) are a core aspect of natural cognitive systems from which other intelligent mechanisms emerge (e.g., the mental models by Johnson Laird have been notoriouslyproposed to model different types of inferences).
  • [5] A brief overview of the logical approaches proposed in the 1970s to deal with commonsensereasoning (e.g., circumscription, fuzzy logic, etc.) is sketched out in the next chapter of thebook. At this point, it is important to point out that the logicist tradition was (is) deeply rootedin the symbolic representation assumption, briefly elaborated on above and further detailed inthe next section of this chapter.
  • [6] As indicated elsewhere, “all the forms of commonsense reasoning can be seen as a boundedrationality phenomenon since they represent a plethora of shortcuts allowing us (i.e.,“bounded-rational” agents) to make decisions in an environment with incomplete and uncertain information” (Lieto, 2020: 56).
  • [7] According to the prototype theory posited by Rosch, concepts are organised in our mindas “prototypes” (i.e., in terms of typical representative elements of that category) and suchan organization explains many types of so-called “typicality effects” (i.e., of commonsenseinferences) that we naturally perform in our everyday reasoning. We will return on this specificaspect later and more extensively in the book (particularly in Chapter 4), since commonsensereasoning represents one of the main areas of possible convergence between Cognitive Scienceand AI.
  • [8] On the role of cognitive architectures for general intelligent systems we remind to (Lieto et al.,2018). We will return to SOAR and to cognitive architectures over the course of the book. Inaddition to the SHRDLU influence, SOAR was heavily inspired by the heuristic search mechanisms already developed in the GPS system.
  • [9] Commonsense knowledge is acquired, according to the Minsky proposal, via “instinctive”or “learned” reactions, and is then processed towards the higher hierarchies of “deliberative”,“reflective”, “self-reflective”, and “self-conscious” thinking at the level of both individual andsocial context.
  • [10] Rodney Brooks is a roboticist and was previously an MIT Professor. He is the creator of“Herbert” the robot, the first mobile robot able to exhibit interesting reactive behaviours without any central controlled activity. For more details about the particular layered architectureproposed by Brooks, known as “Subsumption Architecture”, we refer the reader to the nextsection.
  • [11] As will be clarified in the following pages, the “cognitivist” tradition is deeply rooted in theso-called “symbolic paradigm” and was the dominant perspective during the early days of AIresearch. Cognitivist assumptions differ from those of the “emergentist” approaches, which are,on the other hand, rooted in the notions ofbottom-up self-organisation (see Vernon, 2014).
  • [12] A typical example of the systems developed in this period is Eliza (Weizenbaum, 1966), one ofthe first conversational agents (nowadays called “chatbots”), created to converse with a humanbeing, simulating, at least up to a certain extent, the behaviour of a psychotherapist.
< Prev   CONTENTS   Source   Next >