| Next: | 3: Cognition | Bookmark | |
| Up: | 2: Mind | Monolithic | |
| Prev: | Interlude: Represent, Notice, Understand, Invent |
| NOTE: | This section is about what thoughts do. For an explanation of what thoughts are - how they work, where they come from, and so on - see the previous sections. |
Before the AI can act, it needs to learn. "Learning" can be divided into knowledge-formation and skill-formation. Skill formation happens when mindstuff, reflexes, or other unconscious processes are modified. In humans, the modification is autonomic; in seed AIs, it can be either autonomic or deliberate; but skills are always executed autonomically. (Note that "skill", as used here, includes not only motor reflexes but cognitive reflexes, and that "skill" does not include conscious skills like knowing (in theory!) how to disassemble a motorcycle.) The usual term for the dichotomy between skill and knowledge is "procedural vs. declarative", although this involves an assumption about the underlying representation that isn't necessarily true. In general, "knowledge" is the world-model, the contents of the mind, and "skill" is the stuff the mind is made of. Because skills tend to be located at the concept-level or modality-level, this section focuses on knowledge.
The world-model is holistic or reductionist, depending on whether you're looking up or looking down. We live in a Universe where complex objects are built from simpler structures, and stochastic regularities in the interactions between simple elements become complex elements that can develop their own interactions.
Thus, broadly speaking, there are at least three kinds of knowledge problems. You can look for a regularity in the way an object interacts with another object. You can take an object, an event, or an interaction, and try to analyze it; explain how the visible complexity is embodied in the constituent elements and their interactions. Or you can take elements and interactions that you already know something about, and try to understand the high-level behavior of the system. Starting from what you know, you can look sideways, down, or up.
Actually, this is speaking too broadly. Where, for example, do you fit "taking an object that you know something about, and suddenly understanding its purpose within a higher system"? I suppose you could explain this as a variant of analysis - when the "Aha!" is done, the result is a better understanding of a system in terms of its constituents. But then there are other knowledge problems, like guessing the properties of an element by taking the intentional stance towards the system and assuming the object is well-designed for its purpose. Where does that fit in? The moral, I suppose, is that "reductholism" has its uses as a paradigm, but there are limits.
Maybe we should generalize to generic causal models, regardless of level? Then you could divide activities into noticing a property or interaction, deducing the cause of a property or interaction, or projecting from known causes to the expected results. This model is a little more useful, since it sounds like the three problem types may correspond to three problem-solving methods: (A) Examine the model for unexpected regularities, correspondences, covariances, and so on. (B) Generate and test possible models to explain an effect. (C) Use existing knowledge to fill in the blanks (and, if you're a scientific mind, test the predictions thus created).
Still, even that view has its limitations. For example, asking Why? or looking for an explanation isn't strictly a matter of generate-and-test. In fact, generate-and-test is simply a genteel, thought-level version of that old bugaboo of AI, the search algorithm. It seems likely that some type of "genteel search algorithm" - not "blind", but not really deliberate either, and with a definite random component - is responsible for sudden insights and intuitive leaps and a lot of the go-juice of intelligence on the concept level. On the thought level, however, it's often more efficient to take a step back and think about the problem. One implementation for thinking about the problem is "abstraction is information-loss" classical-AI-type "abstract thought", running the problem through with Unknown Variables substituted in for everything you don't know, to see if there are places where the Unknowns cancel out to yield partial results that would hold true of every possible solution, thus constraining the search space. A more accurate implementation would be "applying heuristics that operate on the general information you have, to build up general information about the answer".
The thought-level is a genuine layer of the mind. There isn't any simple way to characterize it. There's a complex way to characterize it, which would consist of watching people solve problems while thinking out loud ("protocol analysis"), then figuring out a set of generalizations that corresponded to underlying neurology or underlying functional modules of the problem-solving method, and which categorized all the individual thoughts in the experimental observations. This problem is large, but finite; the set of underlying abilities and mental actions is limited. Still, such a project is beyond the scope of this particular section. (What I will attempt to do, in later topics, is describe enough of the underlying abilities - enough that implementing them would give rise to sustainable thought. Remember, seed AI isn't about perfectly describing the complete functionality of humans, it's about building minds with sufficient functionality to work.)
The thought-level is a genuine layer of the mind, and has around the same amount of internal complexity as might be associated with the modality-level or the concept-level. The difference is that thoughts are open to introspection, and thus, when I make sweeping generalizations, my readers can catch me at it. Nonetheless, I hope that the generalizations that have been offered here are sufficient to convey a vague general image of what goes on in a mind searching for knowledge. Noticing interesting coincidences and covariances and similarities (looking sideways), building and testing and thinking about the reason why something happens (analysis, looking down in the holistic model, looking backwards in the causal model), trying to fill in the blanks from the knowledge you already have (prediction, looking up in the holistic model, looking forwards in the causal model). The goal is a holistic model with good high-level/low-level bindings, or a causal model where the consequences and preconditions of a perturbation are well-understood, or a goal-and-subgoal model with plans and convergences and intentionality. The goal is a model that holds together, on all levels, when you think about changing it; a model rich enough to support what we think of as intelligent thought.
It is literally impossible to draw a sharp line between understanding and creativity. Sometimes the solution to a difficult knowledge question must be invented, almost ab initio. Sometimes the creation of a new entity is not a matter of searching through possibilities but of seeing the one possibility by looking deeper into the information that you already have. But, usually, when building the world-model, you're trying to find a single, unique solution; the answer to the question. When trying to design something new, you're looking for anyanswer to the question. Understanding is more strongly constrained, but this actually makes the problem easier, since a solution exists and the problem is finding it... the constraints might rather be called clues.
In invention, each constraint eliminates options and makes it less likely that a solution exists. The distinction between understanding and invention is something like the difference between P and NP, between verifying a solution and finding it. Returning to the quadrivium of Sensory, Predictive, Decisive, and Manipulative binding, and to Manipulation's sub-trinity of qualitative, quantitative, and structural bindings, then invention, or high-level manipulation, adds a fourth binding, the holic binding. It's the ability to take a desired high-level characteristic and specify the low-level structure that creates it. It's the ability to engage in hierarchical design, to start from the goal of rapid travel and move to a complete physical design for a bicycle.
The methods of invention are even less clear-cut than the methods of understanding. Unless the problem is one of qualitative manipulation (choice from among a limited number of alternatives), the design space is essentially infinite. An intelligent mind reduces the effective search space through possession of a holistic model that ultimately grounds in heuristics capable of direct backwards manipulation. In other words, if you can choose any real number to specify the width of the wheel, what's needed is a heuristic that binds it - reversibly - to a higher-level design feature, such as desired stability on turns. If desired stability on turns is itself a design variable, a heuristic is needed that binds it to a known quantity, such as the weight range of the rider. And so on.
Such reasoning acts to reduce the search space from the space of all possible low-level specifications of a design, to the space of cognitive objects constituting reasonable high-level designs. If there are enough heuristics left to constrain the design further, or to specify design features from high-level goals, then the task can be completed without special inspiration. If there's a gap, a high-level feature with no heuristics that directly determine how it might be implemented, then there sometimes comes that special event known as an "insight", an intuitive leap.
Sometimes you try to invent the bicycle without knowing about the wheel. The crucial insight may consist of remembering logs rolling down a hill. It may consist of just suddenly seeing the answer. Or it may lie in finding the right heuristic to attack the problem. The key point is that a wide search space is crossed to find the single right answer, apparently without any guide or heuristic that simplifies the problem. (If the aha! is finding the right heuristic, then the act of creativity lies in crossing the search space of possible heuristics.)
What is creativity? Creativity is the name we assign to the mental shock that occurs when a large and novel load of high-quality mental material is delivered to our perceptions. I would say that it's the perception of "unexpected" material, meaning "unexpected" not in the sense that the delivery comes as a surprise, but in the sense that our mental model can't predict the specific content of the material being delivered. We perceive a thought as "creative", in ourselves or others, on one of two occasions: First, seeing someone thinking outside the box; second, on perceiving a single good solution selected from a nearly infinite search space. In the first case, a concept is redefined, or what was thought to be a constraint is broken; the answer is unexpected, which creates - to the viewer - the mental shock that we name "creativity". The second case consists of seeing the very large gap between "high-speed travel" and "bicycle" crossed; the viewer - unless ve verself has designed a bicycle - has no single heuristic that can cross a gap of that size, that can anticipate the content of the material presented. There's a nearly infinite space of possible paintings, so when we see any single painting of reasonable quality, a large quantity of unexpected cognitive material is delivered to our eyes and we call it "creativity".
It seems likely to me that the experience of creative insight happens when the mind decides to brute-force, or rather intelligent-force, the search problem. The aha! of wheels comes because, somewhere in the back of your mind, possible memories were tested at random for applicability to the problem until the memory of logs rolling down a hill resonated with the problem and rose to conscious attention. This unconscious "blind" search may employ some of the tricks of deliberation, such as searching through memories of objects that were seen traveling very fast. (Or not. It seems likely to me that only deliberate thought produces that kind of constraint.) Even so, it remains in essence a try-at-random algorithm. If there's anything more to subconscious creative insights than that, I don't know what it is.
Since thoughts are reasonably accessible to the human mind, there's a good deal of existing research on how they work. The specific methods are important, but what's more important is getting a working system of thoughts, enough methods that work well enough that the AI can continue further.
Most important to the system of thoughts is introspection. Introspection is the glue that holds the thought-level together. Coherent thoughts don't happen at random. They happen because we know how to think, and because we have the right reflexes for thinking. The problem of what to think next is itself a problem domain. To prevent an infinite-recursion error, our solution to this problem on the moment-to-moment level is dictated entirely by reflex, the channels worn into our neural minds. Even when we deliberately stop and say to ourselves, "Now, what topic should I think about next?", the thinking about thinking proceeds by reflex. These reflexes are formed during infancy, and before they exist, coherent thought doesn't happen. To get past that barrier you'd have to be a seed AI, capable of watching a replay of your own source code in action, or halting and storing the current state of high-level thought to recurse on examining the stuff the thought is made of.
The self is a domain fully as complex as any in external reality. It consists not just of perceiving the self but of manipulating the self. The experience you remember of introspection consists of the occasions when the problems became large enough to require conscious thought. Beneath that remembered, introspection-accessible experience lies perceptions and reflexes that have become so invisible we don't even notice them. The intuitions of introspection are far more basic to thought than Hamlet's soliloquy. The problem of introspection should be approached with the same respect, and the same attention to the RNUI method, that would be given to the problem of designing a bicycle.
Introspection requires introspective senses, perhaps even an introspective modality. But the idea of an introspective modality is a subtle and perhaps useless one. The obvious implementation is to have an introspective modality that reports on all the cognitive elements inside the AI, but what does this add? The AI has already noticed that the cognitive elements are there. How does "the introspective modality" differ from "a useless and static additional copy of all the information inside the AI"? What can you do with the detected feature of "the feature of redness" that you can't do with the feature of redness itself?
To answer this question, it is necessary to step back and consider the problem in context. Sensory modalities don't exist in a vacuum. They are useful because concepts lie on top. The question, then, is not how to build an introspective sensory modality, but how to insure that concepts about introspection can form. This may involve creating a new introspective modality, or it may involve attaching a new dimension to the old modalities and to the other modules of cognition.
Concepts manipulate their referents, as well as extracting information from them. How would you go about tweaking the visual modality so that you could imagine "thinking about redness"? How do you get the AI to notice, declaratively, that a concept has been activated, and how is this perception reversed to give rise to visualizing the consequences of activating a concept?
This design problem may go a bit towards explaining that peculiar phenomenon called "stream of consciousness". You notice a fact, the fact gets turned into a conceptual structure, the conceptual structure gets turned into a sentence by your language centers, and then you speak the sentence "out loud" within your mind. The fascinating thing is this: If you try to skip the step of "speaking the sentence out loud" within your mind, even after you know exactly what the words will be, you can't go on thinking. Why? What new information is added by this act?
One possible explanation is that the human mind notices concepts by noticing the auditory cortex. Humans have no built-in introspective modality, so concepts become "visible" to our mental reflexes when they add recognizable content - words - to the auditory cortex. This closes the loop. Concept activation becomes detectable, and we can form concepts about concepts. I don't think this is the entire explanation, but it's a good start.
What about thoughts? On the thought-level, human introspection is fairly primitive. There's this tendency to lump everything together under the term "I". When we attribute causality, we say "I remembered" instead of "the long-term memory-retrieval subsystem reports..." Perhaps this is because, historically speaking, we didn't know anything about what was inside the mind until yesterday afternoon. Perhaps it's because fine-grained introspection doesn't contribute useful complexity to self-modeling unless you're, oh, writing a paper on AI or something. There's plenty of useful heuristics about the self that can be learned by looking at cause and effect, even when all the causal chains start at a monolithic self-object. A seed AI may have uses for more fine-grained self-models, but with both design and source code freely accessible, it shouldn't be too hard for such a self-model to develop.
When can an AI legitimately use the word "I"?
Understand that we are asking about a very limited and purely technical aspect of self-awareness. We are not talking about the kind of self-awareness that will cause an ethical system to treat you as a person. We are not talking about "qualia", the hard problem of conscious experience, what it means to be a bat, or anything of that sort. These are different puzzles.
The question being asked is: When can an AI legitimately use the word "I" in a sentence, such as "I want ice cream", without Drew McDermott popping up and accusing us of using a word that might as well be translated as "shmeerp" or G0025?
Consider the SPDM distinction: Sensory, Predictive, Decisive, Manipulative. A binding between a model and reality starts when the model "maps" in some way to reality (although this is ultimately arbitrary), becomes testable when the model can predict experiences, and becomes useful when the model can be used to decide between alternatives, with the acid test being manipulation of reality in quantitative or structural ways. Consider also the distinction between modality-level, concept-level, and thought-level.
Self-modeling begins when the AI - let's call it Aisa, for "AI, self-aware" - starts to notice information about itself. Introspective sensations of sensations are hard to distinguish from the sensations themselves, so this ball doesn't really get rolling until Aisa forms introspective concepts. The self-model doesn't begin to generate novel information, information that can impose a coherent view of internal events, until it can make predictions - for example: "Skipping from topic to topic, instead of spending a lot of time on one topic, will result in conceptual structures that are connected primarily through association." Likewise, this information doesn't become useful until it plays a part in goal-oriented decisions - a decisive binding.
When Aisa can create introspective concepts and formulate thought-level heuristics about the self, it will be able to reason about itself in the same fashion that it reasons about anything else. Aisa will be able to manipulate internal reality in the same way that it manipulates external reality. If Aisa is impressively good at understanding and manipulating motorcycles, it might be equally impressive when it comes to understanding and manipulating Aisa.
But to say that "Aisa understands Aisa" is not the same as saying "Aisa understands itself". Douglas Lenat once said of Cyc that it knows that there is such a thing as Cyc, and it knows that Cyc is a computer, but it doesn't know that it is Cyc. That is the key distinction. A thought-level SPDM binding for the self-model is more than enough to let Aisa legitimately say "Aisa wants ice cream" - to make use of the term "Aisa" materially different from use of the term "shmeerp" or "G0025". There's still one more step required before Aisa can say: "I want ice cream." But what?
Interestingly, assuming the problem is real is enough to solve the problem. If another step is required before Aisa can say "I want ice cream", then there must be a material difference between saying "Aisa wants ice cream" and "I want ice cream". So that's the answer: You can say "I" when the behavior generated by modeling yourself is materially different - because of the self-reference - from the behavior that would be generated by modeling another AI that happened to look like yourself.
This will never happen with any individual thought - not in humans, not in AIs - but iterated versions of Aisa-referential thoughts may begin to exhibit materially different behavior. Any individual thought will always be a case of A modifying B, but if B then goes on to modify A, the system-as-a-whole may exhibit behavior that is fundamentally characteristic of self-awareness. And then Aisa can legitimately say of verself: "I want an ice-cream cone."
Humans also throw a few extras into the pot. We have observer-biased social beliefs, a whole view of the world that's skewed toward the mind at the center, which tends to anchor the perception of the self. We attribute internal causality to a monolithic object called the "self", which generates a lot of perceived self-reference because you don't notice the difference between the thought doing the modifying and the cognitive object being modified - the source of the thought is the "self", and the item being modified is part of the "self".
A seed AI will probably be better off without these features. I mention them because they constitute much of what a human means by "self".
| Next: | 3: Cognition |
| Up: | 2: Mind |
| Prev: | Interlude: Represent, Notice, Understand, Invent |