GISAI is a work in progress.  As of version 2.3, the sections "Paradigms" and "Mind" are complete and self-contained.  The section "Cognition" is in progress and may contain references to unimplemented topics.  As additional topics are published, the minor version number (second digit) increases.

Words defined in the Glossary look like this:
    "A °seed AI is an AI capable of self-understanding, self-modification, and recursive self-enhancement."

The key words are "contribute materially".  An architecture can be necessary to thought without accounting for the substance of thought.  The Law of Pragmatism says that if a neural network's rules are simple enough to be formalized mathematically, than the substance of any intelligent answers produced by that network will be attributable to the specific pattern of weightings.  If the pattern of weightings is created by a mathematically formalizable learning method, then the substance of intelligence will lie, not in the learning method, but in the intricate pattern of regularities within the training instances.

The "world-model" for an AI living in that °microworld consists of everything the AI knows about that world - the positions, velocities, radii, and masses of the billiard balls.  More abstract perceptions, such as "a group of °three billiard balls", are also part of the world-model.  The prediction that "billiard ball A and billiard ball B will collide" is part of the world-model.  If the AI imagines a situation where four billiard balls are arranged in a square, then that imaginary world has its own, °subjunctive world-model.  If the AI believes "'imagining four billiard balls in a square' will prove useful in solving problem X", then that belief is part of the world-model.  In short, the world-model is not necessarily a programmatic concept - a unified set of data structures with a common format and °API.  (Although it would be wonderfully convenient, if we could pull it off.)  The "world-model" is a cognitive concept; it refers to the content of all beliefs, the substance of all mental imagery.

Returning to the billiard-ball world, what is necessary for an AI to have a "model" of this world?

Suppose that a cue ball travelling south at 4 meters/second, bumping into a billiard ball travelling south at 2 meters/second, results in the cue ball and the billiard ball travelling south at 3 meters/second.  Suppose, furthermore, that these rules are contained within the AI's internal model of the environment, so that if the AI visualizes a cue ball at {8.2, 6} of radius 1 travelling south at 4 m/s, and a ball at {8.2, 10} of radius 1 going south at 2 m/s, the AI will visualize the balls bumping one second later at {8.2, 11}, and the two balls then travelling south at 3 m/s.  It's a long way from there to knowing - consciously, °declaratively - that two balls in general bumping at 4 m/s and 2 m/s while going in the same direction will travel on together at 3 m/s.  It's an even longer way to knowing that "if billiard ball X bumps into billiard ball Y, then they will continue on together with the average of their velocities".  And it's a still longer way to reversing the rule and knowing that "to get a group of two balls travelling together with velocity X, given billiard ball A with velocity Y, bump it with billiard ball B having velocity (2X - Y)".  Finally, to close the loop, this last high-level rule must be applied to create a particular hypothesized action in the world-model, and the hypothesized action needs to be taken as a real action in external reality.

Without jumping too far ahead, there are a number of properties that a world-model needs to support high-level thought.  It needs to support °time - multiple frames or a temporal visualization - with accompanying extraction of temporal features.  It needs to support predictions and expectations (and an expectation isn't real unless the AI notices when the expectation is fulfilled, and especially when it is violated).  The world-model needs to support hypotheses, °subjunctive frames of visualization, which are distinct from "real reality" and can be manipulated freely by high-level thought.  (By "freely manipulated", I mean a direct manipulative binding; choosing to think about a billiard ball at position {2, 3} should cause a billiard ball to materialize directly within the representation at {2, 3}, with no careful sequence of actions required.)  And for the visualization to be useful once it exists, the high-level thought which created the billiard-ball image must refer to the particular image visualized... and the reference must run both ways, a two-way linkage.

°Time, expectation, comparision, °subjunctivity, visualization, introspection, and reference.  I haven't defined any of these terms yet.  (Most are discussed in 3: Cognition, although you can jump ahead to Appendix A: Glossary if you're impatient.)  Nonetheless, these are some of the basic attributes that are present in human world-models, and which are Necessary (But Not Sufficient) for the existence of high-level features such as causality, intentionality, °goals, memory, learning, association, focus, abstraction, categorization, and symbolization.

NOTE:

I mention that list of features to illustrate what will probably be one of the major headaches for AI designers:  If you design a system and forget to allow for the possibility of expectation, comparision, °subjunctivity, visualization, or whatever, then you'll either have to go back and redesign every single component to open up space for the new possibilities, or start all over from scratch. Actualities can always be written in later, but the potential has to be there from the beginning, and that means a designer who knows the requirements spec in advance.

Modalities in the human brain are mostly preprogrammed, as opposed to learned.  (Human modalities require external stimuli to grow into their preprogrammed organization, but this is not the same as learning.)  Individual neural signals can have meanings that are visible and understandable to an eavesdropper.  Programmers may legitimately take the risk of creating modalities through deliberate programming, with low-level elements that correspond to data structures, and human-written procedures for feature extraction.

Within °GISAI, the term concept is used to refer to the kind of mental stuff that exists as a pattern in the modality.  A learned sequence of instructions that reconstructs a generic, abstracted "light bulb" in the visual modality is a concept.  Symbols, categories, and some memories are concepts.  (Despite common usage, "concept" might technically refer to non-declarative mental stuff such as a human cognitive reflex or a human motor skill.  However, in a seed AI, where everything is open to introspection, it makes sense to call the equivalents of human reflexes or skills "concepts".)  Concepts are patterns, learned or preprogrammed, that exist in long-term storage and can be retrieved.

A structure of concepts creates a thought.  The archetypal example, in humans, is words coming together to form sentences.  Thoughts are visualized; they operate within the RAM of the mind, the "workspace" represented by available content capacity in the sensory modalities, commonly called "short-term memory" or "working memory".  (The capacity of working memory in AIs is not determined by available RAM, but by available CPU capacity to perform feature extraction on the contents of memory.  If you have the data structures without the feature extraction, the AI won't notice the information.)  Thoughts manipulate the world-model.

In chemistry, abstract means remove; to "abstract" an atom from a molecule means to take it away.  Use of the term "abstract" to describe the process of forming concepts implies two assumptions:  First, to create a concept is to generalize; second, to generalize is to lose information.  It implies that, to form the concept of "red", it is necessary to ignore other high-level features such as shape and size, and focus only on color.

This is the classical-AI view of abstraction, and we should therefore be suspicious of it.  On the other hand, our mechanisms for abstraction can learn the concept for "red".  In a being with a visual modality, this concept would consist of a piece of mindstuff that had learned to distinguish between red objects and non-red objects.  Since redness is detected directly as a low-level feature, it shouldn't be very hard to train a piece of mindstuff to thus distinguish - whether the mindstuff is made of trainable neurons, evolving code, or whatever.  A neural net needs to learn to fire when the "red" feature is present, and not otherwise; a piece of code only needs to evolve to test for the presence of the redness feature.  At most, "red" might also require testing for solid-color or same-hue groupings.  Given a visual modality, the concept of "red" lies very close to the surface.

To a twenty-first-century human, trained in arithmetic and mathematics, the concept of "three" has enormous richness.  It must therefore be emphasized that we are dealing solely with the concept of "three", and that a mind can understand "three" without understanding "two" or "four" or "number" or "addition" or "multiplication".  A mind may have the concept "three" and the concept "two" without noticing any similarity between them, much less having the aha! that these concepts should go together under the heading "number".  If a mind somehow manages to pick up the categories of groups-of-three-dogs and groups-of-three-cats, it doesn't follow that the mind will generalize to the category of "three".

To think about infant-level or child-level AIs, or for that matter to teach human children, it's necessary to slow down and forget about what seems "natural".  It's necessary to make a conscious separation between ideas - ideas that, to humans, seem so close together that it takes a deliberate effort to see the distance.

Just because the AI exists on a machine performing billions of arithmetical operations per second doesn't mean that the AI itself must understand arithmetic or "three".  (John Searle, take note!)  Even if the AI has a codic modality which grants it direct access to numerical operations, it doesn't necessarily understand "three".  If every modality were programmed with feature-extractors that counted up the number of objects in every grouping, and output the result as (say) the tag "number: three", the AI might still fail to really understand "three", since such an AI would be unable to count objects that weren't represented directly in some modality.  An AI that learns the concept of "three" is more likely to notice not just three apples but that °ve (the AI) is currently thinking three thoughts.  A preprogrammed concept only notices what the programmer was thinking about when he or she wrote the program.

What is "three", then?  How would the concept of "three" be learned by an AI whose modalities made no direct reference to numbers - whose modalities, in fact, were designed by a programmer who wasn't thinking about numbers at the time?  How can such a simple concept be decomposed into something even simpler?

"When you hear the phrase "triangular light bulb", you visualize a triangular light bulb...  How do these two symbols combine?  You know that light bulbs are fragile; you have a built-in comprehension of real-world physics - sometimes called "naive" physics - that enables you to understand fragility.  You understand that the bulb and the filament are made of different materials; you can somehow attribute non-visual properties to pieces of the three-dimensional shape hanging in your visual cortex.  If you try to design a triangular light bulb, you'll design a flourescent triangular loop, or a pyramid-shaped incandescent bulb; in either case, unlike the default visualization of "triangle", the result will not have sharp edges.  You know that sharp edges, on glass, will cut the hand that holds it."
        -- 1.2: Thinking About AI

Neural networks, when perturbed, are known to seek out what might be called "minimal-energy states".  A network-relaxation model of concept combination could be computationally realistic - an operation that neurons can accomplish in the 200 operations-per-second timescale.  My current hypothesis for the basic neural operation in concept-combination is the resonance.  A neural resonance circuit - perhaps not a physical, synaptic circuit, but a virtual message-passing circuit, established by one of the higher-level neural communication methods (binding by neural synchrony, maybe) - can either resonate positively, reinforcing that part of the concept-combination, or resonate negatively, generating a conflict.  My guess at the network-relaxation method resembles the "potential energy surface" of chemistry in that multiple, superposed alternatives are tried out simultaneously, so that the minima-seeking resembles a flowing liquid rather than a rolling ball.

The high-level, salient facets of the concepts being combined are combined first.  These high-level features then visualize the mid-level features; if no conflict is detected, the mid-level features visualize the low-level features.  If a conflict is detected at any level, the conflict propagates back up to the conflicting high-level or mid-level features causing the problem.  Who wins the conflict?  The more salient, more important, or more useful feature - remember, we're talking about combining two concepts, each with its own set of features along various dimensions - is selected as dominant, and the network relaxation algorithm proceeds.  When one concept modifies another, the "more salient" feature is the one specified by the concept doing the modifying.  (Note also that, in casual reading, not all the facets of a concept may be important, just as you don't fully visualize every word in a sentence.  Only the facets that resonate with the subject of discussion, with the paragraph, will be visualized.)

In the case of "triangular light bulbs", "triangular" is an adjective.  The concept for "triangle" or "triangular" is modifying the concept of "light bulb", rather than vice versa.  The default exemplar for "light bulb" - that is, an image of the generic light bulb - is loaded into the mental workspace, including the visual facet of the exemplar being loaded into the visual cortex.  Next, the concept for "triangular" is applied to this mental image.

The concept of "triangular", as it refers to physical objects, has a single facet:  It alters the physical shape of the target image.  Note that I say "physical shape", not "visual shape".  The default exemplar for "light bulb" is a mental image - not a mental picture, but a mental image; in GISAI, an "image" means a representation in any modality or modalities, not just the visual cortex.  The "light bulb" exemplar is an image of a three-dimensional bulb-shaped object, made of glass, having a metal plug at the bottom, whose purpose is to emit light.  It is this multimodal mental image that "triangular" modifies, not just the visual component of the image.  In particular, the "shape" facet of the light-bulb concept, the facet being modified, is a high-level feature describing the shape of the three-dimensional physical object, not the shape of the visual image.  Thus, modifying the light-bulb shape will modify the mental image of the physical shape, rather than manipulating the 2-D visual shape in the visual cortex.

The "triangular" concept, when applied along the dimension of "shape", manipulates the mental image of the light bulb, changing the 3D model to be triangle-shaped.  However, since the image of a flat light bulb fails to resonate, "triangle" automatically slips to "pyramid".

(I'm not sure whether this conflict is detected at the mid-level feature of "flat light bulb", or whether a flat light bulb actually begins to visualize before the conflict is detected.  The slippage happens too fast for me to be sure.  I suspect that "triangular" has slipped to "pyramidal" before, when applied to three-dimensional mental images; for neural entities, anything that happens once is likely to happen again.  Neurons learn, and neural thinking wears channels in the neurons.  It could be that the non-flatness of light bulbs is salient because of their bulbous shape, and that this resonance with non-flatness causes "triangular" to slip to "pyramidal" before the concept is even applied.)

Pyramids are sharp.  I know, from introspection, that the "sharp pyramidal light-bulb" got all the way down to the visual level before the conflict was noticed.  (The conflict rose to the level of conscious perception, but was resolved more or less intuitively; I didn't have to "stop and think".  So this is probably still a valid example of concept-level processes.)  The particular conflict:  Sharp glass cuts the person who holds it.  We've all had visual experience of sharp glass, and the associated need for visual recognition and avoidance; thus, the mental image of sharp glass would trigger this recognition and create a conflict.  This conflict, once detected, was also visualized all the way down to the visual cortex; I briefly saw the mental image of a thumb sliding along the edge of the pyramid.

Thoughts are created by structures of concept-level patterns.  The archetypal example is a grammatical sentence: a linear sequence of words parsed by the brain's linguistic centers into a more-or-less hierarchical structure, in which the referents of targetable words and phrases (an adjective needs a target image, for example) have been found, either inside the sentence or in the most salient part of the current mental image.  The inverse of this process is when a fact is noticed, turned into a concept structure, translated into a sentence, and articulated out loud within the mind.  (A possible reason for the stream-of-consciousness phenomenon is discussed in 2.4.3: Thoughts about thoughts.)

The current section has discussed concepts as mindstuff-based patterns in sensory modalities - that is, the mindstuff is assumed to pay attention to, or issue instructions to, the sensory modalities and the features therein.  That concepts interact with other concepts, and are influenced by the higher-level context in which they are invoked, has been largely ignored.  This was deliberate.  The farther you go from the mindstuff level, and the more "abstract" you get, the closer you are to the levels that are easily accessible to human introspection.  These are the introspective perceptions that come out in words; the qualities that modern culture associates with above-average intelligence; the levels enormously overemphasized by classical AI.

Still, there are some thoughts that are so abstract as to appear distant from any sensory grounding.  In that last sentence, for example, only the term "distant" has an obvious grounding, and since the sentence wasn't interpreted in a spatial context, it's unlikely that even that term had any direct visualizational effect.  Metaphors do show up more often than you might think, even in abstract thought (see °Lakoff and Johnson, Metaphors We Live By or Philosophy in the Flesh).  Still, there are concepts whose definition and grounding is primarily their effect on other concepts - "abstract concepts".  Why doesn't the classical-AI method work for abstract concepts?

Even abstract concepts, mental images composed entirely of concepts referring to other concepts, exist within a °reductholistic system.  Abstract concepts may not have reductionist definitions that ground directly in sensory experience, but they have reductionist definitions that ground in other concepts.  What are apparently high-level object-to-object interactions between two abstract concepts can, if conflicts appear, be modeled as mid-level structure-to-structure interactions between two definitions.  Abstract concepts still have lower-level structure, mid-level interactions, and higher-level context.

Still, defining concepts in terms of other concepts is what classical AIs do.  I can't actually recall, offhand, any (failed!) classical AIs with explicit holistic structure - I can't recall any classical AIs that constructed explicitly multilevel models to ground reasoning using semantic networks - but it seems likely that someone would have tried it at some point.  (Eurisko and Copycat don't count for reasons that will be discussed in future sections.  Besides, they didn't fail.)  So, why doesn't the classical method work for abstract concepts?

Many classical AIs lack even basic quantitative interactions (such as fuzzy logic), rendering them incapable of using methods such as holistic network relaxation, and lending all interactions an even more crystalline feeling.  Still, there are classical AIs that use fuzzy logic.

What's missing is flexibility, mutability, and above all richness; what's missing is the complexity that comes from learning a concept.  Perhaps it would be theoretically possible to select a piece of abstract reasoning in an adult AI in which the complexity of sensory modalities played no part at all.  Perhaps it would even be possible to remove all the grounding concepts below a certain level, and most of the modality-level complexity, without destroying the causal process of the reasoning.  Even so - even if the mind were deprived of its ultimate grounding and left floating - the result wouldn't be a classical AI.  Abstract concepts are learned, are grown in a world that's almost as rich as a sensory modality - because the grounding definitions are composed of slightly less abstract concepts with rich interactions, and those less-abstract concepts are rich because they grew up in a rich world composed of interactions between even-less-abstract concepts, and so on, until you reach the level of sensory modalities.  Richness isn't automatic.  Once a concept is created, you have to play around with it for a while before it's rich enough to support another layer.  You can't start from the top and build down.

Another factor that's missing from classical AIs is the ability to attach experience to concepts, to gain experience in thinking, to wear a channel in the mind.  Even a concept-combination like "triangular light bulb" has a dynamic pattern, a flow of cause and effect on the concept level, that relies on the thinker having done most of the thinking in advance.  That complexity is also absent from classical AIs.  (And of course, most classical AIs just don't support all the other dimensions of cognition - attention, focus, causality, goals, subjunctivity, et cetera.)

I think this provides an adequate explanation of why classical AI failed.  This is why classical AIs can't support thought-level reasoning or a stream of consciousness; why sensory modalities are necessary to learn abstract thought; and why concepts must be learned in order to be rich enough to support coherent thought.

Before the AI can act, it needs to learn.  "Learning" can be divided into knowledge-formation and skill-formation.  Skill formation happens when mindstuff, reflexes, or other unconscious processes are modified.  In humans, the modification is autonomic; in seed AIs, it can be either autonomic or deliberate; but skills are always executed autonomically.  (Note that "skill", as used here, includes not only motor reflexes but cognitive reflexes, and that "skill" does not include conscious skills like knowing (in theory!) how to disassemble a motorcycle.)  The usual term for the dichotomy between skill and knowledge is "procedural vs. declarative", although this involves an assumption about the underlying representation that isn't necessarily true.  In general, "knowledge" is the world-model, the contents of the mind, and "skill" is the stuff the mind is made of.  Because skills tend to be located at the concept-level or modality-level, this section focuses on knowledge.

The world-model is holistic or reductionist, depending on whether you're looking up or looking down.  We live in a Universe where complex objects are built from simpler structures, and stochastic regularities in the interactions between simple elements become complex elements that can develop their own interactions.

Thus, broadly speaking, there are at least three kinds of knowledge problems.  You can look for a regularity in the way an object interacts with another object.  You can take an object, an event, or an interaction, and try to analyze it; explain how the visible complexity is embodied in the constituent elements and their interactions.  Or you can take elements and interactions that you already know something about, and try to understand the high-level behavior of the system.  Starting from what you know, you can look sideways, down, or up.

Actually, this is speaking too broadly.  Where, for example, do you fit "taking an object that you know something about, and suddenly understanding its purpose within a higher system"?  I suppose you could explain this as a variant of analysis - when the "Aha!" is done, the result is a better understanding of a system in terms of its constituents.  But then there are other knowledge problems, like guessing the properties of an element by taking the intentional stance towards the system and assuming the object is well-designed for its purpose.  Where does that fit in?  The moral, I suppose, is that "reductholism" has its uses as a paradigm, but there are limits.

Maybe we should generalize to generic causal models, regardless of level?  Then you could divide activities into noticing a property or interaction, deducing the cause of a property or interaction, or projecting from known causes to the expected results.  This model is a little more useful, since it sounds like the three problem types may correspond to three problem-solving methods:  (A)  Examine the model for unexpected regularities, correspondences, covariances, and so on.  (B)  Generate and test possible models to explain an effect.  (C)  Use existing knowledge to fill in the blanks (and, if you're a scientific mind, test the predictions thus created).

Still, even that view has its limitations.  For example, asking Why? or looking for an explanation isn't strictly a matter of generate-and-test.  In fact, generate-and-test is simply a genteel, thought-level version of that old bugaboo of AI, the search algorithm.  It seems likely that some type of "genteel search algorithm" - not "blind", but not really deliberate either, and with a definite random component - is responsible for sudden insights and intuitive leaps and a lot of the go-juice of intelligence on the concept level.  On the thought level, however, it's often more efficient to take a step back and think about the problem.  One implementation for thinking about the problem is "abstraction is information-loss" classical-AI-type "abstract thought", running the problem through with Unknown Variables substituted in for everything you don't know, to see if there are places where the Unknowns cancel out to yield partial results that would hold true of every possible solution, thus constraining the search space.  A more accurate implementation would be "applying heuristics that operate on the general information you have, to build up general information about the answer".

The thought-level is a genuine layer of the mind.  There isn't any simple way to characterize it.  There's a complex way to characterize it, which would consist of watching people solve problems while thinking out loud ("protocol analysis"), then figuring out a set of generalizations that corresponded to underlying neurology or underlying functional modules of the problem-solving method, and which categorized all the individual thoughts in the experimental observations.  This problem is large, but finite; the set of underlying abilities and mental actions is limited.  Still, such a project is beyond the scope of this particular section.  (What I will attempt to do, in later topics, is describe enough of the underlying abilities - enough that implementing them would give rise to sustainable thought.  Remember, seed AI isn't about perfectly describing the complete functionality of humans, it's about building minds with sufficient functionality to work.)

The thought-level is a genuine layer of the mind, and has around the same amount of internal complexity as might be associated with the modality-level or the concept-level.  The difference is that thoughts are open to introspection, and thus, when I make sweeping generalizations, my readers can catch me at it.  Nonetheless, I hope that the generalizations that have been offered here are sufficient to convey a vague general image of what goes on in a mind searching for knowledge.  Noticing interesting coincidences and covariances and similarities (looking sideways), building and testing and thinking about the reason why something happens (analysis, looking down in the holistic model, looking backwards in the causal model), trying to fill in the blanks from the knowledge you already have (prediction, looking up in the holistic model, looking forwards in the causal model).  The goal is a holistic model with good high-level/low-level bindings, or a causal model where the consequences and preconditions of a perturbation are well-understood, or a goal-and-subgoal model with plans and convergences and intentionality.  The goal is a model that holds together, on all levels, when you think about changing it; a model rich enough to support what we think of as intelligent thought.

It is literally impossible to draw a sharp line between understanding and creativity.  Sometimes the solution to a difficult knowledge question must be invented, almost ab initio.  Sometimes the creation of a new entity is not a matter of searching through possibilities but of seeing the one possibility by looking deeper into the information that you already have.  But, usually, when building the world-model, you're trying to find a single, unique solution; the answer to the question.  When trying to design something new, you're looking for anyanswer to the question.  Understanding is more strongly constrained, but this actually makes the problem easier, since a solution exists and the problem is finding it... the constraints might rather be called clues.

In invention, each constraint eliminates options and makes it less likely that a solution exists.  The distinction between understanding and invention is something like the difference between P and NP, between verifying a solution and finding it.  Returning to the quadrivium of Sensory, Predictive, Decisive, and Manipulative binding, and to Manipulation's sub-trinity of qualitative, quantitative, and structural bindings, then invention, or high-level manipulation, adds a fourth binding, the holic binding.  It's the ability to take a desired high-level characteristic and specify the low-level structure that creates it.  It's the ability to engage in hierarchical design, to start from the goal of rapid travel and move to a complete physical design for a bicycle.

The methods of invention are even less clear-cut than the methods of understanding.  Unless the problem is one of qualitative manipulation (choice from among a limited number of alternatives), the design space is essentially infinite.  An intelligent mind reduces the effective search space through possession of a holistic model that ultimately grounds in heuristics capable of direct backwards manipulation.  In other words, if you can choose any real number to specify the width of the wheel, what's needed is a heuristic that binds it - reversibly - to a higher-level design feature, such as desired stability on turns.  If desired stability on turns is itself a design variable, a heuristic is needed that binds it to a known quantity, such as the weight range of the rider.  And so on.

Such reasoning acts to reduce the search space from the space of all possible low-level specifications of a design, to the space of cognitive objects constituting reasonable high-level designs.  If there are enough heuristics left to constrain the design further, or to specify design features from high-level goals, then the task can be completed without special inspiration.  If there's a gap, a high-level feature with no heuristics that directly determine how it might be implemented, then there sometimes comes that special event known as an "insight", an intuitive leap.

Sometimes you try to invent the bicycle without knowing about the wheel.  The crucial insight may consist of remembering logs rolling down a hill.  It may consist of just suddenly seeing the answer.  Or it may lie in finding the right heuristic to attack the problem.  The key point is that a wide search space is crossed to find the single right answer, apparently without any guide or heuristic that simplifies the problem.  (If the aha! is finding the right heuristic, then the act of creativity lies in crossing the search space of possible heuristics.)

What is creativity?  Creativity is the name we assign to the mental shock that occurs when a large and novel load of high-quality mental material is delivered to our perceptions.  I would say that it's the perception of "unexpected" material, meaning "unexpected" not in the sense that the delivery comes as a surprise, but in the sense that our mental model can't predict the specific content of the material being delivered.  We perceive a thought as "creative", in ourselves or others, on one of two occasions:  First, seeing someone thinking outside the box; second, on perceiving a single good solution selected from a nearly infinite search space.  In the first case, a concept is redefined, or what was thought to be a constraint is broken; the answer is unexpected, which creates - to the viewer - the mental shock that we name "creativity".  The second case consists of seeing the very large gap between "high-speed travel" and "bicycle" crossed; the viewer - unless ve verself has designed a bicycle - has no single heuristic that can cross a gap of that size, that can anticipate the content of the material presented.  There's a nearly infinite space of possible paintings, so when we see any single painting of reasonable quality, a large quantity of unexpected cognitive material is delivered to our eyes and we call it "creativity".

It seems likely to me that the experience of creative insight happens when the mind decides to brute-force, or rather intelligent-force, the search problem.  The aha! of wheels comes because, somewhere in the back of your mind, possible memories were tested at random for applicability to the problem until the memory of logs rolling down a hill resonated with the problem and rose to conscious attention.  This unconscious "blind" search may employ some of the tricks of deliberation, such as searching through memories of objects that were seen traveling very fast.  (Or not.  It seems likely to me that only deliberate thought produces that kind of constraint.)  Even so, it remains in essence a try-at-random algorithm.  If there's anything more to subconscious creative insights than that, I don't know what it is.

When can an AI legitimately use the word "I"?

Understand that we are asking about a very limited and purely technical aspect of self-awareness.  We are not talking about the kind of self-awareness that will cause an ethical system to treat you as a person.  We are not talking about "qualia", the hard problem of conscious experience, what it means to be a bat, or anything of that sort.  These are different puzzles.

The question being asked is:  When can an AI legitimately use the word "I" in a sentence, such as "I want ice cream", without Drew McDermott popping up and accusing us of using a word that might as well be translated as "shmeerp" or G0025?

Consider the SPDM distinction:  Sensory, Predictive, Decisive, Manipulative.  A binding between a model and reality starts when the model "maps" in some way to reality (although this is ultimately arbitrary), becomes testable when the model can predict experiences, and becomes useful when the model can be used to decide between alternatives, with the acid test being manipulation of reality in quantitative or structural ways.  Consider also the distinction between modality-level, concept-level, and thought-level.

Self-modeling begins when the AI - let's call it Aisa, for "AI, self-aware" - starts to notice information about itself.  Introspective sensations of sensations are hard to distinguish from the sensations themselves, so this ball doesn't really get rolling until Aisa forms introspective concepts.  The self-model doesn't begin to generate novel information, information that can impose a coherent view of internal events, until it can make predictions - for example:  "Skipping from topic to topic, instead of spending a lot of time on one topic, will result in conceptual structures that are connected primarily through association."  Likewise, this information doesn't become useful until it plays a part in goal-oriented decisions - a decisive binding.

When Aisa can create introspective concepts and formulate thought-level heuristics about the self, it will be able to reason about itself in the same fashion that it reasons about anything else.  Aisa will be able to manipulate internal reality in the same way that it manipulates external reality.  If Aisa is impressively good at understanding and manipulating motorcycles, it might be equally impressive when it comes to understanding and manipulating Aisa.

But to say that "Aisa understands Aisa" is not the same as saying "Aisa understands itself".  Douglas Lenat once said of Cyc that it knows that there is such a thing as Cyc, and it knows that Cyc is a computer, but it doesn't know that it is Cyc.  That is the key distinction.  A thought-level SPDM binding for the self-model is more than enough to let Aisa legitimately say "Aisa wants ice cream" - to make use of the term "Aisa" materially different from use of the term "shmeerp" or "G0025".  There's still one more step required before Aisa can say:  "I want ice cream."  But what?

Interestingly, assuming the problem is real is enough to solve the problem.  If another step is required before Aisa can say "I want ice cream", then there must be a material difference between saying "Aisa wants ice cream" and "I want ice cream".  So that's the answer:  You can say "I" when the behavior generated by modeling yourself is materially different - because of the self-reference - from the behavior that would be generated by modeling another AI that happened to look like yourself.

This will never happen with any individual thought - not in humans, not in AIs - but iterated versions of Aisa-referential thoughts may begin to exhibit materially different behavior.  Any individual thought will always be a case of A modifying B, but if B then goes on to modify A, the system-as-a-whole may exhibit behavior that is fundamentally characteristic of self-awareness.  And then Aisa can legitimately say of verself:  "I want an ice-cream cone."

Humans also throw a few extras into the pot.  We have observer-biased social beliefs, a whole view of the world that's skewed toward the mind at the center, which tends to anchor the perception of the self.  We attribute internal causality to a monolithic object called the "self", which generates a lot of perceived self-reference because you don't notice the difference between the thought doing the modifying and the cognitive object being modified - the source of the thought is the "self", and the item being modified is part of the "self".

A seed AI will probably be better off without these features.  I mention them because they constitute much of what a human means by "self".

To support temporal metaphors and temporal concepts - to provide an °API with sufficient complexity for the °mindstuff to hook into - the AI needs modality-level support.  The most obvious method would be to tag all events with a 64-bit number indicating the nanoseconds since 1970 - a plain good-old-fashioned system clock.  The problem is that then the AI can't think about anything that happened before 1970.  Or about °picoseconds.

If we humans have a built-in system clock - there are several candidates, ranging from the heartbeat to a 40-°hertz electrical pulse in the brain - we don't have conscious, abstract access to it.  What we remember is the relative times; that event A came before event B, that event C was between A and B, that a lot of stuff happened between A and B, that D seemed to take a long time, that E seemed to go by very quickly, that E and F happened at the same time, and so on.  If I know that a particular event happened at 4:58 PM on July 23rd 2000, it's because I looked at my watch and associated the visual or auditory label "4:58" with the event.  That's why I can think - at least abstractly - about the age of the Universe or picosecond time frames.  Our abstract concepts for quantitative time aren't really built on our internal modality-level clocks, but on the external clocks we built.  Or rather, the internal modality-level clocks are used for immediate perceptions only, and the abstract concepts create the modality level through a layer of abstraction that can handle millennia as easily as minutes.

Because it's very easy to derive all the relative perceptions of time by comparing absolute quantitative times, we'll almost certainly wind up tagging every event with a 64-bit system-clock time (or equivalent interpreter token), and building any other modality functions on top of that.  It's just important to remember that the really important concepts about time should not be founded directly on the underlying, absolute numbers, because then the AI really can't think about picoseconds or pre-1970 events; the mindstuff making up the concepts will crash.  Concepts about time, if they refer to quantitative numbers at all, should be founded on the relative times of the cognitive events that occur while thinking about a temporal problem.  Thus the AI can imagine a process that takes place on picosecond timescales, and because the visualization itself takes place on nanosecond timescales (or whatever speed the AI's system clock runs at), there's no crash.  It's a kind of automatic scaling.

Another subtlety of human temporal understanding is that our senses are synchronized even though different senses presumably have different processing delays.  It takes time for the visual cortex to process an image, and time for the auditory cortex to process a sound - not necessarily the same amount of time.  But a physical sound and a physical sight that arrive simultaneously should be perceived as simultaneous.  Since a seed AI should be able to tag sensory events as distinct from the derivative perceptual events, this should be relatively easy to handle on the modality level... although it's possible to imagine problems popping up if there are °heuristics or concepts that act on the derivative and possibly unsynchronized high-level features of multiple modalities.

For some cases, this problem can be solved by only allowing multimodality concepts to act on events that have been completely processed by all targeted modalities.  If a vision and a sound arrive at t=10, the sound finishes feature-extraction at t=20, and vision finishes extraction at t=30, then no audiovisual concept can begin acting until t=31, with both the sound and the vision having a perceived time of t=10.  In other words, rather than skimming the cream off the modalities, the perceived now of the AI will lag a few seconds behind real time.

This introduces two new problems:  One, it may introduce severe delays into the system.  Modalities don't just apply to external sensory information; modalities are where all the internal thoughts take place as well.  To some extent this problem may be solvable by not requiring complete processing before concepts can activate, but only that level of processing which is necessary to the concept.  After all, a concept can't act on information it doesn't have.  But this may still lose some efficiency; there may be cases where concepts don't need synchronization.

The second problem is synchronization of subjective time.  If the AI's now lags a few seconds behind, when are thoughts perceived to have taken place?  If the AI thinks "foo!" at a time that looks to the AI like t=10 but is actually t=40, is the concept "foo!" labeled as having taken place at t=10 or t=40?  And what difference does it make?  I can't see that using t=40 makes any difference, so I'm strongly in favor of labeling all events as occurring when they actually occur.  Still, the AI may eventually find useful °heuristics that act on "subjective time".

"A general intelligence needs to be able to perceive and visualize when two events occur at the same time; when one event precedes or follows another event; when two sequences of events are identical or opposite-symmetrical; and when two intervals are equal, lesser, or greater.  Most of this comes under the general heading of having a feel for time as a quantity and time as a trajectory..."
        -- above

Hofstadter, writing about °Copycat - an AI that performs analogies in the domain of letter-strings, such as "abc->abd::pqrs->?" - notes that, despite the simplicity of Copycat's domain, the domain can contain analogy problems so complex as to embrace a significant chunk of human thought.  A few years back, when I was only beginning to think about AI, I set out to brainstorm a list of a few hundred perceptions relating to analogies - "before, next, grow, quantity, add, distance, speed, blockage, symmetry, interval..." - and noticed that most of them could be represented on a linear strip of Xs and Os.  These perceptions I collectively name to myself the linear intuitions - the perceptions that apply to straight lines.

The reactivation of the infrequently-used exploding-grape concept (or perceptual structure, if it doesn't rate a concept) should be enough to suggest that events are being repeated; enough to draw correspondences between each unusual pair of events.  The computational procedure for detecting reflection is simple enough that it could conceivably be run on every consciously perceived event-line where correspondences are drawn between events - at least, with respect to the events salient enough to have correspondences drawn between them.

Simultaneity is when two events occur at the same time.  Perfect simultaneity is when two events are tagged as occurring at exactly the same time, to the limits of the resolution of the modality-level system clock.  Even in AIs that totally avoid parallel processing, sensory modalities will tag all the components of an incoming image as having arrived at the same time, so any mind is full of insignificant simultaneities.  Significant simultaneities are those that are unexpected and that occur in high-level, salient objects.  For example, two objects simultaneously disappearing from a sensory input.

The human perception of intervals is approximate rather than quantitative.  We divide how long something feels into "less than a second", "a second", "ten seconds", "a minute", "ten minutes", "an hour", "a few hours", "a day", "a few days", "a few weeks", "a few months", "a few years", "a lifetime", and "longer than a lifetime".  (That's a guess.  I don't know the actual categories or their boundaries.  It would be an interesting thing to know, if someone has already done the research.)

Temporal precedence is which of two events - A or B - came first.  Precedence is the most often-used and most useful temporal perception; it is the one by which humans order reality.  We don't care about the exact intervals in milliseconds (although an AI might - see above); we care whether event A or event B came first.  Precedence is the most useful temporal intuition because it is the most deeply intertwined with causality - effects follow causes.  (See Unimplemented section: Causality.)

"Quantity" is invoked with every perception containing a real number, as ubiquitous as floating-point numbers in ordinary source code.  When I say "quantity", I do not just refer to a continuously divisible material substance, like water or time; I generalize to the internal use of floating-point numbers in representations and intuitions - all the perceptions that can be "stronger" or "weaker".

Given two quantities, we can notice which is more or less; given two quantitative properties, such as height, we can notice which is higher or lower; given two quantitative perceptions, we can tell which is stronger or weaker.  This perception can operate statically, in the absence of a temporal component.

As discussed in Unimplemented section: whenextract, quantities and comparators are too ubiquitous to initiate thoughts directly, unless the quantities and comparators are properties of very high-level objects; thus, low-level quantities and comparisions would be computed either as preludes to feature extraction, or only when demanded by the context of a higher thought.  Comparisions computed for feature extraction are also generally local.  A human visual pixel is compared with nearby pixels for edge detection, but not with every other pixel in the image, using O(N) instead of O(N^2) comparisions.  A seed AI should be able to compare arbitrary pixels in arbitrary modalities - but only on demand.  For more about the differences between on-demand and automatically-computed perceptions, the difference between low-level and high-level perceptions, and the difference between thought-initiating and guess-verifying perceptions, see Unimplemented section: whenextract.

The list of basic operations that can be performed on static quantities is basically the set of useful arithmetical operations:  Subtraction (in other words, interval calculation), comparision, equality testing.  It would also be possible to include addition, multiplication, division, bit shifting, bitwise & and |, remainder calculations, exponentiation, and all the other operations that can be performed on integers and floating-point numbers; however, these operations are less likely to be useful - less likely to pick out some interesting facet of reality.

A field of quantities, extended across time or space or both, can give rise to the mid-level features called patterns; patterns are higher-level than quantities, and richer, and rarer as a perception (a hundred pixels give rise to one pattern); thus, patterns are more meaningful.  Patterns can be broken, and the high-level feature that constitutes the breaking of a pattern is rarer, and far more meaningful, than either the patterns themselves or the low-level quantities.  (I speak here of modality-level patterns; the problem of seeing thought-level patterns is nearly identical with the problem of intelligence itself.)

One example of a pattern is a rising quantity - "rising" implying either a single quantity changing with time, or a field of quantities changing continuously with with some spatial dimension.

• A:  27, 28, 29, 30, 31, 32.
• B:  29, 31, 33, 35, 37, 39.
• C:  8, 19, 22, 36, 45, 71.

A modality observing D:  8, 16, 32, 64, 128, 256 should notice that the numbers are constantly increasing, and that the rate of the increase is constantly increasing.  A human modality would not notice that the numbers formed a doubling sequence - and neither, in all probability, should an AI's modality, unless the sequence is examined by a thought-level process.  I say this to emphasize that the problem of modality-level pattern detection is limited, in contrast to the problem of understanding patterns in general - if the AI's modality can understand a simple, limited set of patterns, it should be enough.

To notice a pattern is to form an expectation.  When this expectation is violated, the pattern is broken.  Observing a single quantity changing, as in sequence C, the feature "increasing" remains constant.  If C continues but suddenly starts decreasing - 8, 19, 22, 36, 45, 71, 62, 21, 7, 6, 1 - an "edge" has been detected.  On a higher level, this is what is observed:  "...greater than, greater than, greater than, less than, less than, less than..."  Thus the presence of the low-level feature detector for "greater than" or "less than" enables the AI to notice a pattern it could not otherwise notice, and to detect an edge it could not otherwise see.  That is the function of modality-level feature detectors:  To enable the discovery of regularities in reality that would otherwise remain hidden.

As a general rule, notice equality, continued equality, and broken equality in the quantity, in the first derivative, and in the second derivative.  We notice when a constant quantity changes and when a constant rate of change changes, but we humans do not directly perceive changes in acceleration.  We compute the quantity and the quantitative first derivative, but not the quantitative second derivative.  Since the second derivative - for humans - is not quantitative but qualitative, we can notice it crossing the zero line, or notice large (order-of-magnitude) changes, but not notice small internal variances.  An AI might find it useful to perceive the second derivative quantitatively, but computing a quantitative third derivative (and thus a qualitative fourth derivative) would probably not contribute significantly to intelligence outside of specialized applications.

There is still a question of salience.  We would wish a financial AI, or a human accountant, to notice and wonder if a bank account customarily showing transactions measured in hundreds of dollars suddenly began showing transactions measured in millions - the mid-level feature "magnitude", formerly constant at "hundreds", suddenly jumps to "millions".  But we wouldn't want to notice a change from the mid-level feature "magnitude: 150-155" to the mid-level feature "magnitude: 153-160", even though - on the surface - both look like equally sharp inequalities.  (As a °crystalline "compare" operation, "hundreds" != "millions" is neither more nor less unequal than "150-155" != "153-160".)  Similarly, we would not notice a change from the mid-level feature "frequency of numbers ending in 5: 20%" to "frequency of numbers ending in 5: 25%"; or, if we did somehow notice, we wouldn't attach as much significance.

We have learned from experience, or from our cultural surroundings, that money is extremely significant, that people often try to tamper with it, and that the order-of-magnitude of monetary quantities should be paid attention to; we have not learned a similar heuristic for shifts in a few dollars, or shifts in percentage frequency of digits, which is why monitoring either quantity is a specialized technique used only by auditors.

Learning which patterns and broken patterns to pay attention to is a concept-level problem; it's not trivial, but °Eurisko-oid techniques should suffice.

These are the feature extractors that can operate on quantities in general:

• Identity
• Equality and inequality
• Comparators
• Greater-than and less-than
• Perception of qualitative changes
• Expected change (within range of observed variance)
• Fractional change (plus or minus a few percentage points)
• Significant change (plus or minus a few dozen percentage points)
• Order-of-magnitude change
• Quantitative computation of first derivative, which is then another quantitative perception
• Computing a second derivative may sometimes work
• Computing a third derivative is probably useful only for specialized applications

On the concept-level, all these features should be computed for all salient high-level quantities, and for all higher-level features rare enough that computing all the features is computationally tractable.  Figuring out which features to compute for a quantity, and which features to pay attention to, is a major learning problem for the AI; learning in this area contributes significantly to qualitative intelligence as well as efficiency, since compounding extractors can lead to the computation of entirely new features.

On the modality level, these feature extractors can be composed to yield some basic mid-level features, such as edge detection in pixels, although anything more than that is probably a domain-specific problem.  For example, a problem as simple as computing changes in velocity will not fit strictly within the domain of quantitative perceptions, unless the velocity is broken up by domain-specific perceptions into quantitative components of speed and direction.

°Lakoff and Johnson, arguing that our understanding of trajectories is fundamentally based on motor functions, offer this list of the basic elements of a trajectory (quoted from "Philosophy in the Flesh"):

• A trajector that moves
• A source location (the starting point)
• A target (L&J call it a "goal"), an intended destination of the trajector
• A route from the source to the target
• The actual trajectory of motion
• The position of the trajector at a given time
• The direction of the trajector at that time
• The actual final location of the trajector, which may or may not be the intended destination

The concept of a trajectory can be represented in the temporal XO modality.  Zooming out from the following frame, "OOOOOOXOOOOOOOOXOOOXOOOOOO", it could be described as "three points on a line".  Given a temporal sequence of XO frames, the points on the line can "move"; they can have position, speed, direction, and velocity.

The XO modality suffices to represent an example of a trajectory, e.g.:  "XXOOOX", "XOXOOX", "XOOXOX", "XOOOXX"; an observing human would say that the middle X has moved from the starting point defined by the first X to the endpoint defined by the third X.  (Note that I do not yet use the word "target".)

For the sake of form, we should name all the intuitions giving rise to the start-move-endpoint perception.  The largest hurdle is the perception of each middle X as an instance of the same continuous object - that is, that the X at position 2 in t₁, the X at 3 in t₂, the X at 4 in t₃, and the X at 5 in t₄, are all instances of a single object with a continuous existence.  A human makes this interpretation immediately because we have built-in assumptions about the continued existence of discrete objects - domain-specific instincts that become visible within a few months after birth.

An AI could probably make the same interpretation, but it would be more difficult.  To establish a strongly bound perception of each X as a discrete object and the middle X as a continuous object, it would probably take a trajectory lasting, say, ten frames, instead of four.  Assume for the moment that the sequence is expanded to encompass ten frames and ten one-unit steps for the middle X.  In this case, the following facts are visible immediately:  First, that there are the same number of Xs in each frame.  (I will not say "three Xs in each frame", since this implies an understanding of "°three".)  Second, that each frame has an X in position 1 and an X in position 12.  To a human, it is "obvious" that the constant number of Xs implies a constant number of discrete objects; to a human, it is obvious that the three Xs are each different objects; to a human, it is obvious that an X maintaining an identical position in each frame is the same object in each frame; therefore, since the first and last Xs are accounted for, the leftover middle X in each frame must be the third object.  And indeed, the "movement" of the third object (or "shift in the positional attribute", as an AI might see it) is incremental and constant.

A tremendous amount of cognition has just flashed by.  Getting the AI to perceive two experiences as belonging to the same object is almost as deep a problem as that of getting the AI to perceive two objects as belonging to the same category.  Some of the underlying forces are visible in the source code of Hofstadter's °Copycat; Copycat can see two different letters in two different strings as occupying the same role.  (Copycat can also see bonds formed by "movements" in letterspace; it knows that "c" follows "b".)  The general rule, however, goes much deeper than this.

The Rule of Improbability implies that, the wider the range of possible values for an attribute, the more strongly equality of values implies equality of underlying objects.  "XOX" binds to "XOX" much more weakly than "roj" binds to "roj".  "3" binds to "3" much more weakly than "23,083" binds to "23,083".

Thus, even so basic a task as knowing when two experiences are the "same" object requires that the AI have previously have learned which attributes are good indicators of identity, which in turn requires that the AI have watched over objects known to be identical so that it can observe which attributes remain constant.  If this were a seminar on logic we'd be in trouble, but since we're pragmatists we can break the circularity by cheating, just as the human mind does - it seems highly likely that equality of visual signatures and continuous change in position are hardwired into the brain as signals of identity.  Similarly, we can start by identifying a few good attributes to begin with, and giving some sample sets with pre-identified objects, and letting the seed AI work it out from there.

What are the consequences of identifying an object?

(Author's note:  The discussion of objects should probably be somewhere other than 3.1.6: Trajectories, probably the section on categorization, and should have a much longer discussion.)

In what sense does labeling objects as "sources", "trajectors", and "destinations" - we will not use the term target just yet - differ from identifying them as "Object 1", "Object 2", and "Object 3"?  In what sense is a "path" different from a "trajectory"?  What expectations are implied by the labels, and what experiences are preconditions for using the labels?

Conceptually, a path can exist apart from the traversing objects.  If, on multiple occasions, one or more objects is observed to precisely traverse the same path - perhaps at the same speed - then a generalization can be made; an observed feature can be extracted from the single experience and verified to apply across a set of different experiences.  To observe the existence of a path is useful only if the observation is reflected in external reality - for example, if the reason a rolling ball follows a path down a mountain is because someone dug a trench.  A seed AI is unlikely to need to deal with physical trajectories of the type we are familiar with, but the metaphor of "trajectory" extends to the more important modality of source code - a piece of data can follow a path through multiple functions.

Similarly, the conditions that lead us to identify some object or position as "source" is that one or more observed trajectories originate from that source; what leads us to identify a position as "endpoint" is that one or more observed trajectories terminate at that endpoint.  What makes the perception of "source" useful is if there is a causal reason why the position is the source of the trajectory, especially if the object or position is actually generating the trajectors - if a pitcher throws a ball, for example; or, in AI terms, if a function outputs pieces of data that then travel through the system.  Similarly, the perception of "endpoint" is especially useful if the endpoint actually halts the trajector, or consumes it.

One cue that a real cause may exist - that the perception of a position/object as "source"/"path"/"endpoint" is useful - is if multiple, varying paths/trajectories have the same source or endpoint.  Imagine that a randomly moving point darts over a screen, and then the movie is played back three times; the fact that the sources and endpoints were identical may not mean that the sources and endpoints have any particular significance; the rest of the path was identical too.

Thus, the perception of "source" or "endpoint" exists whenever multiple trajectories share an starting position or ending position, and exists more strongly when multiple different trajectories share a source or endpoint but not other characteristics.  The perception of "source" and "endpoint" is useful when the perception reflects the underlying cause of the initiation or termination of the trajectory.

A "source" or "endpoint" can be any characteristic shared by multiple origins or terminating points, not just position.  If the trajectory of a grenade always ends at the location of the blue car, regardless of where the blue car goes, then it's a good guess that someone is trying to blow up the blue car - that the blue car is the endpoint.  The greater the variance, the less probability that the covariance is coincidence, and the stronger the binding.  The more unique the description of the endpoints - e.g., the blue car was the only car which shared a location with all endpoints, and the green car and the purple car were elsewhere - the stronger the binding.  This binding is predictive if it can be used to predict the position of the next trajectory termination by reference to the position of the perceived "endpoint", and manipulative if moving the perceived "endpoint" can change the trajectories - that is, if you can guess where the grenade will fall by looking at the blue car, and make the grenade fall in a particular place by driving the blue car there.  If the binding is strong enough, the endpoint may deserve the name of "target" (see below).

Finally, it is noteworthy that "source" and "endpoint" do not necessarily imply that the trajector goes into and out of existence.  Any interval which bounds the trajectory, or any conditions which bound the trajectory, or any sharp changes within the trajectory, may make salient the location of the trajector during the boundary change.  (To perform the computational operations which check multiple trajectories for binding of sources or endpoints, it is necessary that the source and endpoint be salient - salient enough that the additional processing is performed which discovers the binding.)