| Next: | 1.2: Thinking About AI | Bookmark | |
| Up: | 1: Paradigms | Monolithic | |
| Prev: | 1: Paradigms |
It is probably impossible to write an AI in immediate possession of human-equivalent abilities in every field; transhuman abilities even more so, since there's no working model. The task is not to build an AI with some astronomical level of intelligence; the task is building an AI which is capable of improving itself, of understanding and rewriting its own source code. The task is not to build a mighty oak tree, but a humble seed.
As the AI rewrites itself, it moves along a trajectory of intelligence. The task is not to build an AI at some specific point on the trajectory, but to ensure that the trajectory is open-ended, reaching human equivalence and transcending it. Smarter and smarter AIs become better and better at rewriting their own code and making themselves even smarter. When writing a seed AI, it's not just what the AI can do now, but what it will be able to do later. And the problem isn't just writing good code, it's writing code that the seed AI can understand, since the eventual goal is for it to rewrite its own assembly language. (1).
If "recursive self-enhancement" is to avoid running out of steam, it's necessary for code optimization or architectural changes to result in an increment of actual intelligence, of smartness, not just speed. Running an optimizing compiler over its own source code (2) may result in a faster optimizing compiler. Repeating the procedure a second time accomplishes nothing, producing an identical set of binaries, since the same algorithm is being run - only faster. A human who fails to solve a problem in one year (or solves it suboptimally) may benefit from another ten years to think about the problem; even so, an individual human may eventually run out of ideas. An individual human who fails to solve a problem in a hundred years may, if somehow transformed into an Einstein, solve it within an hour. Faster unintelligent algorithms accomplish little or nothing; faster intelligent thought can make a small difference; better intelligent thought makes the problem new again.
If each rung on the ladder of recursive self-enhancement involves a leap of sufficient magnitude, then each rung should open up enough new vistas of self-improvement for the next rung to be reached. If not, of course, the seed AI will have optimized itself and used up all perceived opportunities for improvement without generating the insight needed to see new kinds of opportunities. In this case the seed AI will have stalled, and it will be time for the human programmers to go to work nudging it over the bottleneck. Ultimately, the AI must cross, not only the gap that separates the mythical average human from Einstein, but the gap that separates homo sapiens neanderthalis from homo sapiens sapiens. The leap to true understanding, when it happens, will open up at least as many possibilities as would be available to a human researcher with access to vis own neural source code.
A surprisingly frequent objection to self-enhancement is that intelligence, when defined as "the ability to increase intelligence", is a circular definition - one which would, they say, result in a sterile and uninteresting AI. Even if this were the definition (it isn't), and the definition were circular (it wouldn't be), the cycle could be broken simply by grounding the definition in chess-playing ability or some similar test of ability. However, intelligence is not defined as the ability to increase intelligence; that is simply the form of intelligent behavior we are most interested in. Intelligence is not defined at all. What intelligence is, if you look at a human, is more than a hundred cytoarchitecturally distinct areas of the brain, all of which work together to create intelligence. Intelligence is, in short, modular, and the tasks performed by individual modules are different in kind from the nature of the overall intelligence. If the overall intelligence can turn around and look at a module as an isolated process, it can make clearly defined performance improvements - improvements that eventually sum up to improved overall intelligence - without ever confronting the circular problem of "making itself more intelligent". Intelligence, from a design perspective, is a goal with many, many subgoals. An intelligence seeking the goal of improved intelligence does not confront "improved intelligence" as a naked fact, but as a very rich and complicated fact adorned with less complicated subgoals.
Presumably there is an ultimate limit to the intelligence that can be achieved on a given piece of hardware, but if the seed AI can design better hardware, the cycle continues. To be concrete, if a seed AI is smart enough to chart a path from modern technological capabilities to nanotechnology - to the hardware described in K. Eric Drexler's Nanosystems - this should be enough computing power to provide thousands or millions of times the raw capacity of a human brain. (3). Whether the cognitive and technological trajectory beyond this point continues forever or tops out at some ultimate physical limit is basically irrelevant from a human perspective; nanotechnology plus thousands of times human brainpower should be far more than enough to accomplish whatever you wanted a transhuman for in the first place.
This scenario often meets with the objection that a lone AI can accomplish nothing; that technological advancement requires an entire civilization, with exchanges between thousands of scientists or millions of humans. This actually understates the problem. To think a single thought, it is necessary to duplicate far more than the genetically programmed functionality of a single human brain. After all, even if the functionality of a human were perfectly duplicated, the AI might do nothing but burble for the first year - that's what human infants do.
Perceptions have to coalesce into concepts. The concepts have to be strung together into thoughts. Enough good thoughts have to be repeated often enough for the sequences to become cached, for the often-repeated subpatterns to become reflex. Enough of these infrastructural reflexes must accumulate for one thought to give rise to another thought, in a connected chain, forming a stream of consciousness. Unless we want to sit around for years listening to the computer go ga-ga, the functionality of infancy must be either encapsulated in a virtual world that runs in computer time, or bypassed using a skeleton set of preprogrammed concepts and thoughts. (Hopefully, the "skeleton thoughts" will be replaced by real, learned thoughts as the seed AI practices thinking.)
Human scientific thought relies on millennia of accumulated knowledge, the how-to-think heuristics discovered by hundreds of geniuses. While a seed AI may be able to absorb some of this knowledge by surfing the 'Net, there will be other dilemnas, unique to seed AIs, that it must solve on its own.
Finally, the autonomic processes of the human mind reflect millions of years of evolutionary optimization. Unless we want to expend an equal amount of programming effort, the functionality of evolution itself must be replaced - either by the seed AI's self-tweaking of those algorithms, or by replacing processes that are autonomic in humans with the deliberate decisions of the seed AI.
That's a gargantuan job, but it's matched by equally powerful tools.
The traditional advantages of computer programs - not "AI", but "computer programs" - are threefold: The ability to perform repetitive tasks without getting bored; the ability to perform algorithmic tasks at greater linear speeds than our 200-hertz neurons permit; and the ability to perform complex algorithmic tasks without making mistakes (or rather, without making those classes of mistakes which are due to distraction or running out of short-term memory). All of which, of course, has nothing to do with intelligence.
The toolbox of seed AI is yet unknown; nobody has built one. This page is more about building the first stages, the task of getting the seed AI to say "Hello, world!" But, if this can be done, what advantages would we expect of a general intelligence with access to its own source code?
The ability to design new sensory modalities. In a sense, any human programmer is a blind painter - worse, a painter born without a visual cortex. Our programs are painted pixel by pixel, and are accordingly sensitive to single errors. We need to consciously keep track of each line of code as an abstract object. A seed AI could have a "codic cortex", a sensory modality devoted to code, with intuitions and instincts devoted to code, and the ability to abstract higher-level concepts from code and intuitively visualize complete models detailed in code. A human programmer is very far indeed from vis ancestral environment, but an AI can always be at home. (But remember: A codic modality doesn't write code, just as a human visual cortex doesn't design skyscrapers.)
The ability to blend conscious and autonomic thought. Combining Deep Blue with Kasparov doesn't yield a being who can consciously examine a billion moves per second; it yields a Kasparov who can wonder "How can I put a queen here?" and blink out for a fraction of a second while a million moves are automatically examined. At a higher level of integration, Kasparov's conscious perceptions of each consciously examined chess position may incorporate data culled from a million possibilities, and Kasparov's dozen examined positions may not be consciously simulated moves, but "skips" to the dozen most plausible futures five moves ahead. (5).
Freedom from human failings, and especially human politics. The tendency to rationalize untenable positions to oneself, in order to win arguments and gain social status, seems so natural to us; it's hard to remember that rationalization is a complex functional adaptation, one that would have no reason to exist in "minds in general". A synthetic mind has no political instincts (6); a synthetic mind could run the course of human civilization without politically-imposed dead ends, without observer bias, without the tendency to rationalize. The reason we humans instinctively think that progress requires multiple minds is that we're used to human geniuses, who make one or two breakthroughs, but then get stuck on their Great Idea and oppose all progress until the next generation of brash young scientists comes along. A genius-equivalent mind that doesn't age and doesn't rationalize could encapsulate that cycle within a single entity.
Overpower - the ability to devote more raw computing power, or more efficient computing power, than is devoted to some module in the original human mind; the ability to throw more brainpower at the problem to yield intelligence of higher quality, greater quantity, faster speed, even difference in kind. Deep Blue eventually beat Kasparov by pouring huge amounts of computing power into what was essentially a glorified search tree; imagine if the basic component processes of human intelligence could be similarly overclocked...
Self-observation - the ability to capture the execution of a module and play it back in slow motion; the ability to watch one's own thoughts and trace out chains of causality; the ability to form concepts about the self based on fine-grained introspection.
Conscious learning - the ability to deliberately construct or deliberately improve concepts and memories, rather than entrusting them to autonomic processes; the ability to tweak, optimize, or debug learned skills based on deliberate analysis.
Self-improvement - the ubiquitous glue that holds a seed AI's mind together; the means by which the AI moves from crystalline, programmer-implemented skeleton functionality to rich and flexible thoughts. In the human mind, stochastic concepts - combined answers made up of the average of many little answers - leads to error tolerance; error tolerance lets concepts mutate without breaking; mutation leads to evolutionary growth and rich complexity. An AI, by using probabilistic elements, can achieve the same effect; another route is deliberate observation and manipulation, leading to deliberate "mutations" with a vastly lower error rate. What are these mutations or manipulations? A blind search can become a heuristically guided search and vastly more useful; an autonomic process can become conscious and vastly richer; a conscious process can become autonomic and vastly faster - there is no sharp border between conscious learning and tweaking your own code. And finally, there are high-level redesigns, not "mutations" at all, alterations which require too many simultaneous, non-backwards-compatible changes to ever be implemented by evolution.
If all of that works, it gives rise to self-encapsulation and recursive self-enhancement. When the newborn mind fully understands vis own source code, when ve fully understands the intelligent reasoning that went into vis own creation - and when ve is capable of inventing that reason independently, so that the mind contains its own design - the cycle is closed. The mind causes the design, and the design causes the mind. Any increase in intelligence, whether sparked by hardware or software, will result in a better mind; which, since the design was (or could have been) generated by the mind, will propagate to cause a better design; which, in turn, will propagate to cause a better mind. (7). And since the seed AI will encapsulate not only the functionality of human individual intelligence but the functionality of evolution and society, these causes of intelligence will be subject to improvement as well. We might call it a "civilization-in-a-box", an entity with more "hardware" intelligence than Einstein (8) and capable of codifying abstract thought to run at the linear speed of a modern computer.
A successful seed AI would have power. A genuine civilization-in-a-box, thinking at a millionfold human speed, might fold centuries of technological progress into mere hours. I won't beat the point to death. I've done so in my other writings - Staring into the Singularity, in particular. It's just that the fundamentalpurpose of transhuman AI differs from that of traditional AI.
The academic purpose of modern prehuman AI is to write programs that demonstrate some aspect of human thought - to hold a mirror up to the brain. The commercial purpose of prehuman AI is to automate tasks too boring, too fast, or too expensive for humans. It's possible to dispute whether an academic implementation actually captures an aspect of human intelligence, or whether a commercial application performs a task that deserves to be called "intelligent".
In transhuman AI, if success isn't blatantly obvious to everyone except trained philosophers, the effort has failed. The ultimate purpose of transhuman AI is to create a Transition Guide; an entity that can safely develop nanotechnology and any subsequent ultratechnologies that may be possible, use transhuman Friendliness to see what comes next, and use those ultratechnologies to see humanity safely through to whatever life is like on the other side of the Singularity. This might consist of assisting all humanity in upgrading to the level of superintelligent Powers, or creating an operating system for all the quarks in the Solar System, or something completely unknowable. I believe that, as the result of creating a Friendly superintelligence, involuntary death, pain, coercion, and stupidity will be erased from the human condition; and that humanity, or whatever we become, will go on to fulfill to the maximum possible extent whatever greater destiny or higher goals exist, if any do.
To return to Earth: There will undoubtedly be many milestones, many interim subgoals and interim successes, along the path to superintelligence. The key point is that while embodying some aspect of cognition may be useful or necessary, it is not an end in itself. Treating facets of cognition as ends in themselves has led traditional AI to develop a sort of "trophy mentality", a tendency to value programs according to whether they fit surface descriptions. (One gets the impression that if you asked certain AI researchers to write the next Great English Novel, they'd write a 20-page essay on toaster ovens and then tear off through the streets, shouting: "Eureka! It's in English! It's in English!") My hope is that the lofty but utilitarian goals of seed AI will lead to the habit of looking at every piece of the design and saying: "Sure, it sounds neat, but how does it contribute materially to general intelligence?" After all, if an aspect of cognition is duplicated faithfully but without understanding its overall purpose, it's a matter of pure faith to expect it to contribute anything.
But that brings us to the next section, "Thinking About AI".
Since this curve folds in on itself, most "reasonable" images of the local curves for intelligence and efficiency, when combined, are likely to result in a breakthrough-and-bottleneck series at the global level. At least, this is what's likely to happen in the prehuman areas of the curve. Once a breakthrough carries the seed AI past the human level, I would expect the nanotechnology-to-Transition Guide "curve" to take over.
| Next: | 1.2: Thinking About AI |
| Up: | 1: Paradigms |
| Prev: | 1: Paradigms |