What is Intelligence?
October 12th, 2007 –
What is intelligence? It’s like pornography — you know it when you see it.
The way people talk, you’d think that intelligence was some sort of magical fairy dust. It’s not. Intelligence is a dynamic system that takes in information about the world, abstracts regularities from that information, stores it in memories, and uses it knowledge about the world to form goals, make plans and implement them.
Simple enough. The definition I give above covers humans, cyborgs, possibly intelligent animals, artificial intelligences, extraterrestrial beings, and whatever other form of intelligence you care to dream up.
There is no intelligence that isn’t a dynamic system. Intelligence is inherently a dynamic process. This one is a no-brainer.
There is no intelligence that doesn’t take in information about the world. It needs that information to learn and make plans. If it isn’t taking in information, something is wrong, like it’s locked in a box. But even an intelligence locked in a box takes in the information that it’s in a box.
There is no intelligence that doesn’t abstract information from its perceptual data. There’s simply too much to store otherwise. It has been estimated that we take in gigabits of perceptual data every second. Only a minority of that makes it into our long-term memory or plays a critical role in concept formation. Unless the data input is artificially constrained, an intelligence will always throw away most of the information it gathers.
There is no intelligence that doesn’t store memories. Even the main character of Memento had a perfectly functioning short-term memory. Without it, intelligence wouldn’t be possible. Lacking memories, we would be completely ignorant of both the past and the future. Not intelligent at all.
There is no intelligence that lacks goals. Even a randomly generated goal is still a goal. Uttering a word, turning your head to look at something, moving aside when a large object is incoming — these are all small goals. Any form of differential desirability constitutes goalhood. Without differential desirability, an intelligence will just sit still until it starves or otherwise runs out of power. An intelligence may derive its goals from external feedback… but that doesn’t mean it doesn’t have goals, just that it copies them from elsewhere.
There is no intelligence that doesn’t make plans. Making plans is an essential part of achieving goals. Intelligences visualize a goal state and then try to come up with a series of actions that will guide the current world into that goal state.
There is no intelligence that doesn’t implement plans. Admittedly, this is the shakiest of the above requirements, as there could exist an intelligence that is an Oracle, making plans but passing them on to others to implement. I would consider that intelligence as implementing its plans indirectly, as making plans inherently implies their possible implementation. Okay, so maybe I cheated on that one.
And so concludes my attempt at coming up with a somewhat detailed nonanthropomorphic definition of intelligence.
[…] (Cross-posted from SIAI blog.) […]
I can think of an intelligence-like thing that doesn’t have or implement goals - a Bayesian reasoning approximation that takes a prior distribution and some evidence and assigns probabilities to propositions. You might be able to feed such a thing experimental data, for instance, and ask for likely theories to explain it. (Wouldn’t a system something like this have to be part of any goal-seeking AI?)
“I can think of an intelligence-like thing that doesn’t have or implement goals - a Bayesian reasoning approximation that takes a prior distribution and some evidence and assigns probabilities to propositions.”
But in this case, the “goal” would be to assign probabilities. Keep in mind that it doesn’t need to be labeled “Goal” in order to be one. Even a two-line script of code can have a “goal”. Even if nobody calls it that. The Friendly goals of an AGI could be pre-assigned either in the procedures/source-code (but probably extremely difficult), or assigned as a repeated input (would require an AGI with the capacity to learn), or both.
Just passing through, and got stopped by the intelligence definition. I take it that the intelligence under consideration is an evolved, developed intelligence. Even so, it’s quite a stretch to include human information processing as a system that “takes in information about the world.” Humans don’t “take in” information, they create it. There are no “inputs” into organic systems.
I understand the need for short hand expressions of complicated processes, but that can be quite misleading. As they say, the devil is in the details. And the relevant details in human information processing is how the information is created and used, and, frequently as not, the usage is the information.
To provide one example: After enduring much criticism and complaint, a wife says to her husband, “Isn’t there anything about me that you do like?” “Sure,” the husband responds flippantly, “I like your dress.” What would be ordinarily be a small-talk compliment becomes a rather cruel insult. What is meant can’t be found in the words but are created in the usage.
Human information processing is semiotically mediated–is based upon the use of signs–and the example given is the essence of sign usage: a sign is something used as something other than what it empirically is. Thus information processing is at every level a creative, not passive, usage. What I’m talking about is probably the same kind of usage that results in a chimp running a stick down into an ant hill, pulling it out and eating the ants off the “ant collector.”
So I’d want to define human intelligence more in terms of the active creative use of information rather than a passive “taking in.” I know nothing of your enterprise, so this may be wholly irrelevant. But look at it this way: I might have been the kind of intruder who hits you up for a hand out.
Humans don’t “take in” information, they create it. There are no “inputs” into organic systems.
Why don’t sense organs count?
Why don’t sense organs count?
I’m going to go out on a limb here, and guess that the commenter’s a constructivist, and is differentiating between knowledge and sensory stimuli. That is, that raw sensory data doesn’t constitute knowledge (or meaning); instead, an intelligence constructs knowledge or meaning from sensory data based on context (such as personal experience, consciously learned heuristics, or unconscious behavioural tendencies).
That is, in the case of humans, there are no inputs of fully qualified knowledge, merely of sensory data that has to be interpreted.
Of course, I’m not so sure I’d claim that constructivist principles like that are a necessary part of intelligence - particularly if you take a layered approach to understanding intelligence, where layers above (very roughly) rely on abstractions produced by layers below).
Michael: How would you apply your definition deal with human (or, in theory, non-human) intelligence based on a breakdown of the intelligence into subparts, a la the society of mind - at what point do the various elements of an intelligence stop constituting intelligences themselves?
Nick,
Sensory systems (sensation and perception involve more extensive pathways than “organs” suggest; for example, you can get motion sickness and vertigo from visual experience in the wide screen movies) are response systems, not “inlet” systems. The neuro-chemical events of your system are entirely of your system’s making. Think of them as a code, a system of variance in some degree of covariance with a reasonably regular system of external variance. Morse code looks nothing like the alphabet but can be considered a “covariant” or analogue of it. Our heat sensitive nerves reliably respond to temperature increase–generally. We have some cold-specific nerves that will respond with the sensation of coldness when touched with a warm object. It is covariance and not isomorphism that we depend upon for reliable information. Fortunately, we have finite sensory capabilities and apparently a non-random universe, so the principles of differentiation and correlation of sensation based upon relevance serves us well.
A quick PS: Does any of what I’ve written have any relevance to what you guys on this list are interested in?
Dumaurier-Smith,
I understand what you’re saying, but I don’t see the contradiction to Michael’s point, and I don’t think it has as much importance as you think it does and isn’t even accurate in some cases.
For one thing, I understand the human brain actually derives the meaning of information and that specific information doesn’t come externally, but this is still just a modification of the externally gathered information. It is completely transformed, no argument there, but the transformation within the brain still takes place on information that was produced 100% outside the brain. So it is in fact an input. The brain can’t emulate the universe inside itself, it’s too small, so it gathers that information from the outside and THEN transforms it.
As for as the comment ” The neuro-chemical events of your system are entirely of your system’s making.”, This is wrong. In order for your rod/cone cells in the retina to fire for example, light particles coming from outside the brain, perhaps coming from many miles away, have to physically ENTER the cell walls of rod/cone cells and interact with their proteins to produce an electro-chemical spike. This spike then gets transformed and relayed many times into a form of information the rest of the brain can use, but there still is a relationship between that end product and original neural spike produced by a foreign photon.
Thoughts?
DavidW:
Well, I’m not all that interested in getting a hearing; I was more interested in hearing artificial intelligence conceptions of intelligence. However . . . , your terms beg the question. You begin with the assumption that information preexists the system’s processing and your terms service that assumption. I have in mind terms such as: “derive,” “modify,” “transform.” So you make assertions, but don’t demonstrate what you claim.
No, I’m not wrong when I say the electro-chemical events of your system are entirely of your system’s making,” and your illustration does not show otherwise. All it does it make the obvious point–which I of course noted–that the system responds to environmental stimuli. There is no difference between the light particles occasioning sensory response and the heat energy from a probe placed on the skin. For there to be input, you’d have to show that the light energy, and not the electrochemical energy of the system, is processed by the system. The system responds to impingement in terms of its own energy system, its responses are not isomorphic with the energy of the stimulus impingements, and even those sensory experiences initially are not intrinsically informational. Learning is required.
So when you talk about information produced 100% outside the brain I can make no sense of that. It sounds metaphysical. How can information be produced outside the human system before the human has access to it? Who’s producing it?
Let me take one step back and try again. All information is mediated; there is no direct apprehension by mortals of anything. Gods may apprehend existentials directly, but not humans. The medium in which information is achieved is irrelevant to the information itself; it is the use of the medium that is informative. Sight, for example, is not about sensory responses to light energy. What we now take to be vision is not what we physiologically experience, but rather the information we’ve gained from learning how to use the sensory medium. If all the information we processes as vision could be translated into auditory sensation, we could see with our ears. It’s the same for all media. When you hear a foreign language, you hear language at the medium level; when you learn the language, you hear information, and it’s difficult to return to the medium level experience. I repeat, information is not input into the organic system. Information is created within it.
That’s gotta be it. I chased a link on a whim to find out what artificial intelligence theorists think intelligence is. I think I’d learn more by just reading.
Dumaurier-Smith,
It doesn’t sound like you really addressed the arguments I made. Let me explain.
You are drawing a hard line between externally produced stimuli, and “Information”, as you define it, that the brain actually uses.
The question is, why should there be this hard line? Why shouldn’t it be more of a gradient from unusefull information to highly usefull information? The pattern and arangement of photons hitting your retina is mathamatically related to the pattern of neural firings that travel to the visual cortex along the optic nerve. There’s a relationship because the “stimuli” is analyzed and acted upon by the retina cells. They do what the light tells them to do. Obviously, that “stimuli” has meaning to the brain otherwise we wouldn’t have evolved to use it and we’d all be blind.
Think about it. The neurons in the retina are just COPYING what the photons are doing. They just copy and transport this INFORMATION to the visual cortex (with some pre-modification I know, but moot point.)
The problem may be you’re thinking of information in purely physical terms, and not it’s existence just on a purely conceptual basis. Information isn’t physical molocules, or isn’t even the photons themselves. It’s DIFFERENTATION, or SPECIFICITY. In other words, think of information as being represented by physical or energetic modes but the information itself is still abstracted from that. And that’s why it’s not usefull to say the pattern of photons hitting the retina is fundementally different than the pattern of neural firings in the visual cortex, because they are all just implementations of a set of differentations. Differentations that were produced 100% externally and have meaning, regardless of when or how that meaning is derived.
Leaving aside the sensory systems (consider REM sleep), the mind still cannot function without input. I suppose it depends on your definition of “input”. A neuron that receives no signal (no input) will perform no useful processing. And a mathematical operation (eg. Add 2) will mean nothing without input to operate on (Add 2 to 3 = 5).The brain uses transitional (or transient) outputs for its processing. Transitional in the sense that the transient output of one neuron is used as input in a different neuron. Think of general anethesia; the algorithms are present, but the throughputs (inputs/transitional outputs) aren’t propagated or processed; and the patient is neither intelligent nor conscious. I think the problem here is a gray area of definitions.
David W:
With all due respect, I saw no arguments, only assertions. As regards mathematical relationships between photons and electro-chemical neural responses, such relationships are a truism for any mediating relationship, for any covarying systems. The regularities of covariance make mediation possible, but are irrelevant to the information derived from the use of the medium. What you’ve pointed out is on a par of noting that the temperature on a thermometer varies reliably with heat energy. However, we don’t confuse the reading with actual heat energy.
Why should there be a hard line? Because covariance between systems does not imply continuity between the systems, and no such continuity can be empirically demonstrated.
You say, “The problem may be you’re thinking of information in purely physical terms, and not it’s existence just on a purely conceptual basis. Information isn’t physical molocules, or isn’t even the photons themselves.” David, that is the exact opposite of what I’ve argued from the outset. I’ve consistently argued that information is not in the external impingements upon the organism, is not in the sensory system (which is all the organism experiences) but results from the creative use the organism makes of the sensosry system as a medium.
” Differentations that were produced 100% externally and have meaning, regardless of when or how that meaning is derived.” As I said before, that sounds metaphysical and I can make no sense of it; when I think of differentiations and meaning, I need to tie those to an actor and interpreter.
Jeffrey H:,
I think you’re right about definitional problems. I’m sure DavidW’s posts make perfect sense to this list’s readership. But the image I get from them is of prefab information and meaning everywhere in the environment being injected into organisms. And my view of information as mediated, created, obviously strikes him and perhaps others here as equally strange. So I”m going to give up on this. My intent to stimulate discussion of intelligence/information relationships by members on the list has resulted in a profound “Clunk!” and I’ve learned nothing.
Dumaurier-Smith,
“With all due respect, I saw no arguments, only assertions. ”
My argument was very clear, there is no conceptual difference between information as it exists in the outside world and as it exists in the brain, only that it is in a different form. You seem more interested in arguing over semantics. If you only saw assertions, then clearly my argument is that those assertions are true. Why bother even making that statement? Don’t mean to seem rude, but those kinds of statements are anoying and completely unproductive to a discussion.
” The regularities of covariance make mediation possible, but are irrelevant to the information derived from the use of the medium.”
Irrelevant? How So? It can’t be completely irrelevant in a general sense, otherwise we wouldn’t have eyes, like I pointed out. What else could you possibly mean? Our senses exist to form a representation of the outside world, because if our understanding of the world was not representative, we would trip and walk into walls all the time. The information we understand about the world is representative of the raw data coming in from the outside. What’s irrelevant about that?
“What you’ve pointed out is on a par of noting that the temperature on a thermometer varies reliably with heat energy. However, we don’t confuse the reading with actual heat energy.”
Uh, Don’t we? When we read a thermometer that says “200 F” don’t we know if we touched that 200 degree object it would feel hotter than a 100 degree object? Isn’t that because the molecules in the object are bouncing around much faster, which in turn caused the thermometer to read 200 F instead of 100 F? Like I noted before, the information is representative of the physical reality.
‘Why should there be a hard line? Because covariance between systems does not imply continuity between the systems, and no such continuity can be empirically demonstrated.”
No continuity between the outside world and the brain? If this were true, the retina cells would fire in the exact same pattern regardless of the light hitting the retina. But as you know, it doesn’t. The pattern always varies based on incoming photons. What’s not continuous about this? What definition of “continuity” are you using that the variance in the retina doesn’t fit the bill?
“As I said before, that sounds metaphysical and I can make no sense of it; when I think of differentiations and meaning, I need to tie those to an actor and interpreter.”
I could not imagine a less metaphysical statement. When light bounces off a rock, the reflection contains variance that was produced by the surface of the rock. Therefore the variance was produced outside the brain, because the rock is not embedded inside of the brain. Again, no brain can simulate the universe, therefore it must take in information FROM it in order to act intelligently TOWARDS it.
Let me rephrase my argument in a different way. Try answering this question:
At what point does “information”, as you define it, get created? Since it’s not in the objects altering photons, Then is it in the retina? The cells directly behind the retina but still in the eye? The first layer of the visual cortex? Second layer? Third? Forth? Or in the white matter of the brain? How can you even know since no one really knows how the brain works at this point. How can you claim to understand information as only that which exists in the brain if you don’t even know how the brain works?
“My intent to stimulate discussion of intelligence/information relationships by members on the list has resulted in a profound “Clunk!” and I’ve learned nothing.”
It’s your choice to leave of course. But you should know that there’s only a few other places on the Internet that are as open-minded and coherent around a common goal as SIAI. Technical discussion will always bring some degree of disharmony no matter where you go.
Good post Michael. My working hypothesis about “intelligence” is that it is best viewed and defined in ways that separate it from “consciousness”. I’d say intelligence is best defined such that it can exist without consciousness or self-awareness. Thus I’d refer to a computer chess program as intelligent, but not conscious or self aware.
I would suggest that intelligence is a prerequisite for consciousness which is a prerequisite for self-awareness, but separating these three things seems to avoid some of the difficulties of explanations that get bogged down as we try to develop models of animal and non-animal intelligence. Also, I think this will describe the development curve of AIs which are already “intelligent”, but none are yet “conscious” or “self aware”. I think consciousness may turn out to be simply a *massive number* of interconnections carrying on intelligent internal conversations within a system - human or AI.
A stumbling block I find very interesting is the absurd notion that human intelligence is fundamentally or qualitatively different from other animal intelligences. Although only a few other species appear to have self-awareness, there are many other “conscious” species and millions of “intelligent” species.
This is the last post! First, to Jeffrey H.: I’ve been on the internet too long to put my ego where it doesn’t belong. I’m not going away mad or because I’m mad. I was just passing through and spent more time than I intended. I didn’t find what I’d hoped for, but that’s not an unusual result in life, and communication’s a transaction. I’m in for at least fifty percent of the blame, no?
To David: I’m sorry we didn’t get anywhere and I certainly did not intend to be rude. I stated my perceptions regarding arguments, and that’s all. I don’t think a justification of the perceptions would be of use now.
Let me make an observation and answer your last question and the floor will be yours for as long as you want it.
[Flag! My perception!] You sometimes appear too impatient to read carefully, and respond tangentially or superficially. For example, I noted that thermometer readings vary reliably with heat energy, but we don’t confuse the reading (!) with actual heat energy. You responded with an “Uh . . .” and a denial. The reading is not the heat energy. You don’t burn your finger on a reading. And that’s not merely a semantic difference. (The thermometer is also a good example of mediated information. We have a column of mercury (or digital read out) of a certain height or amount. What does it mean? That the tomatoes will freeze? That we should wear a jacket? That our heating bill will climb? The information depends upon how we use it, and the use is the information.
As to when information is created and how we know it, I pretty much answered that in my first post. But to be accurate, I never claimed information existed only within the brain. Most neuro-physiological studies of consciousness and perception I know of do not restrict information processing to the brain, and neither do I. There’s all sorts of evidence that information processing, including learning, occur outside the brain. We don’t know how the brain works, true. But let me suggest something to you. Excluding most of the data processed by the autonomic system, and especially including data processed by the exteroceptive, interoceptive and proprioceptive sensory systems: when the last electrochemical event in the human system is charted, we will still be unable to predict how it (and all its involved correlates) will be used in any particular time or circumstance. That is partially explained in my comments on mediation. We do not experience the sensory events of our system as meaningful, as information, until we’ve learned to use them as a medium. And when we’ve learned to use them as a medium, like all medium events–e.g., words– they take on an arbitrary quality and can mean what we learn they mean. Some people even find pleasure in pain.
The answer to how we know when information has been created is what it’s always been. How do we know someone is conscious? We have to have behavioral evidence. Especially in my pragmatic conception of information, we don’t know information has been created until we see it used. But even in syntactic, structural conceptions, something will have to index the efficacy of the information–typically a change of state of some kind.
[…] Michael Anissimov asked over there ”What is Intelligence” and offered up a definition that could apply to human as well as artificial intelligence. […]
I thought this was interesting to come back to. FWIW, I think that both DavidW and Dumaurier-Smith had useful things to say, but definition problems were impeding progress. I would describe it as human neurons “modeling” reality. And not with 100% accuracy. We humans have a *usually* decent understanding of reality. Without understanding it completely. Some people understand it better than others. For example, we don’t see or directly detect quantum events - even though quantum physics is the most “accurate” account of reality that we have presently. We humans have a model of reality that we operate with. Different people have different models as a result of having different algorithms. A mind won’t function without throughputs, but in humans the throughput doesn’t really “originate” from outside the brain. For example, when we see a green light, it’s not because actual green photons are physically traveling through our brain. We “see” green light because an initiating neuron(s) responds “deterministically” (in a degree of correlation to reality) to the stimuli and then propagates the throughput to other neurons. However, I do believe that “information” does exist outside the brain, if perhaps “meaning” does not. If the external Universe was truly, totally random then no patterns at all could ever be “extracted” from the throughputs. In that case, a thought could never exist, no matter how potentially intelligent the algorithm was. And two normal people can usually agree that they are looking at the same thing - evidence of some degree of objectivity in the Universe.
“A mind won’t function without throughputs, but in humans the throughput doesn’t really “originate” from outside the brain.”
Just to clarify, I don’t believe that this negatively impacts the feasiblity of AGI. An advantage of computers is that the input can already be of the same “language” as the algorithms. And doesn’t necessarily require an “intermediary” such as a human sensory system. Although intermediaries can certainly be added if they are desired or beneficial, and there are plenty of examples in modern software.
Yo, Dudes -
Y’all are barkin’ down the tree, when ya should be barkin’ up the tree. Y’all got it all wrong. Ain’t gonna be no AGI. . .wanna know why? Cuz the “general” is in the human. Cain’t put the general in the machine cuz the machine is inert. . .dead. . .lifeless. . .capiche? Ya cain’t make life, but ya can try to fake it. Member now, it’s the good Lord that dun made the soul. . .all ya can do is try to copy the mind, and the mind ain’t from the Lord. . .only the Soul is. But keep on tryin’. . .somethin’ will happen and there will be a market fo it. . .like ten million and second life! OOOOOeeeeeee, wuz this wurl comin’ to!
Hasta la Pasta,
Had nothing to do for a moment, so I thought I’d obsess a little more. Regarding Friendly AI, I believe that no mind will function without throughputs being propagated (including software-based minds). And I suspect that if one can tightly control the throughputs specifically, that one can reliably direct a mind. The Friendly-Goals throughput would already be in the form of information, and it would depend on the AI algorithms(mind) to provide the meaning of that information. The raw information does not need to be “meaningful” initially. It needs only to be eventually “perceptible” (sufficiently probabalistically complex, following sufficient processing). For example, It is the algorithms of my brain(mind) that provide the meaning to the uttered sentence: “Could you bring me a cup of coffee?”. The raw *information* simply took the form of traveling air-pressure gradients (”sound”) - not the meaningful “native language” of the brain by any means. The raw information did not have to be algorithmically encoded at the start. Yet, the end result was that I fetched the cup of coffee as requested. It is true that one could argue that I only fetched the coffee because I wanted to on some level. But that is not necessarily relevant; a Friendly AI will have no innate desire to not follow its Friendly goals. Because our selected goals will be its goals. Any criticism is welcome. Cheers.
But I suppose that a self-improving mind might complicate this somewhat; I assume that a self-improving mind will need to “observe” a model of itself. But perhaps instead of “providing” (as input) the mind with a 1:1 replica of itself, the self-model could be “encoded” and left for the AI mind to “deduce”. IOW, the AI mind intermittently “receives” both the original (unprocessed) *input* (the fixed goal-system) and it “receives” the processed *output* of the same thing (in a fashion similar to a self-reinforcing neuron loop - but using the mind as a whole). And using both the input and the output, it could probabalistically “deduce” what the procedures “had to be”, in a way similar to solving an algebraic equation. For a ridiculously simplified example: 3 (+ X) = 8. One can deduce what the (net) procedure (X) or algorithm was, in this case it was “add 5″. Sorry for the sloppiness of this post, but that’s what happens when you try to write way outside of your secure knowledge base.
But perhaps a “self-model” won’t turn out to be necessary. After all, we humans will likely create an AGI without having a 1:1 replica of our own mind (algorithms). Perhaps having a good breadth and depth of “world knowledge” is adequate to write a new, distinct intelligent program. Maybe our AGI will just need to become a parent. 
“Maybe our AGI will just need to become a parent.”
But one could reasonably argue that rewriting your own mind *is* creating a new mind anyway.
Sorry for the maniacal posting, but there’s something I keep forgeting to include. When using the Friendly Goals as throughput, I presume that they (the goals) could be “dynamically” weighted so as to always have priority over other sorts of throughputs, such as environmental data. Without regulation, different “goals” could inadvertently be derived from both the procedures and the data input - when used as throughput. Perhaps the different dynamic weightings could also be given some degree of “dynamic” permanence as the Friendly Goals become embedded in the AI’s short and long-term memory. I doubt that simply tagging environmental data as “non-goal material” would work on its own.
“Perhaps the different dynamic weightings could also be given some degree of “dynamic” permanence as the Friendly Goals become embedded in the AI’s short and long-term memory.”
After all, when we humans seek to accomplish a multiple-part goal, it is the (dynamic) higher-level “super-goal(s)” that we store in short-term memory, while we focus on first achieving the necessary sub-goals (as part of that particular super-goal). Perhaps favorably weighting the attention allocation would be another useful strategy, to help with Friendliness.
Hey Jeffrey, I believe a lot of your theories on AGI and goals are correct. Friendly or any kind of goals for an AGI would already be in whatever format that AGI needs for it to seamlessy interact with the rest AGI’s mental process. This also means, like you said, transfering knowledge from one AGI to another would be much easier and accurate than through some sensory intermediary. Imaging how many debates could be avoided in humans if you could literally transfer the mental process showing someone is wrong directly into their brain!
Though I think a lot of speculating on the inner workings of an AGI mind really just depends on the specifics of the AGI design, and such speculation can only be proven or disproven when compared against a specific AGI design for refrence.
I think describing something as “information without meaning” is also a good idea. This was my argument: It’s impossible for it to be correct to have information derived from non-information, as it seemed Dumaurier-Smith advocated. Merely deriving meaning FROM information however, is much easier to stomach. Not to say that all information doesn’t have some meaning to it already, just not in the right format for humans to parse directly. I guess my argument is also that the word “information” does not have enough semantic depth to describe both the causual variance in sensory data and the way the brain must transform that information into a format it can intermediate with the rest of the brain. The two are different, One is raw information, the other is brain structured information derived from raw information, but they both contain data produced non-randomly by the universe which means it is usefull to call both “information”.
“Though I think a lot of speculating on the inner workings of an AGI mind really just depends on the specifics of the AGI design, and such speculation can only be proven or disproven when compared against a specific AGI design for refrence.”
I’m going to hazard a guess and make a proclamation. If I turn out to be wrong, someone can tell me “I told you so!” a billion years from now.
I think that any functional AGI design will have to be based on algorithmic probability. Check out the “Gentle (?) Introduction” to AIXI by Marcus Hutter. It’s a mathematical formalization of AGI (but an actually computable version of AIXI hasn’t yet been devised). But I think it’s clear that progress is being made in that direction. I think it will be, *at most*, 20 years before AGI is solved (I would guess, closer to 10). Let’s have Friendliness theory complete before that time.
“any functional AGI design will have to be based on algorithmic probability”
I dunno about that. I’ve read about the AIXI theory, and it basically boils down to
“try everything, and keep doing something if your human master likes it”
This is no functional AGI design.
Not that I think that AGI is an insoluble problem - far from it, but I just think that AIXI is a bit of a dead end.
“I dunno about that. I’ve read about the AIXI theory, and it basically boils down to
“try everything, and keep doing something if your human master likes it””.
If it uses algorithms then the resulting AGI will have to be goal-driven. The goals being (either directly or indirectly) probabalistically derived from the low-level procedures; where the procedures are reflected in the “constructed” throughputs. No throughput - no thoughts and no goal satisfaction. That’s what I’m guessing at least. The possiblity that humanity will select the specific goals is irrelevant from the AGI’s perspective. In fact, it’s physically impossible to even *create* an AGI *without* intentionally or unintentionally, selecting a set of (possibly basically random) initial goals.
“This is no functional AGI design.”
Nobody has claimed that AIXI (as it currently stands) is a practically functional AGI. It’s more of a technical specification of what an *optimal* AGI would be. But I think that it’s an invaluable step in the correct direction. Until AIXI, nobody had been able to provide a universal mathematical formalization of what an AGI even is, precisely.
“Not that I think that AGI is an insoluble problem - far from it, but I just think that AIXI is a bit of a dead end.”
I disagree. But that’s what keeps our world turning.
“No throughput - no thoughts and no goal satisfaction.”
I meant to say: No throughput - no thoughts, and no goal perception or satisfaction.
To add a bit more. By pre-specifying the intial throughputs (in the form of something the baby AGI has already learned something about - eg. “Bring me a cup of coffee.”) and tightly controlling the throughputs while being propagated through the algorithms; one might be able to reliably assign the Friendly goals in that fashion. The Friendly throughput would need to be in a “recognizable” form to start with; but it doesn’t necessarily need to be pre-processed by a sensory system - like human brains include. It could take the form of a *pre-specified* “thought” if you will. The baby AGI doesn’t necessarily need to *hear* or *see* the words : “Bring me a cup of coffee.” Although one could also have it pre-processed too, if desired. Or so I say.
I was thinking about Reflection some more. Is it possible that the reason that the math breaks down is because a self-improving AI will decide *not* to deliberately and directly modify the “the part of itself that decides how to modify itself” (eg. it’s knowledge matrix)? But instead to deliberately modify other parts of itself - for example it’s learning sub-routine, or learning methodology. [Perhaps an AI can’t *improve* its own knowledge matrix without it having first aquired (learned) the additional knowledge to begin with.] And thereby deliberately improving its intelligence indirectly. Maybe human Reflection also works by other (non-deliberate) mechanisms, such as an algorithm recursively processing it’s own continuously changing output. I dunno.
Nah, nevermind. I don’t know anything about Decision Theory, so I’m going to make up a bunch of terms here to get an idea accross. Why? Because it’s fun.
Perhaps the problem isn’t the classical math, but the assumptions about what is being processed. Perhaps human Reflection also involves an “infinite” recursion, among other factors. (The human brain has *some sort* of throughput flowing through it all the time, anyway). And where the recursive throughput is always *changing*, it would indeed just create an endless (and effectively useless) stream of conceived: a sub-goal of a sub-goal of a sub-goal of a sub-goal… ad infinitum. And nothing would ever get accomplished. But what if the “main goal” or what I’ll call the 1st-Tier goal was “preserved” from continuous modification and was repeatedly “released” into the throughput stream? (For example, preserved algorithmically - by being stored in short-term memory like humans seem to use. Or being preserved as a separate data-file that’s fed into the algorithm). For every cycle of the preserved 1st-Tier goal, the algorithm would conceive/generate a (distinct) 2nd-Tier sub-goal. (Distinct perhaps because human cognitive algorithms are constantly changing ever so slightly in response to processing throughputs. Perhaps this effect could be reproduced by virtue of the AI’s learning sub-routine… Or perhaps the 2nd-Tier sub-goals are individually distinct for a different reason). For example, after 50 cycles of the 1st-Tier goal, 50 (distinct) 2nd-Tier sub-goals will be conceived/generated; as well as 50 (distinct)3rd-Tier sub-goals and so on. In the “background” all of the tiered goals are also still being processed in the “infinite” recursion. For example, a cycle of the 1st-Tier goal generates a 2nd-Tier sub-goal, a cycle of the 2nd-Tier sub-goal generates a 3rd-Tier sub-goal… towards generating an Nth-Tier sub-goal. But the important point is that the preserved (unmodified) 1st-Tier goal is repeatedly being released as throughput through the algorithm (along with all the others). Perhaps attention-allocation accounts for the remainder of the phenomenon. Perhaps our human attention allocation remains “focused” “near the top” where our thinking is dominated by the consideration of perhaps mostly the 1st, 2nd, and 3rd-Tier sub-goals and very little attention is allocated to the more distantly generated sub-goals like the 99th-Tier sub-goal. So mostly what we are (repeatedly) thinking about is 2nd and 3rd-Tier subgoals (just to pick out a small number). Perhaps human Reflection is the result of a complex interplay between attention-allocation, a “preserved” 1st-Tier goal (a super-goal) implemented as repeated throughput, and an “infinte recursion” algorithm (eg. the human knowledge matrix). Anyway, I just made this up from ~0 knowledge of Decision Theory, so it might not be useful, but whatever… ‘Till next time.