Comments on: Three Major Singularity Schools http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/ The Singularity Institute exists to confront the challenge of powerful AI, both the opportunity and the risk. Fri, 04 Jul 2008 18:06:46 +0000 http://wordpress.org/?v=2.1.3 By: Cosmic Vortex http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6852 Cosmic Vortex Mon, 01 Oct 2007 01:40:25 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6852 Nice essay. I understand what you are saying about the 3 different projections contradicting each other, I think the odds that they do are actually very small. The only way it would be possible for these views to contradict would be if the superintellgence that emerges chooses not to continue on Moore's path. While this is certainly possible, I think its very improbable that it would choose to do so. I think theres at least one reasonably safe assumption we can make about the intelligence beyond the singularity... it will continue to expand its intelligence and power. This will mean continuing on Moore's path at least until it gets a technology that can replace it altogether. If thats a contradiction between views, I think it's a bit too small to count. Nice essay. I understand what you are saying about the 3 different projections contradicting each other, I think the odds that they do are actually very small.
The only way it would be possible for these views to contradict would be if the superintellgence that emerges chooses not to continue on Moore’s path. While this is certainly possible, I think its very improbable that it would choose to do so. I think theres at least one reasonably safe assumption we can make about the intelligence beyond the singularity… it will continue to expand its intelligence and power. This will mean continuing on Moore’s path at least until it gets a technology that can replace it altogether. If thats a contradiction between views, I think it’s a bit too small to count.

]]> By: Education Futures » Three Singularities, three conversations http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6865 Education Futures » Three Singularities, three conversations Mon, 01 Oct 2007 06:04:55 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6865 [...] Yudkowsky, on the SIAI blog, posted his observations of the emergence of three “logically distinct” schools of thought related to the [...] […] Yudkowsky, on the SIAI blog, posted his observations of the emergence of three “logically distinct” schools of thought related to the […]

]]> By: Joshua Fox http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6872 Joshua Fox Mon, 01 Oct 2007 08:14:29 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6872 Great clarification of these concepts. By the way, the word is "Apocalypticism" rather than "Apocalyptism." Not a big deal, but considering the common mangling of "Singularitarianism," a similar mouthful, it's best to get the term right. Great clarification of these concepts.

By the way, the word is “Apocalypticism” rather than “Apocalyptism.” Not a big deal, but considering the common mangling of “Singularitarianism,” a similar mouthful, it’s best to get the term right.

]]> By: Joe Hunkins http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6887 Joe Hunkins Mon, 01 Oct 2007 15:32:36 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6887 A nice post, though I'm not clear on why these are incompatible ideas. I'm guessing Kurzweil would agree that an intelligence explosion is likely, though he seems to think it'll take some years of conscious computing before that happens (you don't?), and most would agree that the future is *very* difficult to predict even without a singularity, so the event horizon is also a reasonable assertion in the other two scenarios. A nice post, though I’m not clear on why these are incompatible ideas. I’m guessing Kurzweil would agree that an intelligence explosion is likely, though he seems to think it’ll take some years of conscious computing before that happens (you don’t?), and most would agree that the future is *very* difficult to predict even without a singularity, so the event horizon is also a reasonable assertion in the other two scenarios.

]]> By: Kaj Sotala http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6888 Kaj Sotala Mon, 01 Oct 2007 15:55:35 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6888 Accelerating Change is a bit different from the two others, but I'm not sure why there's a need to differentiate between Event Horizon and Intelligence Explosion. Intelligence Explosion, at least, seems to lead to Event Horizon (though not necessarily vice versa - but then, I don't think I've heard of an Event Horizon formulation that wouldn't have included Intelligence Explosion at least implictly). The core claim of Accelerating Change supports the two others in the "why should we belive this should happen soon", though its strong claim is more dubious. Accelerating Change is a bit different from the two others, but I’m not sure why there’s a need to differentiate between Event Horizon and Intelligence Explosion. Intelligence Explosion, at least, seems to lead to Event Horizon (though not necessarily vice versa - but then, I don’t think I’ve heard of an Event Horizon formulation that wouldn’t have included Intelligence Explosion at least implictly).

The core claim of Accelerating Change supports the two others in the “why should we belive this should happen soon”, though its strong claim is more dubious.

]]> By: Roko http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6910 Roko Mon, 01 Oct 2007 20:34:51 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6910 Kaj said: "Accelerating Change is a bit different from the two others, but I’m not sure why there’s a need to differentiate between Event Horizon and Intelligence Explosion." I'll second that, Kaj. Kaj said: “Accelerating Change is a bit different from the two others, but I’m not sure why there’s a need to differentiate between Event Horizon and Intelligence Explosion.”

I’ll second that, Kaj.

]]> By: Jeffrey Herrlich http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6911 Jeffrey Herrlich Mon, 01 Oct 2007 21:16:40 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6911 I'm an Intelligence Explosioner too. I don't favor the Accelerating Change much because it's a speculative extrapolation of *human* ability. Whereas I believe that the first Strong AI will be of a huge qualitative difference. And I don't favor the Event Horizon, in the sense that I believe that the "objectives" of the future *can* be predicted and guided - hence the Friendly AI thesis. It's only the means to the ends that *perhaps* can't be reasonably predicted by us humans. But fortunately a Friendly AI will be a super-expert at strategizing ways to effectively avoid violating its stable goals. I’m an Intelligence Explosioner too. I don’t favor the Accelerating Change much because it’s a speculative extrapolation of *human* ability. Whereas I believe that the first Strong AI will be of a huge qualitative difference. And I don’t favor the Event Horizon, in the sense that I believe that the “objectives” of the future *can* be predicted and guided - hence the Friendly AI thesis. It’s only the means to the ends that *perhaps* can’t be reasonably predicted by us humans. But fortunately a Friendly AI will be a super-expert at strategizing ways to effectively avoid violating its stable goals.

]]> By: Robin Hanson http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6913 Robin Hanson Mon, 01 Oct 2007 21:49:14 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6913 OK, for an analogy, we driving down a mountain and: 1) we can see that the road ahead steadily gets steeper, or 2) the road ahead is straight, but then passes into clouds, or 3) we clearly see both a cliff ahead, and a vertical drop below 4) we are just warned "beware of something weird ahead" To which I'd add: 5) we see the road ahead suddenly turns and gets much steeper. OK, for an analogy, we driving down a mountain and:
1) we can see that the road ahead steadily gets steeper, or
2) the road ahead is straight, but then passes into clouds, or
3) we clearly see both a cliff ahead, and a vertical drop below
4) we are just warned “beware of something weird ahead”
To which I’d add:
5) we see the road ahead suddenly turns and gets much steeper.

]]> By: TONY COHEN http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6924 TONY COHEN Tue, 02 Oct 2007 03:02:18 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6924 20 years ago, if you were a 'naked' Singularity, you'd be at a Black Hole"s Event Horizon about to be sucked in. 20 years ago, if you were a ‘naked’ Singularity,

you’d be at a Black Hole”s Event Horizon about to

be sucked in.

]]> By: Tom McCabe http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6986 Tom McCabe Tue, 02 Oct 2007 22:38:08 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-6986 "This will mean continuing on Moore’s path at least until it gets a technology that can replace it altogether." What is the probability that a superintelligence would develop chips at *exactly* the same speed that humans do? Not even twice as slow, or twice as fast? Consider that the difference in raw FLOPS is going to be several orders of magnitude and up. “This will mean continuing on Moore’s path at least until it gets a technology that can replace it altogether.”

What is the probability that a superintelligence would develop chips at *exactly* the same speed that humans do? Not even twice as slow, or twice as fast? Consider that the difference in raw FLOPS is going to be several orders of magnitude and up.

]]> By: Zubon http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-8221 Zubon Mon, 15 Oct 2007 14:40:56 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-8221 Don't the three lead to one another? Accelerating Change is the source of the Intelligence Explosion, as the curve becomes increasingly steep. This leads to the Event Horizon, since we cannot predict beyond an Intelligence Explosion. Intelligence Explosion may wear out the "smooth curve" part of Accelerating Change, but I don't think Mr. Kurzweil is that strongly committed to it. If nothing else, we are usually slightly off the curve, and "slightly off" becomes "way off" pretty quickly when we increase at x^y^z. And unless you believe that you can predict what a superinteligence will do, Eliezer, Intelligence Explosion implies an Event Horizon. Don’t the three lead to one another? Accelerating Change is the source of the Intelligence Explosion, as the curve becomes increasingly steep. This leads to the Event Horizon, since we cannot predict beyond an Intelligence Explosion.

Intelligence Explosion may wear out the “smooth curve” part of Accelerating Change, but I don’t think Mr. Kurzweil is that strongly committed to it. If nothing else, we are usually slightly off the curve, and “slightly off” becomes “way off” pretty quickly when we increase at x^y^z. And unless you believe that you can predict what a superinteligence will do, Eliezer, Intelligence Explosion implies an Event Horizon.

]]> By: Life, the Universe, and Everything » Objections to SIAI/AGI/FAI http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-22949 Life, the Universe, and Everything » Objections to SIAI/AGI/FAI Sat, 29 Dec 2007 15:08:17 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-22949 [...] There are so many different meanings attached that the term Singularity has ceased to be useful. (Anissimov, Eliezer [...] […] There are so many different meanings attached that the term Singularity has ceased to be useful. (Anissimov, Eliezer […]

]]> By: Accelerating Future » IEEE Spectrum Special Issue on the Singularity http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-54776 Accelerating Future » IEEE Spectrum Special Issue on the Singularity Tue, 03 Jun 2008 07:15:29 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-54776 [...] robots be conscious? 2. Is technological change accelerating? (Core claim of Accelerating Change school.) 3. Does technological change follow smooth exponential curves? (Strong claim of AC school, by [...] […] robots be conscious? 2. Is technological change accelerating? (Core claim of Accelerating Change school.) 3. Does technological change follow smooth exponential curves? (Strong claim of AC school, by […]

]]> By: Russell123 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-55982 Russell123 Sat, 07 Jun 2008 15:02:16 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-55982 Interesting article, but its unclear which one leads to robot sword fights in outer space. http://www.youtube.com/watch?v=DDrbbe7lgyE&feature=related My guess would be apocalypticism. Interesting article, but its unclear which one leads to robot sword fights in outer space.

http://www.youtube.com/watch?v=DDrbbe7lgyE&feature=related

My guess would be apocalypticism.

]]> By: The Singularity Institute Blog : Blog Archive : IEEE Spectrum special report on the Singularity http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-56675 The Singularity Institute Blog : Blog Archive : IEEE Spectrum special report on the Singularity Tue, 10 Jun 2008 09:36:50 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-56675 [...] Russell123 on Three Major Singularity Schools [...] […] Russell123 on Three Major Singularity Schools […]

]]> By: Greg http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-57596 Greg Sat, 14 Jun 2008 23:37:00 +0000 http://www.singinst.org/blog/2007/09/30/three-major-singularity-schools/#comment-57596 I would like to favour the Exploding Intelligence theory for future technological change, with a mixture of both the core claim and the strong claim, If human intelligence could match pace? I don't think it could just by been augmented, if there were other pure super intelligences computer based. For human intelligence to keep pace it would have to go like Kurzweil thinks where humans upload there minds to a computer or android but would the new person recognize themselves as the same person as their former selves. If the former human version still was alive after transfering their mind to an android, wouldn't it be like identical twins. While knowning that eachother are very similar, they both recognize eachother as different persons. Not one and the same, but if a human uploaded their minds to an android and then the instant it was activated the human was killed or at least put into some kind of suspended animation, then the android might be able to recognize themself as the former human. Another way might be to try and keep the knowledge of one version of the person from the other, but how could the original human forget that they had just uploaded. The knowledge of uploading their minds along with all other knowledge would be transfered as well to the android. Unless this information could be deleted from both. It might be possible for the two versions to exist at the same time until the human version died naturally then of course the android version would have evolved a lot different and faster than the human version.and would have evolved into soneone entirely different but still having knowledge from its human past. If humans at that time knew that the uploaded version of themselves became someone entirely different and their own conscious was not transfered then they might not bother to transfer. This type of question will need to be addressed, either starting now or some time in the near future when more knowledge is available. I would like to favour the Exploding Intelligence theory for future technological change, with a mixture of both the core claim and the strong claim, If human intelligence could match pace? I don’t think it could just by been augmented, if there were other pure super intelligences computer based. For human intelligence to keep pace it would have to go like Kurzweil thinks where humans upload there minds to a computer or android but would the new person recognize themselves as the same person as their former selves. If the former human version still was alive after transfering their mind to an android, wouldn’t it be like identical twins. While knowning that eachother are very similar, they both recognize eachother as different persons. Not one and the same, but if a human uploaded their minds to an android and then the instant it was activated the human was killed or at least put into some kind of suspended animation, then the android might be able to recognize themself as the former human.
Another way might be to try and keep the knowledge of one version of the person from the other, but how could the original human forget that they had just uploaded. The knowledge of uploading their minds along with all other knowledge would be transfered as well to the android. Unless this information could be deleted from both. It might be possible for the two versions to exist at the same time until the human version died naturally then of course the android version would have evolved a lot different and faster than the human version.and would have evolved into soneone entirely different but still having knowledge from its human past.
If humans at that time knew that the uploaded version of themselves became someone entirely different and their own conscious was not transfered then they might not bother to transfer.
This type of question will need to be addressed, either starting now or some time in the near future when more knowledge is available.

]]>