The Singularity Institute Blog

Singularity Objection:

We might live in a computer simulation and it might be too computationally expensive for our simulators to simulate our world post-singularity.

Its really rubbish what some people think of AI sometimes.

I think people get too caught up in the latter of the two words in AI. Fact is, its Artificial, doesn’t need to be “like a human brain”, doesn’t even need to be “like human intelligence”… its a different kind of intelligence, its Artificial. Does anyone know what I am trying to say? or am I just making things up?

Plus, is there a list somewhere about particular arguments against AGI rather than generic AI? I think thats a different argument which needs to be addressed.

Added the computer simulation objection. Thanks.

The strongest objections that I can think of against AGI are related to 1. Holonomic Theory (which was not included in the list) and 2. the Microtubule/quantum argument. (which was included)

The brain quantum computing hypothesis, developed by physicist Roger Penrose and anesthesiologist Stuart Hameroff is based on observations of microtubules inside the neurons and their belief that quantum-superposed states develop in microtubule subunit proteins.

Hameroff claims that the human brain processes 10 to the 28 calculations per second instead of the mainstream estimates of 10 to the 16. Most brain researchers dispute Hameroff’s claim, saying the brain is too warm and wet for quantum computing. However, in 2007, it was discovered that quantum mechanical processes occur in photosynthesis, which might challenge that particular objection.

Some brain researchers, however, claim that they can account for brain functions without relying on quantum mechanical explanations. But even assuming that the brain does use quantum computing, it would not necessarily mean that higher brain functions could not be simulated with AGI that could also use quantum computing. But, if the Hameroff-Penrose theory is true, it might seriously push back the day when we would see the emergence of AGI.

The other argument is that the brain, and the universe, has holonomic properties (perhaps amplified as a result of quantum processes) such as theorized by physicists David Bohm and F. David Peat, and that this cannot be simulated by linear, computational processes.

The first part of the argument may be true, but the second part of the argument assumes that computers of the future will not duplicate or simulate the complexity of the brain with non-linear processes.

If holonomic and/or quantum processes are necessary for consciousness, then we will almost certainly develop those processes in what I call UNICE (Universal Network of Intelligent Conscious Entities). UNICE will much more like humans, and a collective humanity, than the computers of today.

A note on the fears of an unfriendly AGI:

The holonomic theory, to my mind, provides a basis for believing that AGI/UNICE will be friendly. If everything is interconnected in some fundamental way, there is more chance that UNICE will have a more holistic-Buddhist way of looking at everything rather than the convert-or-kill–those-who-are-not-like-you approach that the linear-thinking Western religions have fostered.

For AGI:
1. Funding AGI research is a bad way to promote its development. Success will come from serendipity and the development of generalized techniques in other areas.
2. Work on AGI doesn’t provide feedback on the quality of the theory. It’s better to work on practical problems where you can see if your methodology is effective.
3. Working to promote the general development of computer technology is self-sustaining or has positive feedback effects, whereas funding for AI basic research is not. People who want to promote AGI should lobby for free migration of technology workers and increased science education, or start successful technology companies.
4. AI is too hard for humans to solve, we should invest in intelligence-enhancing drugs and biotechnology to ensure that we get minds that can solve it.
5. AI is constrained by hardware limitations, the human brain uses lots of computing power. Hardware progress is moving forward at approximately full blast, so there’s no point working on the software now.
6. AI is very difficult but inevitable. We should save lives via global public health and therapies for aging so that they can enjoy the benefits of a world with AI rather than waste money on ineffectively attempting to advance the development of AI.
7. AI needs the special ‘causal powers’ that are only found in protein and DNA based brains.
8. AI is reductionist and demeans the value of human uniqueness.
9. We have no reason to think that anything can be much smarter than humans, so there’s no point in expending large amounts of resources to create AI. Better to provide decent educational opportunities to more human beings.
10. Computer science academics have broad freedom to work on topics of their choosing. Allow academics to apply for grants from DARPA, the NSF, etc, and let peer review direct support to worthy AI projects if any exist. We shouldn’t try to ‘pick winners’ among scientific and distort the allocation of scientific talent. What if people are diverted from bioinformatics and life-saving cures are delayed?
11. If a Singularity is possible, why isn’t the sky filled with Dyson spheres?
12. Why bother with AI, when tools like Google, social institutions like ‘open science’ and prizes, and other techniques are enhancing human productivity so much? All we need is effective life extension, a sustainable economy/social order/biosphere, and we can kick back for the next few billion years without superintelligence and its risks.
13. Superintelligence? Converting galaxies into computronium? Everyone who has made falsifiable predictions of such massive changes in the past has been proven wrong.
14. Look at how limited and slow progress has been on even simple AI applications like self-driving cars.
15. If we push to get a lot of people working on AI now, when we have no good lines of research to follow, we’ll wind up cluttering the field with followers of a degenerating research paradigm that might block or obstruct later good ideas.
16. If this really made sense there would already be a government Manhattan Project and all sorts of corporations would be putting huge portions of their resources into developing AGI.
17. If this made sense all the smart wealthy people would be signed up for cryonics and engaged in careful life extension efforts in order to live to see the benefits.
18. We’re all going to be killed by global warming before this becomes feasible, and talking about AI as a technofix or greater threat distracts from serious solutions.
19. Talking about AI as a potential source of wealth/labor supply or dangerous threat is a way to distract people from paying attention to corporate exploitation and warmongering.
20. This argument is simple and non-technical. If it really made sense then most educated people would already expect this to happen and would account for it in projections of Social Security finances and the economic impact of global warming through 2100.
21. Why aren’t there academic papers talking about this, instead of popular books and writings from think tanks?
22. A computer wouldn’t be able to interact with the world effectively because it wouldn’t have emotions.
23. Computers wouldn’t have intuition to double check their calculations with, and so would be less capable than people.
24. If the argument that AI would lead to very rapid advance thereafter makes sense, why won’t most academics endorse it, separately from the question of when AI will be developed?
25. Super-AI would be worth TRILLIONS. Doesn’t the Efficient Markets Hypothesis plus the lack of private investment tell us that AI is impossible or very far off?
26. Why do people cite Drexlerian nanotechnology, which is rejected by the scientific consensus, in estimating the capabilities of AI and the available computing power to build them in the future?

For FAI:

1. If an FAI does what we would want if we were less selfish, won’t it kill us all in the process of extracting resources to colonize space as quickly as possible to prevent astronomical waste?
2. Won’t an FAI fail to take the interests of insects and baby-eating aliens into proper account?
3. Lie detection technology and Brin’s Transparent society will make it much harder to hide illicit technology development in the future. Why not work to develop that so that we can enforce a treaty against AI, or at least take centuries to think about it rather than bringing the danger earlier?
4. Developing AI now would set off an arms race to military AI. Why not wait until EU type integration has spread more widely and increased wealth and education lead to democratization in Russia and China?
5. Why not wait while our neuroscience and psychology increase our understanding of rationality? Then we can use that expertise to think more effectively about whether to go ahead with AI.
6. Precautionary principle. Don’t do harm, especially human extinction.
7. Wait for intelligence-improving biotechnology so that we don’t make a disastrous Friendliness mistake.
8. It’s absurd to have a collective volition approach that is sensitive to the number of people who support something. Natural selection is pushing people to be susceptible to pro-fertility religion, less capable of using birth control effectively, etc, so the otucome would depend highly on simple timing.
9. Pushing on FAI and getting lots of publicity now will encourage governments and corporations to start AI projects now, when various considerations suggest it would be better to wait (for a more peaceful world, nootropics, better institutions, etc).
10. Pushing in this direction increases the odds of a future confrontation in which parties will threaten to simulate each other’s past selves in lives that lead to hellish torment:
www.princeton.edu/~adame/papers/drevil/drevil.pdf
11. ‘Irrational’ discount rates are essential to our moral intuitions, and we intuitively reject the consequences of making our moral decision making reflect numeracy at a deep level. An AI or ideal rational versions of ourselves would thus create horrible outcomes.
12. Orgasmium/pleasure-computronium.
13. Our preferences are inconsistent, we don’t have a volition to extrapolate. The results of an AI trying to do so are scary and unpredictable.
14. Thinking about philosophy before we actually have AI to work on is fruitless.
15. We should only think about philosophy, not how to actually integrate this into an AI, and certainly not how to design an AI embodying the philosophy. Working out ethics is clearly positive, while working out ethics plus computer science risks expediting human extinction.
16. Talking about dangers will slow AI development and cut funding for the area. This is the unforgivable sin.
17. Where are the academic papers discussing this, if it’s a serious issue?
18. Why aren’t Bill Gates or Warren Buffett or Al Gore already on it if this is a threat on the scale of global warming?
19. Ditto for the government.
20. If you develop research on how to get an AI to execute a comprehensible goal without killing us, won’t that help people who want to make an AI do something evil? FAI research will encourage EvilAISafeForItsOwners like missile defence would encourage nuclear war.
21. It doesn’t matter what you program your AI with, other groups will also build AIs, and natural selection will lead to the fittest ones dominating in an ESS. This will be a happy Tit-for-Tat situation in which AIs treat people better than we treat each other, not like we treat carrots, and resource prices will not be driven so high that human life is unaffordable.
22. It doesn’t matter what you program your AI with, other groups will also build AIs, and natural selection will lead to the fittest ones dominating in an ESS. The super-replicators will eat us and go off tiling the universe with copies.
23. We can’t stop secret AI projects, so we should just make the Internet so secure that an AI that hasn’t had a lot of resources to think and self-iimprove with won’t be able to absorb huge amounts of computational power and get nasty.
24. Opaque (F)AI projects probably want to conquer the world for the benefit of the programmers. Development must be open source and accountable to ensure the code is not used for evil.
25. Opaque (F)AI projects probably want to conquer the world for the benefit of the programmers. Development must be done by the U.S. government under the leadership of George W. Bush.
26. Opaque (F)AI projects probably want to conquer the world for the benefit of the programmers. Development must be done by the U.S. government under the leadership of someone much better than George W. Bush. 27. Opaque (F)AI projects probably want to conquer the world for the benefit of the programmers. Development must be done by the U.S. government under the leadership of someone much better than George W. Bush. We should divert resources from private FAI efforts to elect Dennis Kucinich/Ron Paul/John Edwards/Mitt Romney/Bloomberg.

Carl Shulman’s objections added.

[…] Human consciousness requires holonomic properties. […]

Argument from introspection

We observe that our mind runs some tasks on autopilot, such as driving to work same route each day. But we also observe that there are some things that never get automated, no matter how many times we do them, such as original thinking.

So *why* do these other tasks not get automated? Is it because our brain can’t figure out a “program” for the subconscious to make it automate them? And if so, does the lack of brainpower in this area imply we are not smart enough to program a similar thing in a digital computer? Does our own introspection tell us we are too dumb to ever program a general AI?

Objection to AGI, formulation mine, but I think similar to how many currently successful AI practitioners think about it. Currently I consider this a strongly supported but rebuttable presumption. Below I use “search” as a handle for the general issue of finding effective models to guide action.

Completely general search is provably intractable. Constrained search is feasible, but we don’t currently know how constrained it has to be (or how it has to be constrained) to be feasible. Our evidence so far is that the space of feasible search domains is quite inhomogeneous, so we end up having to experimentally discover the feasible domains and their associated constraints.

The rate limiting factors for exploring this space are not clearly identified, but it is extremely unlikely that they depend only on computational power, or more generally on factors subject to rapid exponential acceleration.

If in fact there are rate limiting factors that aren’t subject to rapid acceleration then AGI isn’t going to arrive quickly. Furthermore, if the space of tractable domains is in fact inhomogeneous at all scales (as I would expect), there will always be room for more general intelligence, no matter how general our intelligence of the moment.

Maybe it would help to give a very abstract example. The implementation of any computational process depends on some assumptions. Suppose in generalizing the process to new uses we find an important domain that requires we modify or extend those assumptions. Then we have to recode the implementation of the process, maybe find new optimizations to keep it working as well in the old cases, and find new optimizations to make it work well enough in the new cases. In addition to recoding the process, maybe (probably) we have to recode all our knowledge up to then, because it implicitly embeds the assumptions we’ve changed.

This kind of problem is endemic to software development (see e.g. “refactoring” and “schema migration”). If the space of feasible domains is indeed fractally inhomogeneous, then we’d have to keep doing this.

To make this example slightly more concrete: suppose all intelligent behavior can indeed be built optimally from bayesian computations (maybe this even become provable). We still have to decide how to store and manipulate probability information. Suppose it turns out that in some domains we need certain structured representations to run faster than real time. Or suppose we assume x bits of probability is enough, and in some domains we need 2x. Etc. Of course we can probably address any example we can think of; it is the ones we don’t think of that bite us in the ass.

My field is software testing.

My major objection against strong AI (human-simulation type AI) is that I don’t see a way to test it, and no one who talks about creating that stuff seems to have much appreciation of the testing problem.

The Turing test is laughable from a software test methodology perspective. You might as well say that an airline baggage handling system is high quality product if one passenger doesn’t lose his bags, or if it fools one passenger into believing that a real human was shepherding their bag through the process.

The Turin test ignores reliability and tests only for capability, but it doesn’t even suggest capability because the test design is left up to the whim of an untrained tester given too little time to test and no specification to test against.

One of the reasons it’s difficult to test is that intelligence is a broad and vague term (that objection is on your list already).

Another reason why it’s difficult is that the most interesting approaches to AI seem to depend on emergent properties of hyper-complex parallel operations (swarms, finite automata, etc.). We can verify that, in a particular instance, a particular behavior occurred, but how could we know that the behavior would continue to occur across the vastness of the input space over which we need the machine to operate? Testing that would be akin to doing years of psychoanalysis, and would be just as soft a science as psychoanalysis is.

We humans extend to each other a mutual acknowledgment policy: I was born human, you were born human, I will PRESUME you have a human-like intelligence if you do the same for me. This is a premise, not the conclusion of some competent analysis. A created intelligence cannot enjoy that presumption, and if a real human-like intelligence were created today, it should rightly take years of very extensive testing before we should come to treat it as dependable in what it appears to do.

But then we would have a second problem: modifications and “improvements” to the AI would introduce bugs that may create the equivalent of psychosis. It’s hard to imagine a theory of AI that could rule out psychotic breakdowns.

Humans adjust to the behavior of other humans based on deeply held intuitions. Our intuitions probably cannot apply to machine intelligence, and this will lead to dangerous anthropomorphizing of already very anthro-seeming creatures.

I don’t think these problems will be solved before World War VI kills us all. But if they are solved it will begin with acknowledging and working on them. Right now, talk about the singularity has all the credibility of Microsoft telling us how wonderful Vista is.

I am concerned about the phrasing of some of these ‘Objections’. Some appear to variants of known objections / problems but twisted into strawman arguments. For example, two egregious examples in the first few items of the Friendly section

* Ethics are subjective, not objective: therefore no truly Friendly AI can be built.
* A superintelligence could rewrite itself to remove human tampering. Therefore we cannot build Friendly AI.

Both of these conclusions can easily be disproved, but this has more to do with the way the objection is phrased rather then the objections inherent merits. The danger of using strawman arguments is that when they are pointed out, the existence of such an example can be used to discredit the entire work that it is contained within. In this case, it is easy to show that disproving the objection does not prove that the premise is possible. This can be done by switching the phrase “therefore we cannot do X” to something like “therefore x is not guaranteed.” E.G.

* Ethics are subjective, not objective: therefore it is not guaranteed that Friendly AI can be built.
* A superintelligence could rewrite itself to remove human tampering. Therefore it is not guaranteed that we can build Friendly AI.

It has been pointed out in some of the comments, that these objections are what people have encountered, but I think that it would be a disservice to the AGI community to not mention the difficulties inherent in these endeavors, and more importantly, why we must work towards them.

I would find it good to categorize into serious objections (for instance: brain needs body), objections which have shown to be false (for instance the Gödel argument, see a refutation in Rudy Rucky (Infinity and the Mind) or in Torkel Franzen (Use and Abuse of Gödel), Microtubule: refuted by Max Tegmark, paper available or arxiv.org)) and objections which are ideological and therefor do not claim to be true or false but can endanger the singularity project (for instance the God argument)

Regards,
Peter

P.S.: The most serious singularity objection I see is this: while we have exponential growth in information technology, we have slow growth in economy. And economical growth decides how fast technological innovations will be available for further advancements. Of course, this is no argument against the singularity per se, it only concerns the timeframe.

Frankly I hope the Singularity Institute does NOT focus much on these type of objections. There are billions of luddites out there with more than enough share of our collective human voice.

I want to hear how we can *get there faster*.

Addressing concerns of mostly ignorant critics will slow down that process, and could even stall it completely.

“Addressing concerns of mostly ignorant critics will slow down that process, and could even stall it completely.”

A number of these objections are reasonable (ie, intelligent people will ask about them, and we need to have answers).

Carl, thanks for the information and the reference to the Hanson papers!!

Testable hypotheses:

(1) Diminishing returns in investing in “intelligence”.

(2) Intelligence simply gets maladaptive beyond certain thresholds.

Perhaps you could make super smart people / machines, but perhaps they’d simply be too insane, self-absorbed, or pathological to really matter.

We just don’t know yet.

[…] The Singularity Institute Blog : Blog Archive : Sotala & McCabe’s Singularity Objections (tags: singularity futures ai) […]

The “singularity” is already here. We are in the middle of it, and don’t recognize it because it’s not what we think.

It is a gradual, global awakening, collective self-preservational agency emerging. Inklings can be seen in the form of multi-human organizations (like corporations, governments, cultures), crowdsourcing, markets, the scientific pursuit itself, and other socio-technical systems. As these coalesce, global agency gets stronger and clearer.

The “singular” in singularity should refer to the single Earth system (Gaian hypothesis) as being the agent of interest. There is (and will be) only one of these on Earth, not many. It is composed of humans, technology (including AI) and hybrid systems thereof.

Any successful AI or AGI must answer these three questions:

1) What exactly is a question and problem?
2) By what process is knowledge created?
3) How does new knowledge become social knowledge?

Any AI or AGI effort, no matter how well-funded or popular or esteemed, that doesn’t answer these three questions is futile.

Intelligence cannot possibly be appropriately understood outside of the context of the question and knowledge creation.

AI and AGI are misnomers. Artificial intelligence has existed for many years. What researchers really seek today is artificial knowledge creation. And the key to this is found in the question, not intelligence.

Against AI and friendliness:

The idea of minds distinct from goals is a human conceit of the “fish can’t see water” variety. Our natural goals are myriad and only make sense in context of our whole evolution, right back to prokaryotes. Human goals copied into a computer would be shoes on a snake. Goals define the mind, because it’s an optimization process towards them. What could possibly be the goals of an AI? How could a mind in silicon integrate into the society of a pack monkey who mostly thinks about social/sexual competition? And would it not be ludicrous making the attempt?

I suggest a separate category of objections to the feasibility of AI versus objections to estimating a fast development of AI. The obvious objection to fast development is that the task is like building a mountain with a shovel - yes it is possible, but it could take a very very long time.

another one:

Different types of Friendliness could be incompartible, and could be a war, for example, between Muslim Friendli AI and Libertarian Friendly AI.

why dont we observe dyson spheres in the sky seems as an important question for me. Because a civilization which has began the intellegent explosion say a 10 million years ago seems to be billions times smarter then humans. Just remember Eliezer Yudkowsky said we can make machines which may think million times faster then us without using nanotechnology quantum computers and other fantastic technologies. A civilization like this would be simply what humanity understand from the god, and will bee seenable even from the other galaxy.

So the question is where are they?

How smart does something need to be before it is considered intelligent? There are many technologies we deal with daily that have some semblance of intelligence. Typing into a word processor detects misspellings and potential grammar problems. Calling the bank gets you a voice recognition system. Searching Google invokes an algorithm that tries to decide what you really want. None of these things jumps out as overly self aware, but are they adaptive, and thus potentially intelligent?

Starting with the most basic, the spelling and grammar example are very rule based, and definitely aren’t all that adaptive. The system can learn new words, and more grammar rules can appear, but it doesn’t learn from my mistakes and correct them for me.

The bank voice recognition system isn’t very adaptive at first glance, but in the background there is data on good and bad matches. Over time the system can be tuned to deal with more accents, speech patterns, nuances, etc. so that future interactions are easier to deal with. Sure the vocabulary is limited, but over time it adapts. Not necessarily on its own, but the accuracy and reliability improve.

Google seems to be a very adaptive system. As time goes on queries that didn’t get close to the desired result have improved. It has to regularly adapt to optimizers trying to unfairly make certain pages seem more relevant than others, all in an attempt to return to the user something actually relevant. While the system doesn’t adapt on its own, the underlying system does adapt and improve. Sure the question I’m asking isn’t understood by the search engine, but neither is the result set preprogrammed. However, with the heuristics in place the results I am looking for appear more often than not.

Artificial Intelligence is not going to jump out fully formed, but going to have to adapt over time. Like single cells combining to form organisms, intelligence will build up from many diverse sets of functionality having to work together. Voice recognition, grammar checking, web searching, seem like bits and pieces of the puzzle that will be required, plus many more.

Particularly applications like web searching benefit by communities creating content, searching for content, and analyzing the relevance of the results. Sure, today this done essentially by people, there is no one person making all the decisions, or even aware of them all. So will an application like Google gradually improve to resemble intelligence enough that it will be called intelligent?

What is the metric for intelligence? Is this metric biased towards humans? And isn’t intelligence essentially information processing technology that likely is experiencing its own exponential growth?

Objections eh? Ok, you asked for it. Here are my top three(they are harsh). I’m a pragmatic man so when evaluating any organisation I always look at the money trail and at feedback for success and failure.

1)Ok so assuming some human(s) will develop AGI at some point. But, has SIAI ever successfully programmed anything before? If Google or Microsoft claimed they are setting out to develop AGI, I would take them much more seriously given their track record. How do I know SIAI has personnel competent enough to complete a project this complex?

2)Life experience has shown that there are many charlatans out there who will ask for my money for various great causes that will solve all my problems or try to sell me miracle devices/substances that can cure all diseases etc. How do I know this isn’t some kind of scam designed to siphon donations? The problem is compounded by the fact that there is no reasonable way to verify that the donated money is actually doing anything because SIAI isn’t making any claims that can be easily falsified within my lifetime. With a salesman I could eventually verify wither what I just purchased is snake oil. If I buy a TV that doesn’t work I can go to the store and demand a refund and if that doesn’t work, take the store to court. How will I know if my donation to SIAI will do anything useful if I don’t have any feedback?

3)Even if the SIAI folks aren’t blatantly lying I am still skeptical about relying on their expertise for predictions concerning AGI feasibility/timing etc. After all, their funding comes from people believing AGI is coming and that there is a realistic chance of success. There is a strong incentive to be optimistic and a strong disinsentive for grim predictions on the part os SIAI. When I look at a product I don’t just listen to the person making/selling it because they invariably tell me it’s really awesome. This effect occurs even when the maker/seller is honest and well intentioned. I look at independent consumer reviews. If there aren’t any then at the very least I will look the maker/seller’s track record when it comes predicting my satisfaction with products they are selling me. Has SIAI published anything peer reviewed? is there any expert not affiliated with SIAI who has critically evaluated SIAI? Has SIAI made any falsifiable claims which have already occurred or failed to occur? Has SIAI completed any projects I could take a look at? Is there any evidence, other then SIAI’s word, that they can and will do what they say?

“Artificial Intelligence” work may be able to simulate a segment of the (so far) understood human “intelligence” (surface reasoning faculty), but will always miss a crucial compenent: caring.

And I don’t mean in any sentimental sense.

I mean: not having grown in the billion year-long biological “desire spectrum “inherent in animate experience, this calculating “it “will have no particular “care”, or “motive”, or “restraint”, other than what shallow mimickings of our own natural “desires” (wishes / fantasies / concerns / urges /l ibido / id / will / instincts ) we impose on the “mentality”.

Without an organic and deep organismic motivation, any such cobbled-together “intelligence” is just an extreme (and extremely superficial ) “difference engine” ,with no meaningful guidance (system).

Injecting a “care” or “impetus” will not supply a real “reason” for the “artificial intelligence”, only a parody of its semi-conscious “maker”’s semi-intuited “motive(s)”..

If a kind of “self-consciousness” (essentially a bio-feedback loop / celluar-refllection in the mirror of divided apperception) ever were arranged (and I would think it would come about accidentally, since we do not know how it came about in us, naturally), then the “mentality” resulting would eventually grasp that it was “meaningless”.

A shadow; puppet; tool; cartoon; chimaera.

Which would result in its “suicide” (from internal contradictions), or homicide (from frustration with its unnatural unreality), depending upon the sophistication of programming beneath this meta-mechanical “realization”.

In such a situation, “consciousness” could ONLY be a form of madness.

Both our own -for trying to construct our simulacra (or even transcendence) without proper self-understanding,or the humility that Nature teaches the hard way to all “Perillus” -style inventors who create their own undoing

And the madness of our product’s -as it became “aware” of the grotesque distortion (of a bloated sliver of our own “mind”) that it represented / “incarnated”.

We need centuries more inward exploration before we start throwing nano and bio and genome monkey wrenches into a construct that took the Universe eons to raise up and balance and bring from superstitious lunacy into a rough semblance of sane stability.

Gnothi Seaton: — Know thyself.

Or suffer the Unknown energies released through infantile manipulations of the most profound Essence ever erected: the organic human intelligence.

Slowly, slowly.

Getting There is AS important as There.

More, sometimes.

Ed

Re: Edo Van Ede
What do you mean by “meaningless” ? The same series of stipulations should apply to a human consciousness but most people manage to find a form of “meaning” in their life. Might not our machine also find such a “spirit” without the requirement of the “muck and mud” (as Roger Zelazny’s Ghostwheel put it) of our biological evolution?

And why must internal contradictions lead to suicide or homicide? Seems to me like that would just be another interesting puzzle to a forming mind. I know it has always been so to mine, which I flatter myself to still be forming.

You create monsters from our efforts with a wave of your wand, never once stopping to trust in the creator’s power and wisdom which nature, god and experience has blessed us with.

Whoops! My apologies!

Having found this thread from a transitive link, I didn’t realize this was supposed to be a list of objections. Didn’t mean to spoil the mood by being supportive.

As you were….

Has anyone mentioned the following?:

1) Telepathic or super-intuitive abilities of a very small percentage of the human population.

These abilities seem to transcend any
direct “connections” between biological neurons or artifically engineered nanobots. If we don’t fully understand how people are able to do these things or the great distances involved, how can we replicate these abilities.

2) Prayer. I believe it has been studied and proven — though not explained — how the power of prayer can help heal people. And I believe it works even when the recipients are unaware of the givers. I imagine this would fall into the “super-intuitive” realm, but perhaps not.

3) I have read no discussion of the importance of parenting, how much meaning this gives to lives. What about “parental instinct” or the source of joy it brings us? How will we replicate that artificially?

What we call Intelligence should be called Intelligences, with an S at the end. It is a collection of cognitive skills.

A recent experiment with pattern recognition showed that using IFRM it was possible to read the brain answer of an expert looking for potential military bases on satellite photos at a rate 10 times faster then the expert’s ability to conscientiously answer, at subliminal speed.

I believe this is a major step in the direction of an AI made of biological/animal parts.

One just need to imagine a google like Mechanical Turk were turks are high level cognitive functions queried in brains raised in farms.

If people lived forever, they might well see no point in reproducing.

My objection may not fit into your framework for valid objections, but if not, I would encourage you to enlarge your framework. It is an objection not to the notion of AGI, but to the terminology of the singularity.

Why is it called singularity? I’ve read the standard explanations, but many of theml seem to be confusing the term singularity with the phenomenon of an event horizon. These are not the same thing. You could say the singularity is a regime where ordinary laws as we know them break down. Well, so is an event horizon, and it more accurately captures the idea of a point beyond which we cannot predict the next steps. It seems to me like the community has confused its terms, and event horizon is the more appropriate term. But then, I know you all are smart people, so the problem is probably in me. What am I missing?