Comments on: Should ethicists be inside or outside a profession? http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/ The Singularity Institute exists to confront the challenge of powerful AI, both the opportunity and the risk. Thu, 04 Dec 2008 05:29:12 +0000 http://wordpress.org/?v=2.1.3 By: Nato Welch http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-8964 Nato Welch Mon, 22 Oct 2007 08:23:12 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-8964 More important than whether scientists or engineers are ethicists is that ethicists be scientists and engineers that are well-informed about the subjects they examine the ethics of. But I wonder if Minsky is talking about whether scientists/engineers must //necessarily// be ethicists. It's one thing to say all scientists don't have to be ethicists; it's another the say no ethicists should be scientists. More important than whether scientists or engineers are ethicists is that ethicists be scientists and engineers that are well-informed about the subjects they examine the ethics of.

But I wonder if Minsky is talking about whether scientists/engineers must //necessarily// be ethicists. It’s one thing to say all scientists don’t have to be ethicists; it’s another the say no ethicists should be scientists.

]]> By: Roko http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-8980 Roko Mon, 22 Oct 2007 12:13:30 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-8980 "My nightmare here is people saying, “Aha! A randomly selected AI researcher is not guaranteed to be ethical!” So they turn the task over to professional “ethicists” who are guaranteed to fail" Indeed - this is a good point, I completely agree. Of course this is not likely to happen in s fast-takeoff scenario, because in such a scenario, no-one (including professional ethicists and politicians) will take any notice of AGI until its already too late to do anything about it. In a slow takeoff scenario, you could end up with technically illiterate "professional ethicists" icing a cake of AGI researchers. What would the result of such a system be? Anyone got any ideas? I somehow suspect that the ethicists' main job would be rationalizing whatever the most politically popular position was. “My nightmare here is people saying, “Aha! A randomly selected AI researcher is not guaranteed to be ethical!” So they turn the task over to professional “ethicists” who are guaranteed to fail”

Indeed - this is a good point, I completely agree.

Of course this is not likely to happen in s fast-takeoff scenario, because in such a scenario, no-one (including professional ethicists and politicians) will take any notice of AGI until its already too late to do anything about it.

In a slow takeoff scenario, you could end up with technically illiterate “professional ethicists” icing a cake of AGI researchers. What would the result of such a system be? Anyone got any ideas? I somehow suspect that the ethicists’ main job would be rationalizing whatever the most politically popular position was.

]]> By: Tom McCabe http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9005 Tom McCabe Mon, 22 Oct 2007 19:38:05 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9005 "Losing the standard view, throwing away the human conceptual language, forces you to think for yourself about ethics, rather than parroting back things that sound Deeply Wise." Perhaps we should invent a new dialect of English, specifically for avoiding cognitive biases and anthropomorphicisms. "What would the result of such a system be? Anyone got any ideas?" AGI research will get tangled up in political "ethical issues", like the current stem cell nonsense. None of the politicians and "ethicists" will realize how important AGI is, and they will seek to get it bogged down for years. Real AGI research will continue under cover of darkness, probably under a different label. “Losing the standard view, throwing away the human conceptual language, forces you to think for yourself about ethics, rather than parroting back things that sound Deeply Wise.”

Perhaps we should invent a new dialect of English, specifically for avoiding cognitive biases and anthropomorphicisms.

“What would the result of such a system be? Anyone got any ideas?”

AGI research will get tangled up in political “ethical issues”, like the current stem cell nonsense. None of the politicians and “ethicists” will realize how important AGI is, and they will seek to get it bogged down for years. Real AGI research will continue under cover of darkness, probably under a different label.

]]> By: Keith Elis http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9088 Keith Elis Tue, 23 Oct 2007 21:19:16 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9088 Eliezer, do you think it is possible for a non-professional to obtain adequate mastery of a subject matter such that his or her ethical commentary is not irrelevant (say, a patent clerk mastering physics)? If so, then this post doesn't seem to support its answer to the title question. Eliezer, do you think it is possible for a non-professional to obtain adequate mastery of a subject matter such that his or her ethical commentary is not irrelevant (say, a patent clerk mastering physics)?

If so, then this post doesn’t seem to support its answer to the title question.

]]> By: Josh Brighton http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9091 Josh Brighton Tue, 23 Oct 2007 22:08:59 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9091 How is it that someone, such as yourself, who knows nothing about "art" or painting, as one particular art, writes about art? In fact, I recall having heard you claim art was useless and a waste of time. How is it that someone, such as yourself, who knows nothing about “art” or painting, as one particular art, writes about art?

In fact, I recall having heard you claim art was useless and a waste of time.

]]> By: Peter de Blanc http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9122 Peter de Blanc Wed, 24 Oct 2007 08:41:08 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9122 <i>In fact, I recall having heard you claim art was useless and a waste of time.</i> That would really surprise me if true. Can you provide a reference? In fact, I recall having heard you claim art was useless and a waste of time.

That would really surprise me if true. Can you provide a reference?

]]> By: Roko http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9158 Roko Wed, 24 Oct 2007 16:32:03 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9158 "Real AGI research will continue under cover of darkness, probably under a different label." Yes, that does sound plausible to me. And I think that would probably the worst possible outcome! “Real AGI research will continue under cover of darkness, probably under a different label.”

Yes, that does sound plausible to me. And I think that would probably the worst possible outcome!

]]> By: Jeffrey Herrlich http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9172 Jeffrey Herrlich Wed, 24 Oct 2007 20:22:18 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9172 "Should ethicists be inside or outside a profession?" Considering that, in the future, a single researcher could ruin the entire Universe, I'd say that researchers should definitely consider the implications of what they are doing. If you'd like to label that consideration as "ethics" then go ahead. I used to spend more time thinking about what is "ethical" and what isn't. But when I realized that someone could potentially ruin the Universe forever, I decided that pragmatic thinking was more important at this point in time. “Should ethicists be inside or outside a profession?”

Considering that, in the future, a single researcher could ruin the entire Universe, I’d say that researchers should definitely consider the implications of what they are doing. If you’d like to label that consideration as “ethics” then go ahead. I used to spend more time thinking about what is “ethical” and what isn’t. But when I realized that someone could potentially ruin the Universe forever, I decided that pragmatic thinking was more important at this point in time.

]]> By: Jeffrey Herrlich http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9280 Jeffrey Herrlich Fri, 26 Oct 2007 01:26:32 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9280 I've recently become completely convinced that there is no such thing as "objective human morality". There are existence proofs of this, literally everywhere you look. So, since I've recently rejected "objective morality" I've also recently rejected "objective immorality". If it is not objective, then does immorality, in the platonic realm, have any meaning? I actually don't think so. To borrow a line from the movie "3:10 to Yuma" : "Every man is right by his own mind". I think there is great wisdom here. Before people misunderstand, I'm not saying that morality is unimportant. Just that we should be focusing on the pragmatic, not the romantic. That what matters most at this time is the *actions* of intelligent agents. Not the emotional/philosophical ruminations that lead humans to act and to claim what is ethical and what is unethical. Minds are deterministic. If people need someone to blame occasionally for the terrible injustices of the world, let them blame the mechanics of this Universe - for not having been a nicer, just place to begin with. Let's now focus on the technical problems of reshaping the Universe into a good place. The good place that we should have had from the beginning. I’ve recently become completely convinced that there is no such thing as “objective human morality”. There are existence proofs of this, literally everywhere you look. So, since I’ve recently rejected “objective morality” I’ve also recently rejected “objective immorality”. If it is not objective, then does immorality, in the platonic realm, have any meaning? I actually don’t think so. To borrow a line from the movie “3:10 to Yuma” : “Every man is right by his own mind”. I think there is great wisdom here. Before people misunderstand, I’m not saying that morality is unimportant. Just that we should be focusing on the pragmatic, not the romantic. That what matters most at this time is the *actions* of intelligent agents. Not the emotional/philosophical ruminations that lead humans to act and to claim what is ethical and what is unethical. Minds are deterministic. If people need someone to blame occasionally for the terrible injustices of the world, let them blame the mechanics of this Universe - for not having been a nicer, just place to begin with. Let’s now focus on the technical problems of reshaping the Universe into a good place. The good place that we should have had from the beginning.

]]> By: Joe Hunkins http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9324 Joe Hunkins Fri, 26 Oct 2007 08:52:40 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9324 I think this is an important early question to ask the first conscious computers, and I suspect they'll say "inside or outside, it's all the same to us". I'll be OK with that answer. Of course if they say "What is ethics?" do we need to pull the plug before they figure out how to keep us from doing that? I think this is an important early question to ask the first conscious computers, and I suspect they’ll say “inside or outside, it’s all the same to us”. I’ll be OK with that answer.

Of course if they say “What is ethics?” do we need to pull the plug before they figure out how to keep us from doing that?

]]> By: Eliezer Yudkowsky http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9523 Eliezer Yudkowsky Sun, 28 Oct 2007 01:51:16 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-9523 Robin Hanson had a nice way of putting this: We need <a href="http://www.overcomingbias.com/2007/04/expert_at_versu.html" rel="nofollow">experts at</a> AI ethics, not <a href="http://www.overcomingbias.com/2007/04/expert_at_versu.html" rel="nofollow">experts on</a> AI ethics. Robin Hanson had a nice way of putting this: We need experts at AI ethics, not experts on AI ethics.

]]> By: I want to achieve immortality by not dying « Entitled to an Opinion http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-18243 I want to achieve immortality by not dying « Entitled to an Opinion Tue, 04 Dec 2007 05:49:06 +0000 http://www.singinst.org/blog/2007/10/21/should-ethicists-be-inside-or-outside-a-profession/#comment-18243 [...] about the “wisdom of repugnance”. Eliezer Yudkowsky dumps on those same bioethicists here, but he and I apparently disagree (I am not sure exactly how we differ) on how we regard ethics in [...] […] about the “wisdom of repugnance”. Eliezer Yudkowsky dumps on those same bioethicists here, but he and I apparently disagree (I am not sure exactly how we differ) on how we regard ethics in […]

]]>