Research Grants

Academic Paper Grant


Digital Intelligences and the Evolution of Superorganisms


Research summary:

Philosophical thought experiments such as Derek Parfit's Teletransporter, involving the construction of high-fidelity imitations of human beings, are used to test concepts of personal identity: would you be willing to sacrifice your life to save that of such a copy and give it a hundred dollars, even after copy and original had undergone different experiences? For digital minds, whether whole brain emulations or de novo Artificial General Intelligence (AGI), high-fidelity copying could be an everyday experience, and preferences regarding such cases could have enormous practical implications. This paper will outline some reasons to think that digital intelligences would eventually develop preferences such that related lineages would share self-sacrificing commitment to some common value(s), acting as collective super-organisms, and explores implications for estimating and reducing existential risk.


Planned contents include:

  • A taxonomy of preferences that software intelligences might have regarding themselves, related intelligences, and impartial aims.
  • Analysis of the pragmatic effects of these various preferences on the capacities and desire-attainment for digital intelligences who hold them, especially the advantages of self-sacrifice.
  • Exploration of pressures on the initial design of AGI or selection of brain emulation candidates.
  • Discussion of Darwinian natural selection on digital minds competing in a competitive Malthusian economic environment.
  • Reasons for self-modifying digital intelligences to alter their descendants to behave as a superorganism.
  • Treatment of military and political implications of superorganisms, including enhanced capacities and incentives to form a global regulatory order, or singleton.

Prior related work:

 

Venues for presentation or publication:

The Minds and Machines special issue on "transhumanism, cognitive enhancement, and AI" (submission deadline January 15).


Total budget:  $7,200
How research costs are estimated:
  • Person-months for research and writing: 3 (our standard estimate of the time required for journal articles[1]).
  • Dollars required to support one skilled full time researcher-month[2]: $2,400

[1] Our base estimate is 1.25 person-months per conference paper,and 3 per journal article, for an experienced full-time researcher. This estimate takes the planning fallacy, and the importance of an outside view in avoiding that fallacy, into account. While typical rates of article production by professors are extremely low, the distribution is strongly skewed towards research-oriented universities and departments, and informal surveys of researchers working on existential risks give data consistent with this estimate for full-time work required per paper. Visiting Fellows vary in their experience levels, so that mean productivity is expected to be lower, but a team mix can be selected to account for this.

[2] This billing rate reflects an estimate of financial outlays for SIAI to create the equivalent of one full-time skilled researcher-month, including stipend or hosting expenses, workspace, and administrative or management time, and other supporting expenses. Actual person-months may be greater or lower depending on the labor mix for a particular project, with shortfalls made up from general funds. This rate is not reflective of the money researchers could earn in the competitive labor market. Think of this as a matched donation. You donate the living expenses; our researchers donate the surplus value of their labor.


Target dates for: 

Journal submission: Jan 15, 2010.

 

Follow-up steps (Brainstorming, and drafting proposals for, any follow-up publications. Are there related research papers that should be considered?) : Jan 30, 2010.

 

How this paper will help reduce existential risk:

Research benefits (What ideas will the paper explore?  How will that knowledge help reduce existential risk?): 

  • This analysis is directly focused on better estimating the potential for digital minds of certain types to increase or reduce existential risk. 


Influence benefits(What target audience will the paper impact, how?  How will that impact help with existential risk?):

  • The paper will draw on well established methods of evolutionary biology and decision theory to examine some dynamics of digital intelligences with positive and negative implications for existential risk, possibly drawing greater attention from those interested in evolutionary biology and memetics to the area; this could pave the way for future collaboration.

 

Human capital benefits, or network benefits (Will writing this paper help new Visiting Fellows become familiar with key research domains?  Will it help create relationships with outside co-authors?  Will it give folks interested in existential risk entry into new communities where valuable contacts may be found?): 

  • This paper provides an opportunity for closer collaboration with the Future of Humanity Institute.
  • Once completed, the paper may enable networking with academics with related interests in the evolution of values and memes.


Credit card transactions are securely processed through PayPal.

PayPal – Grant Donation

(200 characters max)