Reason Strikes Back: New Rationalisms

Course at the Melbourne School of Continental Philosophy

Originally Taught: Winter School 2024

Recordings available at https://mscp.org.au/past-courses/reason-strikes-back-new-rationalisms

In the contemporary philosophical context of what has been called “the revenge of reason,” this course explores two of the most prevalent new rationalisms today. On the one hand, there is the “orthogonalist” school most often associated with Nick Bostrom’s Future of Humanity Institute at The University of Oxford and Eliezer Yudkowsky’s Machine Intelligence Research Institute in Berkeley, as well as the Less Wrong and Overcoming Bias online forums so frequented by Silicon Valley tech workers. It is the orthogonalists’ rationalist—or, more precisely, decision and game theoretical—model of an advanced “artificial general intelligence” that would rapidly self-improve into a superintelligence in order to execute the goals we gave it but in unexpectedly catastrophic ways, that is behind the doomsday scenarios which appeared all over social media and the mainstream news in the wake of ChatGPT3’s release. On the other hand, there is the less widely known but steadily growing rival “neorationalist” school most often associated with philosophers Ray Brassier, Peter Wolfendale and Reza Negarestani. Largely influenced by the rationalist tradition stretching from Hegel to Robert Brandom, neorationalism has mounted a major critical examination of the orthogonalists’ conception of intelligence as a predominantly single agent that could be locked into pursuing the potentially catastrophic goals we inadvertently give it for all time. For the neorationalists instead, any general intelligence can only emerge through a community of rational agents collectively choosing their goals forever anew. This course considers how orthogonalism and neorationalism differ insofar as the former conceives of intelligence as a single agent that is programmed to pursue a fixed goal where the latter conceives of it as a multi-agent system that freely revises its values. Along the way, we will also see that despite these differences the two rationalisms share a commitment to the Humean is/ought, Moorean fact/value or Sellarsian causes/reasons dichotomy that leads to a conception of reason as having no intrinsic values or goals. Instead, all values and goals are freely determined by either human—or at least humanlike—beings.

Week 1. The Many Meanings of Reason

Before turning to the new rationalisms directly, we will first look at the many possible meanings of reason or general intelligence, such as those proposed by psychologist Howard Gardner’s “theory of multiple intelligences”: Linguistic; logical-mathematical; bodily-kinesthetic; musical; interpersonal; intra-personal; and several more besides. We will also look at a popular contemporary conception of rationality that emerges out of the modern AI research program and its efforts to build an artificial general intelligence (AGI) that is able to emulate human reason. As we will see, it is in this context that both Bostrom’s and Negarestani’s new rationalisms emerge.

Suggested readings:

• Howard Gardner, “The Idea of Multiple Intelligences,” in Frames of Mind: The Theory of Multiple Intelligences (New York: Basic Books, 2011), 3-11.

• Ashok K. Goel and Jim Davies, “Artificial Intelligence,” in The Cambridge Handbook of Intelligence, eds. Robert J. Sternberg and Scott Barry Kaufman (Cambridge: Cambridge University Press, 2011), 468-480.

Weeks 2-3. “Literally Everyone on Earth Will Die”: Bostrom’s Orthogonalism

The second and third weeks focus on the most prevalent and influential rationalist school today known as orthogonalism. After an initial survey of the history of the modern AI research program out of which orthogonalism arises, we will turn to Bostrom’s 2014 book Superintelligence: Paths, Dangers, Strategies to draw out five key conceptual axes that prove pivotal to the orthogonalist conception of intelligence and rationality. Firstly, notwithstanding some remarks about the possibility of a collective superintelligence, Bostrom tends to conceive of AGI as an individual agent or what he calls a “singleton.” He proceeds to analyze this AGI in terms of two theses. “The orthogonality thesis” holds that intelligence and final goals are orthogonal in the sense that any level of intelligence can be programmed to pursue practically any goal whatsoever. “The instrumental convergence thesis” holds that AGI will invariably pursue certain subgoals, “convergent drives” or “basic AI drives” like “self-preservation,” “goal-content integrity,” “cognitive enhancement,” “technological perfection” or “creativity” and “resource acquisition,” because they are a universally useful means of achieving any primary goal it might have. Bostrom’s hypothetical “AI takeover” scenario then proceeds to argue that this singleton AGI subject to the instrumental convergence and orthogonality theses would pose “existential catastrophic risks” (or x-risks), not by turning against us, but by fulfilling our goals in disastrous ways that we did not expect. It is the threat of this AI takeover scenario that leads Bostrom to his final point of proposing a “friendly AI project” for ensuring that AGI remains safe and friendly to humans.

Suggested readings:

• Nick Bostrom, “The Superintelligent Will” and “Is the Default Outcome Doom?”, in Superintelligence: Paths, Dangers, Strategies (Oxford: Oxford University Press, 2014), 105-126.

Weeks 4-5. “Nothing Can Stop the Rise of Intelligence”: Negarestani’s Neorationalism

While the orthogonalist school undoubtedly provides the predominant account of intelligence and rationality in the contemporary literature, we will see in the fourth and fifth weeks that it can be productively contrasted with the rival neorationalist school. In turning to Negarestani’s 2018 book Intelligence and Spirit, we will first trace neorationalism back to his critique of what he calls the “antihumanist” conception of general intelligence for being inadvertently humanist to the extent that it mistakes certain contingent epistemic and cognitive structures as the universal and necessary basis for all human thinking. We then zero in on Negarestani’s critique of Bostrom’s argument that an AGI could be locked into pursuing a disastrous goal for all time when in fact any truly general intelligence on Negarestani’s account must be able to revise its values and goals through the dialectical game of giving and asking for reasons with other rational agents. We will then be in a position to examine Negarestani’s constructive conception of general intelligence as only arising through a whole community of rational, language using agents autonomously determining and redetermining their values, purposes and goals in dialogue with one another. Negarestani ultimately uses this conception of a self-determining general intelligence to critique what he calls “aborted nihilism” and its efforts to collapse the Humean is/ought, Moorean fact/value or Sellarsian causes/reasons dichotomy by treating certain facts about nature like reason’s inevitable extinction as supposedly objective tendencies or norms in nature, which ought to limit what reason should strive to do in the here and now. In comparing and contrasting neorationalism and orthogonalism along the way, we will also be able to examine how they diverge from each other in some crucial respects while also sharing certain basic rationalist principles in others.

Suggested readings:

• Reza Negarestani, “Conception and Transformation,” in Intelligence and Spirit (Falmouth: Urbanomic, 2018), 1-86.

• Peter Wolfendale, “Rationalist Inhumanism,” in Posthuman Glossary, eds. Rosi Braidotti and Maria Hlavajova (Sydney: Bloomsbury, 2018), 379-382.