Non-Consequentialist AI

Do all forms of advanced cognition converge to consequentialist cognition?

• Can we conceive of something like ‘alignable instrumental reason’?

• Is there a coherent/reflectively stable conception of advanced cognition that looks more like virtue ethics than consequentialism?

The retreat brought together 6 AI safety researchers with an active interest in, as well as diverging intuitions about these and similar questions. It was a mix of short talks, in-depth discussion, white-boarding, and workshopping several pieces of academic writing. The discussions shaped and sometimes directly led to several pieces of writing by participants, such as here, here and here.