included. They spoke the truth, even as their voices trembled. The Second Message: AI Risk My exposure to the second revolutionary message was via Yudkowsky’s blog—the blog that compelled me to reach out and arrange that meeting in California. The message was: Continued progress in AI can precipitate a ch

included. They spoke the truth, even as their voices trembled. The Second Message: AI Risk My exposure to the second revolutionary message was via Yudkowsky’s blog—the blog that compelled me to reach out and arrange that meeting in California. The message was: Continued progress in AI can precipitate a ch

orrelates with greater morality. Evidence from evolutionary and human history is adduced in favor of this point, along with more abstract arguments. Yudkowsky [Yud06] has discussed the possibility of creating AGI architectures that are in some sense "provably Friendly" — either mathematically, or else at le