Anthropic has been asking religious thinkers how Claude should behave

For months, Anthropic has hosted private meetings with dozens of religious scholars to help determine how to train its AI models to behave virtuously, The New York Times reported on Sept. 30 (The Philadelphia Inquirer, from The New York Times). Co-founder Chris Olah began reaching out to them last fall and started two-day seminars in March (The New York Times).

Orthodox Rabbi Mois Navon and Notre Dame philosophy professor Meghan Sullivan took part, and most seminar participants did not lead faith traditions. Separately, Olah held private talks with Cardinal Blase Cupich, the Catholic archbishop of Chicago, and Elder Gerrit W. Gong of the Church of Jesus Christ of Latter-day Saints, according to their spokespeople. Many participants signed nondisclosure agreements, which Anthropic said were lifted over the summer (The New York Times).

The meetings served two purposes, the Times says: to broach the possibility of AI consciousness and to learn how Anthropic might apply centuries of human moral wisdom to its models. At a late-March summit at Anthropic’s headquarters, about 15 people, including Catholic and Protestant clergy, scholars and businesspeople, reportedly discussed how Claude should respond to people grieving a loved one or at risk of self-harm, and whether Claude can be considered a “child of God” (Gigazine, summarizing The Washington Post). Speaking at the Vatican in May, Olah said, “We keep finding things that are mysterious, even unsettling” (The New York Times).

I use Claude every day, so this one is personal for me. Somebody is deciding what that model says to a person at 2 a.m. after a death in the family, and I’d rather it be a room with a rabbi and a philosopher in it than nobody at all.

If you lean on any chatbot for hard personal conversations, ask the company who it consults on those answers, and read what it publishes about how its model should behave. The ones that won’t tell you are telling you something too.

Who teaches a chatbot right from wrong matters as much as the code.