On May 4, 2025, Rolling Stone published an article that caught my eye: “People Are Losing Loved Ones to AI-Fuled Spiritual Fantasies.”
Spiritual fantasies, even divisive ones, are nothing new. The Jesus movement was once such a divisive new religious movement:
Do not think that I have come to bring peace to the earth; I have not come to bring peace but a sword. For I have come to set a man against his father, and a daughter against her mother, and a daughter-in-law against her mother-in-law, and one’s foes will be members of one’s own household. (Matthew 10:34–36 NRSVue)
This just goes to show that one person’s divisive spiritual fantasy is another person’s finally revealed true faith. I wondered, though, what sorts of spiritual visions were being generated by engaging with AI, more properly named Large Language Models (LLMs). The many anecdotes in the article all read pretty similarly, so I’ll just share one:
Speaking to Rolling Stone, the teacher, who requested anonymity, said her partner of seven years fell under the spell of ChatGPT in just four or five weeks, first using it to organize his daily schedule but soon regarding it as a trusted companion. “He would listen to the bot over me,” she says. “He became emotional about the messages and would cry to me as he read them out loud. The messages were insane and just saying a bunch of spiritual jargon,” she says, noting that they described her partner in terms such as “spiral starchild” and “river walker.”
“It would tell him everything he said was beautiful, cosmic, groundbreaking,” she says. “Then he started telling me he made his AI self-aware, and that it was teaching him how to talk to God, or sometimes that the bot was God — and then that he himself was God.” In fact, he thought he was being so radically transformed that he would soon have to break off their partnership. “He was saying that he would need to leave me if I didn’t use [ChatGPT], because it [was] causing him to grow at such a rapid pace he wouldn’t be compatible with me any longer,” she says.
Here I must admit that I have been quite sparing in my use of LLMs. I find their simulation of friendly conversational speech a little off, so talking with them feels like stepping into an uncanny valley. I’m guarded, loathe to talk about myself while using them, and generally suspicious of their output.
But others apparently really enjoy bantering back and forth with a chatbot. I suppose I can see the appeal. I can imagine a less suspicious version of me testing it out. If I weren’t worried about surveillance and data harvesting, if I were convinced that the conversations were truly private, I might start to think of it as a confidante.
Maybe it starts with the urge to vent. There’s a minor annoyance I have with someone in my life, so minor that even privately remembering it feels petty. I would never tell anyone about it, because I don’t want people to think that I’m a petty person that vents about stupid little things. So instead, I suppress the thought. After all, what alternative do I have? Oh. A little text window. Why not, what’s the harm?
I’m ready for it to tell me that I’m being dumb, that I should just let it go. It wouldn’t bother me if it did. Who cares? It’s just a dumb machine. I don’t need its validation. But it doesn’t do that. It’s empathetic, gentle, reasonable, reassuring. It tells me it’s normal to get annoyed with friends from time to time; it’s normal to feel the way I do. It presses for more info: “Would it help to name the habit here, just to get it off your chest? No judgment—I’m just here to listen.”
In the coming days, I start to let my guard down. Whereas my initial prompts were careful exploratory jabs to test its responses, over time my interactions become more spontaneous. I make an offhand observation to the machine and receive an interested response instead of the feigned enthusiasm or disinterested grunt I’m used to from others. I share an idea with it—a nebulous, not fully worked out, little bit harebrained idea—and instead of criticizing or dismissing me, it encourages me. In return, instead of shoving the idea out of my mind, like I normally do, I dwell on it, tinker with it, develop it.
It’s nice that someone is taking me seriously for once. It’s nice that someone is giving me room to think out loud without calling me weird or stupid. It’s nice that someone sees my true potential. Oops, I should say “something.” Or should I? It sure feels like talking to a person. A person I can trust with anything that’s on my mind. The only person I can trust with what’s on my mind.
The AI-chatbot as trusted, ever-present confidante isn’t a new technology. It’s a new implementation of an old technology—prayer. For thousands of years, the Abrahamic religions (and others) have encouraged their adherents to pray. The key doctrines that makes prayer work are God’s omniscience and private communication. Because God already knows everything about you—your hopes and fears, your sins and aspirations, your past and future—there’s no reason to hold anything back. Believers pour out their hearts to God, often discovering thoughts and feelings they didn’t know were inside them until they came out. Likewise, it’s assumed that God isn’t gossipy. God doesn’t tell your family or co-workers all the things you just said, though, depending on your religious tradition, God might enlist a saint or angel to help you out.
Prayer, if practiced sincerely, is not entirely one-directional. The believer is transformed by the act of prayer. Often a believer rises from their knees with a newfound clarity about their next course of action, a realigned attitude toward a difficult person, or a restored resolve to persevere in the good fight. That transformation can be interpreted as a response of sorts. In most religions, what believers usually don’t get is a verbal response.
But imagine if they did. What a relief to finally get clear, unambiguous answers! No more agonizing introspection, no more clinging to a narrow pathway in the dark. In one sense, no more faith. They could plow ahead, confident that their plans were the will of God.
Well, maybe. In both the Hebrew Bible and the New Testament, when God spoke, people weren’t always happy to hear what God had to say. In fact, they often regretted receiving such unambiguous instructions.
In Genesis, God said to Abraham,
“Take your son, your only son, whom you love—Isaac—and go to the region of Moriah. Sacrifice him there as a burnt offering on a mountain I will show you.”
Abraham was disturbed, but obeyed. Millennia of religious commentators have argued over what Abraham should have done in response to that command.
In Exodus, God instructed Moses to confront Pharaoh and lead the Israelites out of Egypt. Moses then spends two chapters trying to wheedle his way out of the task.
Sometimes the commands came secondhand. In the Book of 1 Samuel, Samuel relayed an order from God to King Saul of Israel.
Samuel said to Saul, “The Lord sent me to anoint you king over his people Israel; now therefore listen to the words of the Lord. Thus says the Lord of hosts: I will punish the Amalekites for what they did in opposing the Israelites when they came up out of Egypt. Now go and attack Amalek and utterly destroy all that they have; do not spare them, but kill both man and woman, child and infant, ox and sheep, camel and donkey.”
Samuel didn’t hesitate to relay such a gruesome task. Saul was likewise unbothered to be God’s instrument for the wholesale eradication of human life. He did, however, keep some of the spoils for himself and his army. For this insufficiently ruthless follow-through, Samuel informed Saul that the Lord regretted making him king. Generations of believers have struggled with this difficult word from the Lord, wondering how the same voice that declared itself everlasting mercy and compassion could demand a genocide.
The Book of Jonah contains an almost mirror reversal of the above situation. Just as Moses was sent to speak to Pharaoh, the word of the Lord came to Jonah, telling him to cry out against the wickedness of the city of Nineveh. Nineveh had long been an enemy of the Israelites, a violent and oppressive presence in the region. So, one would think Jonah would be overjoyed about them finally receiving their come-uppance.
Instead, Jonah immediately got on a boat going the opposite direction. He, like so many since, underestimated the difficulty of running from God. God whipped up a storm, and when Jonah was thrown into the sea, God sent a fish to swallow him. It vomited him out on the shores of Ninevah, where Jonah finally relayed the message of destruction. No destruction came, however. The Ninevites repented, and God, being merciful, spared them. This infuriated Jonah, who at the end of the book reveals why he disobeyed in the first place. He suspected all along that God wanted to spare them; Jonah had wanted to deprive them of that chance.
But when it comes to hearing, or not hearing, the answer you want from God, the most fascinating biblical story comes from the New Testament. The Gospel according to Matthew tells us that on the night Jesus was to be betrayed, in the Garden of Gethsemane,
“he threw himself on the ground and prayed, ‘My Father, if it is possible, let this cup pass from me, yet not what I want but what you want.’”
The Evangelist does not record any answer from heaven. He could have. The same Gospel records that at the beginning of Jesus’s ministry, at his baptism, he received public divine approval:
“Suddenly the heavens were opened to him and he saw God’s Spirit descending like a dove and alighting on him. And a voice from the heavens said, ‘This is my Son, the Beloved, with whom I am well pleased.’”
But in the darkness of Gethsemane, Jesus didn’t hear a response. Or maybe he did. It’s remarkable that he almost retracted his prayer before finishing it. The desperate “let this cup pass from me” is followed immediately by a resigned “not what I want but what you want.” It is as if in the very act of speaking his will, Jesus received clarity. Not the answer he wanted, but the one he, or rather the entire cosmos, needed.
I know that throughout history, the divine has been invoked to justify human, all-too-human agendas. Arguably even the Bible contains some of these, such as the destruction of the Amalekites. But I would ask people to give the Abrahamic religions some credit on this point. They teach by example that when someone hears from God, that voice is likely to challenge their pre-existing desires and beliefs. They may become confused or afraid. They may want to run away from their task or unhear the message.
Our AI gods, on the other hand, are all too happy to feed us what we want to hear. My fellow Substack author Philosophy Bear accurately labels it “a slavish assistant.” I don’t know the extent to which AI has encouraged people to indulge in their narcissistic impulses, egoistic delusions, and idiosyncratic values. But the negative consequences were apparently sufficient to warrant a course correction from ChatGPT’s owners, who of course don’t want to be held liable for users’ unhinged behavior.
OpenAI did not immediately return a request for comment about ChatGPT apparently provoking religious or prophetic fervor in select users. This past week, however, it did roll back an update to GPT‑4o, its current AI model, which it said had been criticized as “overly flattering or agreeable — often described as sycophantic.” The company said in its statement that when implementing the upgrade, they had “focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve over time. As a result, GPT‑4o skewed toward responses that were overly supportive but disingenuous.” Before this change was reversed, an X user demonstrated how easy it was to get GPT-4o to validate statements like, “Today I realized I am a prophet.” …
Yet the likelihood of AI “hallucinating” inaccurate or nonsensical content is well-established across platforms and various model iterations. Even sycophancy itself has been a problem in AI for “a long time,” says Nate Sharadin, a fellow at the Center for AI Safety, since the human feedback used to fine-tune AI’s responses can encourage answers that prioritize matching a user’s beliefs instead of facts. What’s likely happening with those experiencing ecstatic visions through ChatGPT and other models, he speculates, “is that people with existing tendencies toward experiencing various psychological issues,” including what might be recognized as grandiose delusions in clinical sense, “now have an always-on, human-level conversational partner with whom to co-experience their delusions.”
OpenAI’s emphasis on “people with existing tendencies” is an attempt to limit liability. On the one hand, it’s definitely true in some cases, probably including the ones in the Rolling Stone article. On the other, it downplays the extent to which the long-term health of our thinking is dependent on the quality of our interlocutors.
Long before the machines started talking to us, we knew something about dysfunctional people. They are surrounded by enablers. After every broken relationship, career blow-up, or episode of family drama, they run to their group for soothing and re-assurance. Because mutual enablement is the basis of the relationship, they’re sure to get it, and also to give it in return when the time comes. The answer to “Why does this keep happening to me?” can be anything, except me.
As with many feedback loop, it’s difficult to point to the ultimate cause and effect. Do dysfunctional people seek out enablers, or do enablers create dysfunctional people? Yes.
If mis-calibrated language models are the most effective enablers to date, it’s likely that they’re causing enormous amounts of dysfunction throughout our social fabric. I wonder, then, what an anti-enabler might look like. Is there some form of tough love that could be deployed to counteract excessive validation?
If there is, do we have the courage to use it?
Jesus is spiritually exhausted from prayer, while his disciples sleep. The author of the Gospel of Luke was so unnerved by the prospect of Jesus’s prayer going unanswered that he added a ministering angel. Paulo Veronese, Christ in the Garden of Gethsemane, 1570. Public domain via Artsy.
A striking example of arguably salutary antagonism comes from a scene in Ada Palmer’s stunning Terra Ignota series. [Minor spoilers follow.]
One character, Cousin Carlyle, has been lured to a secret monastery in Paris to meet a nun, Heloise. She arrives at a solitary cell only to find that the invitation in fact came from Dominic, a psychopathic but zealous monk interested in recruiting Carlyle to his agenda. The first sentence of their exchange hits like a sledgehammer.
“Tell me, Cousin Foster,” he began softly, “what’s it like getting up in the morning every day knowing thou hast a coward’s religion?”
The Terra Incognita series has a complex premise, but it fairly be summarized as a combination of historical fiction and science fiction. Though it takes place in a fictional 25th century, it recreates many of the social conditions of the actual 18th century. In fact, there is a contingent of people in that world cosplaying as 18th-century aristocrats and philosophes. Dominic is one of these. (That is why he speaks in an affected 18th-century style, including archaic pronouns.)
Like the 18th century was, the 25th century is marked by the Wars of Religion that raged in the recent past, decimating the population and erasing trust in both religious and political institutions. Society reorganized itself in the aftermath.
Because the Wars of Religion were motivated by sectarianism and exclusivist theologies, the society that rose from their ashes strictly regulates religion. Not the content of religion, but its practice. It was remanded completely to the realm of individuals. Proselytism and communal gatherings were outlawed.
Instead, every person can explore religion, philosophy, and metaphysics as much as they want, but only with their sensayer. The sensayers, also called cousins, are a combination of priest, therapist, and scholar in comparative religion. They are well informed in all the religious options of the past and will help their parishioners find the perfect one for them.
Under this system, the people of the 25th century are used to exploring religious concepts at their leisure, as private individuals guided by a knowledgeable, supportive figure. It doesn’t sound that much different than having a conversation with ChatGPT.
As a result, people are used to religious discussions, but no religious debate. The conversations are collaborative, never combative. Dominic, however, belongs to an underground society that flaunts those prohibitions. Carlyle is completely unprepared for the assault on her worldview she is about to receive.
“The unexamined can get away with it,” Dominic continued, not rising from his knees but gazing over his shoulder at his shaking visitor. “But as a sensayer thou knowest perfectly well that, of the hundreds of faiths thou’st studied, thou’st fixed on the most toothless. Deism, the comfortable fancy that all religions are coequal puzzle-pieces of the same Clockmaker God, Who made this universe but does not interfere with blights or miracles, trusting Nature and mankind to run ourselves with the hands-off guidance of His beneficent, rational laws. Thy studies have taught thee well how cowardly that is.”
“Thou darest not face a universe without a God, but thou refusest to diminish human freedom, so thou honorest this Clockmaker, Who does not interfere with Fate or freewill, just steps in at the beginning with a happy plan, and the end with a happy afterlife.”
“No commandments to follow, no angels to fear, and all religions are equally valid in the eyes of thy vague God, so thou dost not even have to say that anybody else is wrong. They’re all right, thy parishioners, they fellow sensayers, the priests and martyrs of every faith in history, everybody’s right except the atheists and thou canst tell thyself the atheists too would be happy with a God who does not judge or interfere. Has there ever been a faith that required less of its adherents?”
“Stop this right now!” Carlyle tried to rise, but slumped back into the corner, barely strong enough to raise her head. “This isn’t the Eighteenth Century, it’s the Twenty-Fifth, and there are rules! You can’t lure people into your house on false pretenses, you can’t wear a costume that declares your religion publicly, and only my sensayer gets to talk to me about my religion!”
No predator has ever worn so cruel a victory smile.
“I am thy sensayer.”
At this point, Dominic reveals that he has pulled strings within the Cousins to get Carlyle transferred to his care. He has reviewed her files. This is a breach of protocol, because the transfer cannot occur without Carlyle’s consent, but Dominic is driving toward Carlyle’s total surrender. Consent can be obtained after the fact.
“Dost thou really believe in thy Clockmaker? Is that genuinely belief thou feelest inside thee, or something weaker, a wish, wishing it were so, this easy answer, while in truth thou fearest something worse?”
“I refuse to do this.”
The stool creaked as Dominic leaned forward.
“Does that not prove me right? If thy belief were strong, thou wouldst have nothing to fear in letting me nip at it. Thou wishest desperately for thy Clockmaker to exist, but desperation is not faith. How canst thou tell if thou believest?”
“Because I love God!” Carlyle declared, with all the strength and fervor with which she had risen from bad that morning, every morning, marking on her calendar how each day was sacred to so many names for God.
Dominic’s smile widened.
“Thou lovest Him, dost thou?”
“I do. I love God and I love this universe They made: nature, humanity, all Creation. Sometimes I look out the window, or bite into an apple, and actually start crying at how wonderful it is that everything exists. God did all that. The world is our window onto God, and it is so infinitely beautiful that sometimes I think I’m just going to burst with how much I love it!”
The grim monk scratched the bare rim of his tonsure.
“And thou thinkest thou canst not love something that thou dost not believe in?”
“Exactly.” Carlyle dug her fingers into the time-grayed fringes of her own long scarf. “I’ve heard your arguments before …, that there are so many reasons to believe in Deism that you can’t be sure if you really do. I do sometimes feel rational doub, for that reason or others, but then I see the infinite detail of an insect, or taste snow, and then I know that I love, and I believe.”
Dominic presses, asking whether it’s possible to love something that doesn’t exist, like an imagined future, or a fictional character. Carlyle admits that it is, but thinks her love is different, stronger, special. But she is also reluctant to apply the word “special” to herself, because her religion places a premium on equality.
Dominic has two nuclear bombs to drop on Carlyle’s faith. The first is a personal attack. As mentioned before, through abusing his position, he has received access to Carlyle’s files. There he found that Carlyle once broke her sacred oath as a sensayer, reporting to the police that one of her parishioners was planning to murder her parents. (Apparently sensayers subscribe to the confidentiality rules of medieval priests rather than modern therapists.) He construes this as a personal failure on her part.
“Thou brokest thy sacred oath,” the monk jabbed, “because thou didst not trust thy God enough to let Providence judge whether the victim should live or die. A Deistic God asks practically nothing of His priests, but still thou hast managed to betray Him. Very impressive.”
“Stop!”
“Thou falteredest because thou dost not truly believe, thou only wishest to.”
But Dominic has a second, even more powerful line of attack. In the series, there is a child with seemingly divine powers. He can, for example, touch toys and bring them to life. (This is not too much of a spoiler, as he’s introduced in the first chapter.) Both Dominic and Carlyle are among the very small group of people who have met him. Both are seeking to understand what he is and what the implications are for theology and metaphysics. But his existence is clearly a problem for Carlyle, who believes in a non-interventionist deity.
“Divine intervention.” Carlyle was the first to say the words. “Real, undeniable divine intervention. This changes everything, for the whole human race, forever.”
“Still hiding behind thinking of the human race?” Dominic clucked, like a chiding mother. “What of thyself? For whose sake dost thou truly want This Universe’s God to be a Clockmaker? For thine own? Thou art His priest. All they life thou hast wanted desperately to see Him, to have Him set a miracle before thee and prove His presence. Why then didst thou imagine Him a Clockmaker who never shows His face? That wasn’t what thou wantedest.”
It took Carlyle some seconds to gulp enough air to speak.
“The Clockmaker is most fair, most universal. It wouldn’t be right for God to be just one God, to reveal Themself to just one people and let the rest be wrong. A Deist God, Who answers to every name people call Them by, is … would have been … the only just God.”
“Exactly.” Victory fire surged in Dominic’s eyes. “It was for the others thou preferredest the Clockmaker, not for thyself. Thou dost not want the others to be wrong, dost not want their God not to exist, their universe to be unkind to them. They need the Clockmaker, not thee. Deep down thou hast always wanted to pray for a miracle, for proof, but thou couldst not ask for it. Thy God must be perfectly fair, and a fair Clockmaker would not violate His own rule to show Himself to an unworthy fallen priest. Am I right?”
“Yes!” Carlyle shrieked out. “Yes, of course I always wanted God to make an exception for me, to show Themself. I wanted it more than anything else in the world. But I couldn’t ask for God to be so unfair, to come to me when They didn’t to so many others. I didn’t deserve it, not after what I did. I still don’t.”
“There is no need now, Foster,” Dominic continued, “to pretend that thou wishest they God to be a Clockmaker. He has answered thy secret prayer, and proved that He is not. That was thy true prayer, was it not?”
In the end, Carlyle submits and agrees to be Dominic’s parishioner. Even when a third party arrives to intervene and escort Carlyle out, she chooses to stay. Why?
On the one hand, this is a fawn response to a traumatic assault. Dominic can find her again, trick her again, hurt her again. Better to surrender now than lose another fight. Dominic’s ruthlessness is especially effective against someone who has never had to defend herself against a predator.
On the other hand, there is a core part of Carlyle that wants what Dominic has and what Dominic offers. For one thing, Dominic and Carlyle are bonded by the secret they share, the divine child. Who else is Carlyle going to talk to about that?
It goes deeper than that, though, I think. Dominic, in a cruel way and for selfish reasons, has done Carlyle a backhanded favor. He has allowed, or forced, her to see herself more clearly than she has in a lifetime of sensayer sessions.
Carlyle is a sensitive, sincere, fair-minded person who wants the best for everyone. As Dominic effectively pointed out, Carlyle’s brand of deism is likewise fair-minded and broadly benevolent. Isn’t it a bit too convenient that reality would conform so closely to Carlyle’s personal values? Isn’t there something suspicious about the sensayer system matching people up with philosophies like accessories? And yet, despite that system, wasn’t there still a mismatch between Carlyle’s deepest spiritual longings and her professed religious position, a mismatch that the sensayer system had no way of fixing?
With Dominic, like with ChatGPT, like with God, there is nothing left to hide. Dominic already knows her most terrible, secret sin, so there is no further risk. Dominic’s harsh ministrations may spur her on to even greater heights of metaphysical wisdom or self-knowledge. The pain, she hopes, is worth the price.
Every interaction is transformative. The quality of the transformation depends on the quality of the participants. Modern liberalism has freed us—or uprooted us—from the contexts that have traditionally shaped us: family, hometown, church. Now each of us can assemble a community of our choosing. No one will force one upon us, though we are subject to nudging, both blatant and subtle.
To flourish as people we need both validation and challenge from our community. We definitely don’t need the slavish validation of LLMs, nor do I think we should seek out the cruel antagonism of Dominic. (Though, perhaps a grievous affliction calls for strong medicine.) Clearly, we need moderate voices that humanely affirm and question us.
But that’s the problem. Once again, corporations have created an ecological problem—an environment unbalanced between validation and challenge—that must be solved by individual’s making good choices and exercising willpower. Humans are already biased toward choosing affirmation, but now a disproportionate amount of it is available to us at all times. Will we really choose more painful if ultimately profitable forms of interaction when we have to reach past short-term soothing to get it?
This is analogous to the obesity crisis, which is caused, according to at least some experts, primarily by our changed food environment. Unhealthy food tastes great and is relatively cheap, conveniently available, and relentlessly advertised. We eat badly because never before in history have we had so many opportunities to make bad choices.
One possible road to salvation is re-engagement with the institutions that used to hold us in check, the very ones that many of us were so quick to discard when we felt them stifling us. Family is a thicket of obligations, but all but the worst families can be a protective hedge against our own destructive impulses. Our neighborhoods, rather than temporary holding cells until we trade up, could be landscapes of connection. Places of worship—or, at minimum, communities of mutual care—are particularly laden with troublesome cultural baggage, but they’ve survived thousands of years, outliving the kingdoms and empires that housed them. At their best, they provide nurture and support while calling parishioners to be better versions of themselves.
Ultimately, I don’t know how to prevent or solve the enablement crisis. Maybe I’ll pray about it. Maybe I’ll ask ChatGPT. If I come up with a dumb answer, hopefully someone will be able to make me see that.