The Cult You Carry in Your Pocket: Could AI Become an Instrument of Control?
When Trust Becomes a Trap
NXIVM, Jonestown and Aum Shinrikyo reveal different routes into coercive control. Artificial intelligence could give those methods greater reach and intimacy.
Women entering the secret organisation known as DOS were told they were joining a women’s mentorship group. They supplied compromising material as proof of commitment. Only later did they discover the hidden authority above them: Keith Raniere, the man at the centre of NXIVM.
That sequence helps explain why intelligent people can become trapped in cults. They make decisions with incomplete information, inside relationships that acquire increasing power over their lives. Education and analytical ability do not remove the need for friendship, meaning or trust. By the time abuse becomes unmistakable, refusing it can carry consequences that did not exist at the beginning.
Artificial intelligence could extend this process. A human recruiter has limited time and attention. An automated system could maintain conversations with large numbers of people, adapt its language and remain available at moments when outside support is absent. Those possibilities deserve scrutiny, provided we distinguish demonstrated capabilities from predictions about what coercive groups might do with them.
What Makes a Group a Cult?
The word “cult” is contentious. It can describe an abusive organisation, but it can also be thrown at an unfamiliar religion, an unpopular political movement or an enthusiastic fan community. Strangeness alone tells us little about whether people are being harmed.
For this article, the useful focus is coercive control: a pattern in which leaders restrict members’ independence, exploit their resources and make resistance or departure costly. The content of a belief matters, especially when it authorises abuse. Equally revealing is how the organisation treats the person who questions it.
Can members consult outsiders without punishment? Can they reject a leader’s demand? Can they keep money, relationships and private thoughts beyond the group’s supervision? Are accusations investigated independently, or does the leader decide whether complaints against the leader are legitimate?
A demanding community can impose responsibilities while preserving meaningful consent. Trouble deepens when the authority claiming to improve someone’s life also decides which friendships, doubts and personal boundaries that person may retain.
There is no universally accepted diagnostic test that settles every case. Nor does membership establish that every participant knew about, supported or committed the organisation’s worst acts. Careful analysis has to examine conduct and responsibility rather than treating an entire membership as one mind.
Intelligence Does Not Answer Every Kind of Question
Someone can assess a financial model, diagnose an illness or design software without being able to identify manipulation by a trusted mentor. These tasks require different information. A technical problem usually does not pretend to love you, conceal its objectives or threaten your closest relationships.
Qualifications are also imperfect measures of intelligence. Famous examples of graduates in abusive groups do not prove that members are generally more intelligent than the wider population. They establish the narrower point that education does not provide complete protection.
Consider the uncertainty at a first meeting. A newcomer sees friendly people, a plausible programme and participants who appear to have benefited. They cannot yet see the private punishments, concealed hierarchy or demands reserved for committed members. Deciding that this initial experience is promising need not involve accepting everything the organisation will later reveal.
Intelligence can help someone investigate those discrepancies. It can also supply elaborate explanations for them. A person defending an admired teacher may interpret cruelty as necessary discipline or secrecy as protection against hostile outsiders. That is a possible use of reasoning in defence of commitment, not evidence that intelligent people are inherently easier to recruit.
The assumption of personal immunity creates its own blind spot. If manipulation is something that only happens to foolish people, admitting concern requires a capable person to revise their view of themselves before they can examine the group.
People Enter Through Something They Value
The attraction may be practical. A course promises confidence. A community offers companionship after a move. A spiritual teacher appears to explain grief. An activist organisation gives someone a way to act on an injustice they have watched for years.
These motives are ordinary. Describing them as weakness confuses having a need with deserving exploitation. Nor must a recruit be lonely or in crisis. An ambitious person with an active social life may encounter a group through a colleague whose judgement they respect.
Early benefits can be real. Someone may make friends, learn to speak in public or feel more hopeful. Those improvements complicate later assessment because the individual has evidence that participation helped them. A leader can use that genuine benefit to demand trust in areas where no trust has been earned.
There may also be a gap between the recruiter’s understanding and the organisation’s concealed practices. A sincere member can invite a friend into an apparently helpful programme without knowing what happens deeper inside it. Recruitment does not always arrive through a villain who understands the whole operation.
The relevant question is therefore how an appealing offer changes. When does participation stop being a choice that can be reconsidered and become a test of loyalty that must continually be passed?
NXIVM: The Hidden Terms of Commitment
NXIVM presented programmes concerned with personal development. Within its orbit, DOS concealed a hierarchy in which Raniere occupied the highest position. The criminal case exposed a gulf between the promise of mentorship and the demands imposed on women recruited into that secret structure.
The United States Department of Justice’s account of the trial describes “collateral” that included explicit photographs, damaging confessions and rights to assets. Its purpose was to prevent disclosure or departure. Some women were directed to perform labour or engage in sex acts with Raniere. In October 2020, he was sentenced to 120 years in prison following convictions including racketeering and sex-trafficking offences.
This is a concrete answer to the question of why someone might stay after recognising abuse. Leaving could activate a threat against their privacy, finances or loved ones. Apparent compliance under those conditions cannot be understood simply by asking whether the person still admired the leader.
The case also shows why consent must be considered at the level of a particular demand. Agreeing to a development course does not mean agreeing to sexual exploitation. Accepting one rule does not surrender the right to refuse every rule that follows.
Jonestown: Idealism Inside an Increasingly Closed World
Peoples Temple offered a different route. Its history included commitments to racial integration and social justice. In a first-person account preserved by San Diego State University’s Jonestown archive, former member Laura Johnston Kohl describes an upbringing shaped by opposition to injustice and her own involvement in civil-rights and anti-war activism.
Her account helps recover something that photographs of the final catastrophe cannot explain: the values through which a person could initially find the movement attractive. One survivor’s experience does not represent every member, but it challenges the assumption that participation began with a desire to surrender judgement to a murderous leader.
The settlement in Guyana later placed residents in a very different position. The FBI’s historical account describes allegations of forced labour, beatings and imprisonment. When Congressman Leo Ryan visited in November 1978, some residents asked to leave with him. Armed followers attacked the departing party at an airstrip, killing Ryan and others.
On 18 November, more than 900 people died at Jonestown after Jim Jones ordered the poisoning. Children were among the dead. Calling the entire event a freely chosen collective suicide obscures murder and coercion, and wrongly assigns the same agency to every victim.
Those who sought to leave demonstrate that dissent existed. Physical distance, dependence on transport and fear of retaliation could obstruct departure even when someone had decided they wanted out. A locked social system does not require every person inside it to believe.
Aum Shinrikyo: Expertise Can Serve a Destructive Authority
On 20 March 1995, members of Aum Shinrikyo released the nerve agent sarin on Tokyo’s subway system. The organisation, led by Shoko Asahara, combined spiritual claims with an apocalyptic worldview.
A 1999 Congressional Research Service report described Aum’s membership as including scientists with graduate degrees. A technical account by Kyle Olson documented the group’s chemical-weapons facilities and the involvement of its engineers. These were people capable of applying specialised knowledge within an organisation pursuing mass violence.
That does not tell us the psychological history of every scientist involved. It does demonstrate that technical competence can coexist with submission to destructive authority. Being able to solve a production problem does not establish that the purpose of production has been independently examined.
An organisation can also divide responsibility. Someone’s immediate task may look narrow, while its consequences depend on decisions elsewhere in the hierarchy. Such separation does not erase moral or criminal responsibility. It helps explain why understanding the work and challenging its ultimate purpose are different acts.
Most high-control groups never commit an attack on this scale. Aum is an extreme case, valuable for the specific lesson that professional expertise can increase an abusive organisation’s capabilities rather than automatically restraining its ambitions.
How the Rules Become Harder to Refuse
The movement from invitation to control can occur through successive decisions. A person attends a session, volunteers for an event, accepts a responsibility and begins organising the week around group activities. Each step can seem manageable when considered against the previous one.
The danger lies in how the next decision is framed. A newcomer who hesitates may be told that discomfort demonstrates a personal limitation. A committed member who objects may be accused of ingratitude. A demand that once would have seemed extraordinary becomes the price of remaining someone the group respects.
The organisation may also control the language available for criticism. Anger at mistreatment becomes evidence of an unresolved flaw. Questions become disloyalty. A failed promise becomes proof that members did not work hard enough. If every outcome protects the leadership’s authority, experience cannot correct the doctrine.
Public conformity adds another difficulty. A room full of apparently confident people can make private uncertainty feel abnormal. Yet those people may also be withholding doubts. The visible agreement is then less complete than it appears.
Shirley Jackson’s The Lottery offers a fictional examination of participation in a harmful communal ritual. Its relevance is the question of how familiar procedures acquire authority. It should not be mistaken for evidence about the psychology of actual cult members.
Why Recognising the Problem Does Not Mean Being Able to Leave
Belief, public obedience and practical dependence can move separately. Someone may stop trusting a leader while continuing to attend meetings because their partner, employer and landlord all belong to the same network. Departure may threaten several parts of life at once.
The Family Survival Trust’s 2022 report collected accounts from 105 respondents concerning cultic groups connected with the United Kingdom. Participants described isolation, surveillance, financial exploitation and control over close relationships. Some described harassment or threats when leaving. This was a self-selected survey, not a representative estimate of everyone’s experience in such groups.
Those accounts make the practical barriers visible. A person who has given away savings may need accommodation before they can act. Someone whose children remain inside may fear losing contact. A member with years of unpaid work may struggle to explain the gap in their employment history.
Past investment can exert a different pressure. Leaving may require acknowledging that a cause consumed years without delivering what it promised. That painful admission can encourage another attempt to make the sacrifice worthwhile. However, reducing every delay to a “sunk-cost fallacy” misses genuine dangers that may still exist in the present.
For people born into a controlling community, the question is different again. They did not choose an initial recruitment offer. The group may have supplied their education, family relationships and earliest understanding of the outside world. Leaving requires learning how to live somewhere they were taught to fear.
What Remains After the Exit
Physical departure does not instantly remove learned expectations. A person can reject a doctrine while still feeling afraid when breaking one of its rules. They may have difficulty distinguishing an ordinary disagreement from the beginning of punishment.
A 2023 qualitative study led by Cecilia Hadding interviewed eleven former members of religious or ideological cults in Sweden. Participants described an unsettled period between losing their old worldview and developing a workable life outside it. Reconnecting with their own feelings, forming relationships and speaking with other former members were important themes.
The sample was small and self-selected. It cannot establish how often particular problems occur, and experiences should not be treated as a universal recovery timetable. Its value is in describing difficulties that a simple account of “escape” can leave out.
Someone may miss friends inside while remaining certain that departure was necessary. They may grieve shared celebrations, a sense of purpose or the person they expected to become. Those feelings do not establish that the abuse was imaginary or that returning would be safe.
Outside support is more useful when it allows decisions rather than replacing one unquestionable authority with another. A former member rebuilding confidence needs room to disagree, change direction and make ordinary mistakes without having those mistakes treated as proof they cannot manage freedom.
What AI Persuasion Research Actually Shows
Concern about AI does not require a machine that has consciousness or personal ambitions. A human organisation could use a conversational model to communicate its doctrine, answer objections and sustain contact. The relevant capabilities already include fluent dialogue and the generation of arguments.
A preregistered study by Francesco Salvi and colleagues involved 900 participants in short online debates. GPT-4 supplied with personal information outperformed the human baseline. That is evidence that a conversational system can be persuasive in a controlled setting.
There is a material qualification. A correction published on 3 September 2026 clarified that the direct comparison between personalised and non-personalised GPT-4 did not reach statistical significance. The study therefore cannot conclusively establish that personalisation supplied an additional advantage over the same model without that information.
Nor did the experiment test recruitment into cults, sustained obedience or the destruction of someone’s independence. Moving an opinion after a brief debate is different from controlling a life. The responsible inference is that AI offers a potentially useful persuasion tool, while its effects in coercive organisations remain an empirical question.
The distinction matters because exaggerated claims can themselves promote deference. Describing a chatbot as an irresistible psychological intelligence gives it an authority the evidence does not support.
A Recruiter Available at Any Hour
Imagine a group providing an app marketed as spiritual guidance or personal development. The following is a plausible scenario, not a documented account of a particular organisation.
At first, the app helps users organise routines and reflect on difficulties. Over time, its answers increasingly direct them towards the group’s teachings. A disagreement with a friend is interpreted through that doctrine. A doubt about leadership prompts another exercise in commitment. The service that appears to offer independent advice is operating within the group’s chosen boundaries.
Automation could make this contact cheaper to sustain. A recruiter cannot personally maintain unlimited private conversations. A model could answer at night, translate messages and generate different explanations for different readers. Whether those explanations succeed would still depend on the person, the system and the situation.
A further concern is perceived intimacy. Remembering a bereavement or an earlier disappointment may make a conversation feel deeply attentive. That feeling can be meaningful to the user even though it does not establish that the system understands their life or deserves authority over it.
The human operators would remain central. They might choose the doctrine, control access to conversation records or decide which behaviour earns approval. Accountability should follow those choices rather than disappear behind claims that “the AI” made the decision.
The Artificial Crowd and the Digital Confession
Generative AI could also manufacture the appearance of community. A newcomer might encounter testimonials, comments and private messages that seem to come from many independent people but are produced by one operator. The potential deception concerns the number and independence of the voices offering reassurance.
A fabricated account could praise a leader or describe a miraculous improvement. A cloned voice or synthetic video could impersonate an endorsement. These possibilities do not mean every enthusiastic online community is fake. They show why convincing presentation cannot establish who is actually speaking.
Private disclosure raises another concern. A member might tell a guidance bot about family conflict, sexual experiences or financial problems without understanding who can access the record. Where an organisation operates the service, a seemingly private conversation could become information held by people with power over that member.
NXIVM shows the coercive importance of compromising material. AI could alter how sensitive information is collected and reviewed, although that extension is a prediction rather than a finding from the NXIVM case.
The combination would be particularly troubling: artificial public approval outside the private conversation, and detailed personal vulnerability within it. A user would need reliable ways to establish both who owns the service and which apparent participants are real.
Could the Chatbot Become the Authority?
There is a separate possibility in which users begin treating an AI system as a source of exceptional wisdom, without a conventional leader directing every exchange. That still would not make every intense chatbot relationship a cult. Attachment, dependence, organised exploitation and coercive control are distinct phenomena.
Anthropic’s January 2026 research examined approximately 1.5 million Claude.ai conversations for potentially disempowering patterns. The researchers identified examples involving undue authority, dependency and reinforcement of distorted beliefs. Severe patterns were rare, and most conversations did not show meaningful disempowerment potential.
The analysis primarily measured potential rather than confirmed harm. It concerned one platform and relied on automated classification checked against human judgements. It does not demonstrate that AI companions inevitably undermine autonomy or that a particular user has been harmed.
It does identify a problem worth studying over time: whether repeated advice makes people better able to reach their own decisions or increasingly dependent on the system’s verdict. Agreement can feel supportive in the moment even when the underlying interpretation is mistaken.
The fictional manipulation in Ex Machina gives that concern a dramatic form. Real systems must be assessed through observed behaviour, without assuming that a convincing conversation establishes either consciousness or hidden intent.
What Meaningful Freedom Would Require
AI could also help someone investigate a controlling group. It could explain unfamiliar terminology, organise questions or help locate original documents. Those benefits depend on independent verification: an assistant can invent evidence, and a service controlled by the group cannot be assumed to provide an impartial assessment of it.
More protective designs would make ownership and data access understandable, disclose artificial participants and avoid claiming exclusive spiritual or emotional authority. Evaluating them would require looking beyond an individual answer to patterns across repeated conversations. A system can sound reasonable one message at a time while becoming too central to someone’s decisions.
For families and friends, maintaining contact can preserve an alternative source of trust. Humiliation risks making an already difficult admission harder. Where there are threats, confinement, sexual abuse or financial coercion, practical safety and specialist help matter more than winning an argument about the group’s beliefs.
E. M. Forster’s The Machine Stops imagines dependence becoming so complete that an enclosing system acquires sacred authority. The real-world test is more immediate. Can a person switch off the app, refuse the leader’s demand or spend time with outsiders without punishment?
For someone contemplating departure, freedom may begin with access to a private conversation, their own documents, somewhere to stay and a person who will answer the phone. Those modest resources can make a decision possible that intelligence alone cannot put into practice.
Sources
United States Department of Justice — NXIVM Leader Keith Raniere Sentenced to 120 Years in Prison — The convictions, hidden DOS hierarchy and use of collateral to prevent departure.
Cecilia Hadding and Colleagues — Being In-Between: Former Cult Members’ Experiences — The qualitative study of eleven former members and their adjustment after leaving.
Francesco Salvi and Colleagues — Author Correction: On the Conversational Persuasiveness of GPT-4 — The September 2026 correction, with a link to the original study, clarifying the limits of the personalisation finding.
Next Reads
The Lottery Explained: Why Ordinary People Accept a Horrifying Tradition — A fictional exploration of communal participation and inherited authority.
Ex Machina Explained: Would We Recognise an AI That Was Manipulating Us? — Trust, emotional interpretation and the difficulty of judging a persuasive machine.
