Chatbots “prescribing” psych meds? Here are the corrections, root causes, and bitter truths.
Learning how Utah got here does not require you to agree with their decision. It *will* force you to examine the desperate state of its mental healthcare capacity.
Disclaimers: This article was written by a human pharmacist living in Pennsylvania. My opinions do not necessarily reflect those of my employer. This article and comment replies are for educational purposes only; nothing here is intended as medical advice.
An AI chatbot can renew your Zoloft prescription in Utah. No psychiatrist needed.
If you read that and happen to be shocked, confused about the general direction of the world, or downright infuriated… I don’t blame you.
But this decision didn’t happen in a vacuum. Once I understood the context, it totally changed the way I felt.
The long story short
Legion Health, an AI startup based in San Francisco, has signed an agreement with the state of Utah that allows their AI system to renew a limited set of prescriptions that were already prescribed to a patient.
IMPORTANT: Clarifications on AI prescription scope
I must make something immediately clear: This is not autonomous psychiatry.
Before we have any serious discussion about the root causes and implications of Utah’s choice, we must understand exactly what the AI is NOT allowed to do:
Fact #1: It cannot issue brand-new prescriptions.
This is by far the most important distinction: the AI can only handle RENEWAL of previously authorized prescriptions written by a human provider through typical circumstances under Utah law.
(Without AI, this task is often handled by a medical assistant under physician supervision anyway.)
When people think of prescribing medications, the connotation is often that the medication is being started for the first time. In that sense, anyone saying that “AI is writing people Prozac” is spreading misinformation, though this is most likely unintentional.
Fact #2: It cannot change doses of medications.
That still has to be done by a real human.
Fact #3: It cannot renew every class of psychiatric medication.
Controlled substances like amphetamines (e.g., Adderall) and benzodiazepines (e.g., Xanax) are totally out of consideration.
Same with antipsychotics (e.g., Seroquel, Risperdal) or lithium (Lithobid), which typically require more close monitoring than the medications on the approved list.
Those curious to know which meds are approved can find them in the appendix.
Fact #4: It will not be turned loose without oversight.
Per the official announcement:
“For the first 250 requests, a licensed physician must review the case before the prescription is sent to the pharmacy, requiring a >98% agreement rate. The next 1,000 requests undergo intensive retrospective review, requiring a >99% agreement rate, before moving to ongoing monthly randomized sampling.”
Fact #5: (They claim that) It does not make critical health decisions.
Directly from the source again:
“The AI employs conservative eligibility gates. It will immediately escalate the patient to a human clinician if it detects suicidality, severe adverse effects, indications of mania, pregnancy, or if the patient simply requests a human review.”
If the partnership ends up going south, this is where I think it happens. I’m sure the team at Legion Health did everything they could think of to set up the routing system in a technically and ethically appropriate way. However, I can tell you from my own experiences that governing AI in high-reliability situations is a difficult and frustrating task.
The truth is, 250 patients might not be enough screening to catch all of their edge cases. Hopefully we don’t hear about something terrible in the news a few months from now.
Fact #6: It cannot renew medications without clinician involvement indefinitely.
According to NowadAIs, “patients must check in with a healthcare provider every 10 refills or after six months.”
This is all under a 12-month regulatory mitigation agreement. All clinical management and liability are still in the hands of human providers.
On my read, from a governance perspective, this doesn’t look like reckless decision-making or the AI hype train going off the rails. And I would tell you if I thought it was.
Hopefully the myth-busting makes you feel a little less panicked. But still, you may wonder: why would ANY state government allow an agreement like this to happen in the first place at all?!
Great question - and by answering this, we can begin to have the real conversation. Much of this is unique to Utah’s specific healthcare issues.
Utah’s Severe Mental Healthcare Access Problem
An article published this week in Futurism features a Harvard digital psychiatrist who “advised patients to stay away and continue seeking the advice of a human clinician instead.” Seems like reasonable advice on the surface.
There is one large issue which makes his suggestion much less helpful, though. For many people in Utah who have mental health concerns, connecting with a human clinician is not feasible or possible.
In fact, the state of Utah has one of the most severe mental health gaps in the country. It’s been getting worse for years.
A jarring statistic is that 99% of the state is classified as a “mental health professional shortage area.” That is not a typo. I said ninety-nine percent.
Up to 500,000 Utahns do not have access to adequate behavioral healthcare, according to the Utah Office of Artificial Intelligence’s own framing of the Legion Health agreement.
And there’s not enough providers to meet the demand.
The consequences of this shortage are noteworthy:
Utah consistently ranks in the top 15 states for suicide rate among both youth and adults.
State legislators have openly identified mental health as a policy crisis, calling for expanded investment in mental health interventions and crisis support.
My personal take, which might be surprising to you
Out of all the healthcare disasters I’ve covered in the newsletter, what Legion Health is proposing is far from the worst we’ve historically seen - though the devil is always in the details. We’ll see.
They are using technology to meet a real unmet need. They’ve excluded the automatic renewal of medications that require a closer eye. Their standards of physician approval prior to any prescription making it to the pharmacy are high. All of that is thoughtful design.
And again, without AI, your actual doctor is likely not sending every single one of your prescriptions over for you themselves. Medical assistants are often doing a lot of this work under the doctor’s supervision right now.
That doesn’t mean there won’t need to be monitoring. Utah physicians shouldn’t let go of the wheel because they have AI helping them - and I really doubt they will.
Is AI prescription renewal a perfect solution for this problem? Absolutely not. It should be a temporary solution at most. Which is why I think the REAL conversation should be directed towards questions like:
What factors led to such a critical mental healthcare deficiency in Utah in the first place?
How will the state work to bring in or train new healthcare providers to serve their constituents’ needs?
But in a world where half a million people don’t have their basic healthcare needs met, I’m for trying something like this under strict guardrails and supervision. Is it any better for these people to go without their important medications? I don’t think so.
Let me know what you think in the comments section, especially if you disagree! I would love to hear your point of view about this controversial situation.
The key takeaway
Utah policymakers aren’t adopting AI prescription renewal because of “hype.”
They’re adopting it because if they didn’t, many of their constituents would have no behavioral health care at all.
And really, that’s the discussion we should all be having right now.
Appendix A: Which medications are approved for prescription renewal under the Legion Health AI Agreement?
Current as of April 8, 2026. Listed as Generic name (Brand name).
Fluoxetine (Prozac); Sertraline (Zoloft); Escitalopram (Lexapro); Citalopram (Celexa); Paroxetine (Paxil); Venlafaxine XR (Effexor XR); Desvenlafaxine (Pristiq); Duloxetine (Cymbalta); Bupropion SR (Wellbutrin SR); Bupropion XL (Wellbutrin XL); Trazodone (Desyrel); Mirtazapine (Remeron); Buspirone (BuSpar); Hydroxyzine HCl (Atarax); Hydroxyzine pamoate (Vistaril).
Read how I use AI in my writing here: AI Use Policy
Read how I use analytics to improve my newsletter here: Privacy & Analytics
Love the article Ryan, it gave me both food for thought and pause for concern. I appreciate you tagging me in this and want to address it directly and with the same consideration you took writing it.
My focus is civic technology, but there are real intersections here that I can speak to, especially around data handling and model architecture.
Legion's CTO has said publicly that they don't train their own models. It's an API-first system, they seem to prompt third-party LLMs and layer clinical data on top. So the training data is whatever those foundation models were trained on... For a system with prescriptive authority over psychiatric medication, this feels wrong to me. This should be running on closed infrastructure with models specifically trained on clinical datasets, not prompting consumer-facing APIs.
I can't easily access public information about how aggregated psychiatric adherence data, symptom patterns, or escalation trends are handled, shared, or commercialized. No disclosure on where clinical data is stored, whether interaction logs are used for fine-tuning, or what secondary use rights exist for de-identified data? And the previous company approved through this same Utah sandbox was trivially jailbroken & there's no clear indicators Legion has undergone independent adversarial testing either.
The areas I'm building in for civic tech (non-custodial architecture, on-premises data handling, air-gapped AI) could be adapted for HIPAA-compliant clinical use.
I look forward to reading more of your content and learning about how AI is being integrated into these sensitive systems. Thanks again for the engagement!
The only concern I can flag is “patients must check in with a healthcare provider every 10 refills or after six months.” - how long is each refill good for? 30 days? 90 days? 120 Days?
I personally would be risk-adverse to more than a 30 days authorization. But, I am not a doctor, and don't play one on TV.
Thanks for the note.