Hello to everyone reading the newsletter. Thanks for opening this email message or clicking my article on Substack!
I’ve not put my thoughts on here in a while. There have been a lot of developments in the AI space that I’m trying to take in, and I have several projects that I’m working on all at once.
This article will be much less formal and structured than normal, but I wanted to get some thoughts out there anyway. Big things coming ahead! Stay tuned.
For over a year, I’ve been thinking about how artificial intelligence can be responsibly integrated into healthcare.
This is a mission that can be described in one sentence. Yet, the process of discovering this answer has turned out to be more complex than I could have imagined.
One consistent pattern has emerged: as I try to answer one question, I am forced to answer a more abstract question as well. This keeps happening recursively.
For example, question #1 of my journey was “how do we stop healthcare in AI from hurting patients?”
Then I learn about data. If you don’t have the right kind of data, that can bias a model. If you don’t have good quality data, that can too. Even with the correct kind of quality data, if you don’t have enough of it, it still might not be enough.
Maybe question #2 is then “what kind of data is needed for AI to be safe and effective in the healthcare setting?”
I think that that answer is very dependent on how involved the AI model is going to be, how much human oversight is, and what the clinical function entails.
The next follow-up question is “how does a company or hospital decide what kinds of data are necessary to have in a given situation?”
At this point in the abstraction, I believe we start to enter uncharted territory. But how do you find the answer?
Try to go up another layer of abstraction. The question is now: “who is allowed to decide what data are necessary?”
Up again.
Who decides who’s allowed to decide?
How do we know that we’re measuring the data itself in a way that appropriately reflects clinical reality?
And what happens when someone makes a clinical AI service without asking any of these important questions?
So I spent many months trying to answer increasingly abstract questions.
I figured out my goal was to find the question at the top of the chain. The one that allowed me to start working my way back down the chain, providing satisfactory answers to areas I still feel are mostly incomplete.
Not too long ago, I think I finally found the “atom-sized” question:
What actions can someone take when they’re trying to create a clinical AI service?
The test is if the next step gets more specific (better) or less specific (worse). Let’s see.
What is a “clinical AI service”? More specific.
Who—or what—is “someone”? Does it have to be a person? Could it be an algorithm, or possibly even generative AI? Definitely more specific!
What actions are even possible to take? Do they have to be performed in a specific order? What happens when you decide not take certain actions?
Now I’m finally getting somewhere with this, thank goodness, because this whole thing was really starting to mess with my head.
The third question there, about the list of possible actions and their interactions, was interesting. Because that’s what I think governance would ultimately try to target.
Following the discovery of that question, I tried to classify as many actions as I could. Actions that I thought were good for patient safety as well as actions I thought would not be. Ones that make logical sense as well as ones that didn’t make sense at all.
Then, I started thinking about pairing. For example:
What happens when you do X task before Y task?
Or:
What two tasks should never happen at the same time?
Having asked enough of these questions, I started to build out a pretty comprehensive framework.
Then, I examined all the categories under each action. Things that interact with each other in unfavorable ways. Things you can’t do in certain scenarios.
As I was looking at one of these actions, I began to recognize a very familiar shape; it looked a great deal like a drug monograph.
And if that analogy holds up to my stress testing, then I am essentially working on creating a “drug information resource”/pharmacopoeia for AI governance.
That would make the Philly’s AI Pharmacist title WAY more literal than I ever intended it to be.