Interesting stuff, @Ryan Sears, PharmD - thanks for sharing
Still have questions, but seems to imply strongly that the LLM is essentially just an interface. If true, I’m generally fine with that type of construct even for a high-consequence case IF there’s verification of the inputs (visible to the user) that are going into the automated (non-inference) system and the outputs are direct from the automated system without inference in between.
If that’s the design, then all the LLM is doing is creating an interactive form, which is effectively a UI design and compute cost choice. I’d still vote for a scripted interaction or traditional form. But if people want natural language, you could do this pretty well with a very limited power language model.
I mean - I'd want to verify that this is explicitly true. But assuming it is, the balance of questions are important but mostly not AI questions about implementation. If I were being asked to assess the system I'd have a lot of questions about the decision logic, triggers and how they were defined, security and privacy, etc. None of this is "hard" in principle but it certainly matters.
Given how easy it can be to backdoor an LLM I'd want to know exactly what that tool touches and with what permissions, but in principle these are basic enough precautions that I'd expect a well designed system to be fine. Should literally only have very limited data sharing permissions on defined paths and probably nothing else. But people have messed this up before.
I'd also want to know how powerful a model they chose and why. And why it's superior to a set of relatively scripted responses that are called by inference without the possibility of the LLM saying something insane (also way less compute and fewer/none of the ethical questions)
This is really interesting! From what I gathered they are good steps with privacy and the LLM is more of an interface. I found question 16 to be the most interesting as I feel you are right. Until cases occur where a patient is harmed as a result of the technology we won’t know who is truly accountable. It sort of like social media. I am definitely sure with time Canada will consider this tech, so my advice stand even if this is not in your backyard right now, it could be coming.
“It is a reading posture for every healthcare AI story that is going to land in your feed over the next year. Companies will engage.”
this is a great view of what you discuss. really nice read, getting up to speed on this. thanks a ton, i think you have a solid perspective here.
Thanks so much for reading and the kind words! We have to team up when (not if) the next healthcare AI cybersecurity breach hits the news!
absolutely! that would be awesome :)
Interesting stuff, @Ryan Sears, PharmD - thanks for sharing
Still have questions, but seems to imply strongly that the LLM is essentially just an interface. If true, I’m generally fine with that type of construct even for a high-consequence case IF there’s verification of the inputs (visible to the user) that are going into the automated (non-inference) system and the outputs are direct from the automated system without inference in between.
If that’s the design, then all the LLM is doing is creating an interactive form, which is effectively a UI design and compute cost choice. I’d still vote for a scripted interaction or traditional form. But if people want natural language, you could do this pretty well with a very limited power language model.
Thanks for the inspiration for the model architecture questions, David!
Curious to know, what questions still remain for you?
I mean - I'd want to verify that this is explicitly true. But assuming it is, the balance of questions are important but mostly not AI questions about implementation. If I were being asked to assess the system I'd have a lot of questions about the decision logic, triggers and how they were defined, security and privacy, etc. None of this is "hard" in principle but it certainly matters.
Given how easy it can be to backdoor an LLM I'd want to know exactly what that tool touches and with what permissions, but in principle these are basic enough precautions that I'd expect a well designed system to be fine. Should literally only have very limited data sharing permissions on defined paths and probably nothing else. But people have messed this up before.
I'd also want to know how powerful a model they chose and why. And why it's superior to a set of relatively scripted responses that are called by inference without the possibility of the LLM saying something insane (also way less compute and fewer/none of the ethical questions)
This is really interesting! From what I gathered they are good steps with privacy and the LLM is more of an interface. I found question 16 to be the most interesting as I feel you are right. Until cases occur where a patient is harmed as a result of the technology we won’t know who is truly accountable. It sort of like social media. I am definitely sure with time Canada will consider this tech, so my advice stand even if this is not in your backyard right now, it could be coming.