Healthcare providers face a difficult reality as general-purpose AI becomes instantly available to clinicians and patients. A public tool may be easier to reach than an approved alternative, even where the approved system is designed to protect patient data, fit clinical workflows, and retain an audit trail.
That makes adoption friction a safety issue as much as a technology issue. In this TNGlobal Q&A, Dr. Ilya Burkov, Global Head of Healthcare & Lifesciences at Nebius, discusses what it takes to make governed AI practical in clinical settings, how local validation should work, and where human judgment must remain explicit.
How can hospitals make the safe, validated AI option easier to use than public tools?
Shadow AI is partly a usability problem. If a clinician can open a public AI tool in seconds, but the hospital-approved alternative requires several steps to access, people may naturally gravitate toward the easier option.
The safe tool needs to become the path of least resistance. In practice, that means embedding validated AI into systems clinicians already use, such as electronic medical records, imaging, or documentation platforms, with identity, permissions, and secure access to relevant data handled in the background.
The infrastructure matters too. Hospitals need control over where sensitive data goes, which models can access it, what is retained, and whether interactions are auditable. A good blueprint is Singapore’s AI in Healthcare Guidelines, which assigns responsibilities across developers, healthcare organizations, and professionals.
Adoption friction is now part of the safety equation. A validated tool that clinicians routinely work around is not an effective solution. Clinical AI needs to reflect how healthcare professionals actually work under pressure.
What separates a technically accurate AI output from one that is clinically safe or actionable?
A model can produce an answer that is factually defensible and still be clinically inappropriate. If a patient enters several symptoms and receives five possible diagnoses, every diagnosis might genuinely be associated with those symptoms. A clinician, however, will consider how likely each diagnosis is for that particular patient, given their age, history, medication, examination, test results, and disease prevalence.
That is why accuracy alone is an incomplete measure of medical AI. We also need to know how often a system misses something important, creates unnecessary alarms, performs across different patient groups, and behaves when information is incomplete. Most importantly, we need to understand what happens when someone acts on its output.
The World Health Organization’s guidance on generative AI in healthcare similarly warns that plausible but inaccurate or biased outputs can create health risks.
In medicine, an answer needs to be accurate, relevant, appropriately weighted, and connected to the right next action.
What should meaningful local validation test in a market such as Singapore?
All of those factors matter, but I would start with the intended clinical use rather than a checklist of demographic variables. The fundamental question is whether a system performs safely for the population, workflow, and decision for which it is intended.
Population matters because disease prevalence, ethnicity, genetics, and comorbidities can differ between markets. In Singapore’s multi-ethnic population, aggregate performance can also conceal differences between patient groups.
Validation cannot stop with the dataset. Hospitals need to test how AI behaves inside the actual clinical workflow. Does it receive the same quality of information it was developed on? Does an alert appear at the right point in the consultation? Does it improve a decision, or simply create another notification clinicians learn to ignore? For patient-facing systems, language and the way people actually describe symptoms also matter.
Singapore is already taking this direction. Synapxe has described AI systems being tested against local clinical data and in clinical environments before wider deployment, including its HealthVector Diabetes system.
I would think about local validation in three layers: population validity, clinical validity, and workflow validity. Passing only one is not enough to deliver safe, high-quality clinical outcomes.
What works better than simply publishing a policy against unapproved AI use?
There are four things to consider:
- Give clinicians an approved environment that solves the problems driving them toward public AI in the first place. If doctors are using general-purpose tools to summarize information, draft notes, or search medical literature, blocking those tools without providing an alternative leaves the underlying demand untouched.
- Govern access technically. Organizations should know which models are available, which data they can access, where that information is processed, and whether prompts and outputs are retained.
- Create traceability. For clinically meaningful AI, hospitals should be able to determine which model produced an output, what information was available to it, and what happened afterward.
- Continue governance after deployment. Patient populations, workflows, and models can all change. Singapore’s HSA digital-health framework takes this lifecycle approach, including post-market monitoring of real-world performance.
Healthcare AI should be treated like any other safety-critical system. Engineer the operating environment, build safeguards into it, monitor performance, and establish clear responsibility when something goes wrong.
None of this works if governance is written by one function alone. The organizations getting this right bring clinicians, nurses, IT, ethics, and compliance together from the start. The people who will live with the system are best placed to help govern it.
How should providers respond to patients using general-purpose AI for health information?
We should start from the reality that patients will use these tools. Telling them not to is unlikely to work when AI is instantly available on the device in their pocket.
Healthcare providers should instead become the trusted place where patients can bring what they have learned. A patient should be comfortable saying, “AI told me this about my symptoms and that I may have a particular illness. Should I be worried?” That conversation tells the clinician something valuable about the patient’s concerns and understanding.
Providers can also offer trusted digital channels of their own. They do not need to compete with consumer AI on breadth. Their advantage is clinical grounding, clear escalation routes, and reliable connections to care. The key distinction is between information and diagnosis. AI can help patients understand terminology, prepare questions, or navigate health information. Once it begins interpreting symptoms, determining urgency, or recommending treatment, the risk changes substantially.
The objective should be to channel AI use rather than deny it, and to give patients somewhere clinically grounded to take the question next.
How should health-oriented AI communicate uncertainty without increasing anxiety?
Medicine is fundamentally about probabilities, and health-oriented AI needs to communicate that. If someone enters “headache” and receives a brain tumour among a list of possibilities, the model may technically be correct. Without prevalence, patient history, severity, duration, and other symptoms, that information can be more frightening than useful.
Responsible systems should distinguish common explanations from uncommon but serious ones, communicate uncertainty explicitly, and explain which warning signs would materially change the level of concern. If there is not enough information to make a useful assessment, the system should say so rather than generate a more confident-sounding answer.
There should also be clear escalation. When a defined safety threshold is crossed, the correct response is not necessarily more AI-generated text. It may be directing the patient to an appropriate healthcare professional or service.
Developers should test behavioural outcomes as well as answer quality. Does the system cause unnecessary emergency visits? Does it falsely reassure someone who needs care? Does the patient understand what to do next?
A responsible system should help users understand not only what might be true, but how uncertain that conclusion is and what they should do with that uncertainty.
Where should the boundary remain between AI assistance and human clinical judgment?
There is no permanent boundary because AI will keep evolving and improving. A more useful way to think about it is that AI can already take on a great deal of documentation, note summarisation, literature search, and pattern spotting. This is where it earns its keep, freeing clinicians from repetitive work so they have more time for patients.
A large part of what clinicians provide is not information at all. It is reassurance, empathy, and the sense of being cared for by another human being. No model replicates that, and patients do not want it to.
The stakes change as consequences rise. Starting or stopping treatment, making a diagnosis that changes someone’s care, deciding whether a patient needs urgent intervention, consent, and end-of-life care are very different from drafting a note. AI can still help, even as a strong second opinion, but there needs to be a named clinician who can challenge its recommendation, bring in context the system does not have, and own the final call.
Auditability matters just as much. For decisions with real consequences, we should be able to trace exactly what the AI saw, which model produced the recommendation, what it recommended, and what the clinician actually decided. Good infrastructure captures this automatically, so it does not become another form to fill in.
The clinics of the future will be led by clinicians who use AI and know how to use it well. AI can extend what a clinician can see, analyse, and act on, but a human must remain accountable when a patient’s life is at stake.
Editor’s note: This Q&A has been lightly edited for clarity and TNGlobal house style. The substance of the interviewee’s responses has been preserved.
Share your perspective: TNGlobal welcomes contributed insights and expert commentary from across Asia’s technology and innovation ecosystem. Submit a contribution for editorial consideration, or explore more conversations in our TNGlobal INSIDER and TNGlobal Q&A and Interviews archive.
Klaviyo’s Brian Kealey on what agent-led commerce changes for brands [Q&A]