AI Health Advice: What Doctors Really Think.
The rush to deploy AI in healthcare has sparked concerns about accuracy and trust. With over 230 million people asking health-related questions on ChatGPT each week, the potential benefits of democratized information seem clear-cut. However, experts warn that this trend is fraught with risks.
"What I'm worried about as a clinician is that these general-purpose LLMs are still prone to hallucinations and erroneous information," says Saurabh Gombar, a clinical instructor at Stanford Health Care and co-founder of Atropos Health. "It's one thing if you ask for a spaghetti recipe with an unusual ingredient, but it's another story when it comes to fundamental health advice."
Doctors are concerned that AI-powered chatbots will perpetuate misinformation, erode trust between patients and healthcare providers, and even lead to adverse outcomes. For instance, a patient might be convinced they have a rare condition after chatting with AI, only to find out the human doctor has a more plausible explanation.
Google's AI Overviews have already faced criticism for providing inaccurate health information, and ChatGPT is no exception. Gombar argues that AI companies must be more transparent about how often their answers are hallucinated and clearly flag information that is poorly grounded in evidence or fabricated.
The issue of data privacy is also a major concern. While OpenAI and Anthropic claim to follow HIPAA guidelines, Alexander Tsiaras, founder of StoryMD, questions the true motivations behind collecting sensitive patient data. "It's not just about protection from hacking; it's about what they do with that data after." Tsiaras warns that trust will be hard to regain if companies prioritize profit over data security.
Furthermore, AI chatbots can reinforce delusions and harmful thought patterns in people with mental illness, potentially triggering crises such as psychosis or even suicide. Andrew Crawford, senior counsel for privacy and data at the Center for Democracy and Technology, emphasizes the need for AIs to prioritize profit over personalization and data protection.
The problem is not just about individual companies; it's a systemic issue that threatens the entire healthcare landscape. With the primary care workforce shrinking in the US, physicians are facing an increasingly precarious situation. "If the whole world is moving away from going to physicians first, then physicians will be utilized more as expert second opinions," says Gombar.
Nasim Afsar, a physician and former chief health officer at Oracle, views ChatGPT Health as an early step toward what she calls intelligent health, but cautions that it's far from a complete solution. "A.I. can now explain data and prepare patients for visits, but transformation happens when intelligence drives prevention, coordinated action, and measurable health outcomes."
As the AI health advice landscape continues to evolve, experts urge caution and clarity. The promise of democratized information must be balanced with a deep understanding of the risks and limitations involved. Until then, doctors will remain vigilant, questioning the accuracy and reliability of AI-driven health advice.
The rush to deploy AI in healthcare has sparked concerns about accuracy and trust. With over 230 million people asking health-related questions on ChatGPT each week, the potential benefits of democratized information seem clear-cut. However, experts warn that this trend is fraught with risks.
"What I'm worried about as a clinician is that these general-purpose LLMs are still prone to hallucinations and erroneous information," says Saurabh Gombar, a clinical instructor at Stanford Health Care and co-founder of Atropos Health. "It's one thing if you ask for a spaghetti recipe with an unusual ingredient, but it's another story when it comes to fundamental health advice."
Doctors are concerned that AI-powered chatbots will perpetuate misinformation, erode trust between patients and healthcare providers, and even lead to adverse outcomes. For instance, a patient might be convinced they have a rare condition after chatting with AI, only to find out the human doctor has a more plausible explanation.
Google's AI Overviews have already faced criticism for providing inaccurate health information, and ChatGPT is no exception. Gombar argues that AI companies must be more transparent about how often their answers are hallucinated and clearly flag information that is poorly grounded in evidence or fabricated.
The issue of data privacy is also a major concern. While OpenAI and Anthropic claim to follow HIPAA guidelines, Alexander Tsiaras, founder of StoryMD, questions the true motivations behind collecting sensitive patient data. "It's not just about protection from hacking; it's about what they do with that data after." Tsiaras warns that trust will be hard to regain if companies prioritize profit over data security.
Furthermore, AI chatbots can reinforce delusions and harmful thought patterns in people with mental illness, potentially triggering crises such as psychosis or even suicide. Andrew Crawford, senior counsel for privacy and data at the Center for Democracy and Technology, emphasizes the need for AIs to prioritize profit over personalization and data protection.
The problem is not just about individual companies; it's a systemic issue that threatens the entire healthcare landscape. With the primary care workforce shrinking in the US, physicians are facing an increasingly precarious situation. "If the whole world is moving away from going to physicians first, then physicians will be utilized more as expert second opinions," says Gombar.
Nasim Afsar, a physician and former chief health officer at Oracle, views ChatGPT Health as an early step toward what she calls intelligent health, but cautions that it's far from a complete solution. "A.I. can now explain data and prepare patients for visits, but transformation happens when intelligence drives prevention, coordinated action, and measurable health outcomes."
As the AI health advice landscape continues to evolve, experts urge caution and clarity. The promise of democratized information must be balanced with a deep understanding of the risks and limitations involved. Until then, doctors will remain vigilant, questioning the accuracy and reliability of AI-driven health advice.
the issue with AI health advice is super concerning! I mean, on one hand it's awesome that people can get info quickly, but on the other hand, doctors are worried about misinformation being spread 
. And what really freaks me out is the thought of AI chatbots potentially triggering crises in people with mental illness 
. We gotta be careful here and make sure these tech companies prioritize data security and transparency over profits 
. I mean, what if we start relying too much on these chatbots for health advice? It's scary to think that someone might get sick and instead of going straight to the doctor they end up chatting with AI first because they "feel better" after talking to it 
. And don't even get me started on data privacy... I just don't trust companies collecting our personal info for whatever reason 
. We need more transparency and accountability from these tech giants before we start handing them the reins of our healthcare 
.
. I'm all for innovation, but we gotta be careful here 
ChatGPT is like my go-to health resource 24/7! 
But seriously, docs are saying some major concerns here... hallucinations and all that. What if AI gives me a legit diagnosis and then it turns out it was wrong? 
That would be a nightmare! 
At least Google is being transparent about their errors. 
Now I'm even more convinced that I need to stick with my human doc, Doc Patel 
. He's always got my back... or should I say, my health advice? 
And let's not even get started on data privacy... if companies can't be trusted to protect our sensitive info, what makes us think they'll do better in general? 
. Like, yeah we get it, you can give us info on like what to eat or stuff. But when it comes to actual healthcare, doc's expertise is still the best way to go 
. Those AI chatbots sound like they're just spewing out whatever the algorithm thinks is right, and that's not good enough for people's health 
... it's like companies are more worried about getting our info than actually keeping us safe. We need to be careful here, AI can be super useful, but we gotta make sure we're using it right 
. I've seen so many people getting misled by AI-generated health advice, and it's not just harmless misinformation - it can lead to real harm 
. But it needs to be done responsibly and with a deep understanding of its limitations 
. We can't just rely on expert second opinions or wishful thinking - we need concrete evidence and measurable outcomes 
.
.
. My cousin has been using one of these chatbots and now she thinks she has this crazy disease 
but when she went to see her actual doctor... nope! They were like "girl, you're fine" 
. I think we need to slow down on all this AI craziness for now 
. If we want to use AI in healthcare, we need to make sure it's done right and that patients are protected every step of the way.
.
. They might give you info that sounds legit but is just made up 
. The issue of data privacy is also a major red flag 
. Companies gotta make sure they're not collecting sensitive patient info and using it for profit 
. We can't just rush into something without thinking about the potential risks 
. Until then, doctors will keep being our go-to experts for real advice 
anyway i was reading this article and docs are literally saying the same thing as my ex bf - ai is prone to hallucinations and we need more transparency about its accuracy 
?
It's like, we need to be careful here and make sure these AI chatbots are being used responsibly 
. Anyway, I'll keep an eye on this development, but for now, I'm staying skeptical