Generative AI for Healthcare

The script begins with Jack VanHorn from the University of Virginia welcoming the audience to the 2023-2024 Foundations of Biomedical Data Science seminar series. He highlights the support from various institutions and emphasizes the focus on data science methodologies, AI, deep learning, and statistical modeling in biomedical and health sciences. The theme for the year is on building partnerships for generative AI training, and the excitement around the program and its potential to bring together AI developers, biomedical researchers, and educators is palpable. The speaker, Dr. Roxanna Danu from Stanford University, is introduced, and her background in biomedical data science, dermatology, and AI is detailed.

Dr. Roxanna Danu discusses the broken healthcare system, using a relatable beach scenario to illustrate the challenges in accessing dermatological care. She emphasizes the potential of AI to streamline healthcare, though she clarifies the tension between AI replacing versus aiding physicians. Notably, she mentions the rapid advancements in AI, particularly in dermatology, and the need for AI to enhance primary care rather than replace specialist care. The direct-to-consumer aspect of AI in healthcare is also touched upon, highlighting the transformative impact of AI on the practice of healthcare.

The discussion shifts to the impact of generative AI on healthcare, with a focus on computer vision and large language models. Dr. Danu highlights the exciting possibilities, such as drug discovery and synthetic data generation, while also acknowledging the need for extensive research and evaluation frameworks to understand the efficacy of these models. She expresses surprise at the speed of AI integration into healthcare, given the industry's typically slow adoption of technology. The conversation then moves to specific stories of generative AI in healthcare, including computer vision research and the use of large language models by physicians.

Dr. Danu delves into the application of computer vision AI tools in healthcare, noting the prevalence of black box models that use deep neural networks. She discusses the importance of explainable AI to understand the factors influencing the algorithm's decision-making. A novel methodology using generative AI to create counterfactuals is introduced, aiming to reveal the features that AI models use to make assessments. The study's findings reveal both promising and concerning factors, underscoring the need for careful model development and the potential for spurious correlations.

The use of synthetic data in training AI models is explored, with a focus on addressing the lack of diverse skin tone representation in dermatology AI research. Dr. Danu presents a study that examines the impact of synthetic images on model performance, cautioning against the potential for continued bias if synthetic images are used imbalanced. She emphasizes the need for more real images across diverse skin tones to build fair AI models, noting that synthetic images alone cannot solve the problem of bias.

A survey study on physician use of large language models (LLMs) in clinical care is discussed, revealing that a significant number of dermatologists use LLMs for clinical decision-making. Dr. Danu expresses surprise at the high usage and daily reliance on LLMs by physicians. The tasks for which LLMs are used and the perceived accuracy of their responses are also covered. The discussion highlights the gap in physicians' understanding of LLMs, their biases, and the potential risks associated with their use in clinical practice.

Dr. Danu describes a red teaming event held at Stanford to identify vulnerabilities in the use of large language models in healthcare. The interdisciplinary nature of the event and the methodology used to assess model responses are detailed. The findings reveal a significant percentage of inappropriate responses, highlighting issues of safety, privacy, factual inaccuracies, and biases. The discussion underscores the importance of understanding and addressing these vulnerabilities before widespread implementation of AI models in healthcare.

Dr. Danu concludes her talk by acknowledging the reality of AI's presence in medicine and its potential to improve models through auditing and synthetic image training. She also warns of the potential for inaccuracies and harm due to the use of large language models. The talk ends with a Q&A session where Dr. Danu addresses concerns about the 'WebMD effect', the dangers of self-diagnosis through AI, and the need for both patients and physicians to be aware of confirmation and automation biases. The conversation touches on the rapid adoption of AI in healthcare and the potential risks it poses, emphasizing the need for a nuanced understanding of its benefits and drawbacks.

The session concludes with closing remarks from the host, expressing gratitude to Dr. Roxanna Danu for her insightful presentation. The importance of the topic is reiterated, and the audience is wished a wonderful weekend before the session ends.