While the researchers claim this important first step could change the medical field, their paper leaves a lot undiscussed.
A team of medical researchers from the Icahn School of Medicine at Mount Sinai recently conducted a study on artificial intelligence (AI) chatbots wherein they determined that “generative large language models are autonomous practitioners of evidence-based medicine.”
According to pre-print research published on arXiv, the Mount Sinai team tested various off-the-shelf consumer-facing large language models (LLMs), including both ChatGPT 3.5 and 4, and Gemini Pro, as well as open-source models LLaMA v2 and Mixtral-8x7B.
The models were given prompts engineered with information such as “you are a medical professor” and then asked to follow evidence-based medical protocols to suggest the proper course of treatment for a series of test cases.
