@dr-go
“ So when applied to the world of Medicine, Bots do not do so well:
ChatGPT, a popular artificial intelligence language-processing model, failed a gastroenterology self-assessment test several times in a recent study. Versions 3 and 4 of the chatbot scored only 65% and 62%, respectively, on the American College of Gastroenterology (ACG) Self-Assessment Test. The minimum passing grade is 70%.”
Chat GPT’s knowledge base consists of both professional based as well as non professional Information and misinformation (see my query below). To complicate matters, professional level information is in a constant state of flux. In my own experience of Board Certification and multiple Recertifications, the questions often remained the same but the answers changed each exam, on some occasions reverting back to answers that were correct 20 or 30 years before. Taking all this into consideration, and that Chat GPT was not designed as a medical device, it is not surprising that it had a mediocre score answering GI questions.
However, this is Is just the beginning. Imagine an advanced bot given access to only state of the art valid medical & scientific information with interpretation. Given the proper knowledge base as well as accurate input of a patients clinical data, I would predict that it would be William Osler and “House” on Steroids!
Query to ChatGPT:
Has your knowledge base been tested against standardized medical licensure examinations, specialty board examinations, or medical competency examinations?
As an AI language model, my responses are generated based on a mixture of licensed data, data created by human trainers, and publicly available data. I have not been directly trained on specific medical licensure examinations, specialty board examinations, or medical competency examinations. While I strive to provide accurate and up-to-date information, it's important to note that my responses may not always reflect the most current medical guidelines or practices. Therefore, it's always a good idea to consult a medical professional or trusted source for specific medical advice or information.