For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.
So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.
Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.
I agree that we shouldn't get ahead of ourselves with the current technology, but what you said earlier applies to practically every industry and science. What you learn at the actual job is always far more up to date than what you learn in school, no matter if it's being a engineer, doctor or just a lowly programmer.
Yes but the difference is most engineers, and pretty much all lowly programmers are unlicensed. AI or some non-human accessible in the countryside so you don't have to order questionable "fish" antibiotics or "cat" anti-parasitic would be a nice step up from the current gatekeeping of medicine from people with limited access.
The only significant ways I'm aware of that people get needed rx medications outside of physicians and mid-level practitioners is to leave the country or use "vet"/"animal" drugs. What are the simpler other alternatives currently available?
The medical cartel loves to cloak their policies under the auspices of safety. In the end you'll find their policies magically result in massive profits for bureaucrats and chokepoints that constrain the supply of gatekeepers. This is not accidental.
Incentives. Plenty of people would go into rural medicine if it wasn't paying crap and dealing with nothing but elderly Medicare patients because of the lack of health insurance for people just marginally above the Medicaid line. Of course this means there's basically no support currently for the hospitals and more than half are being bought up by private equity anyways and shipping it all to the smallest bottom line.
What if the AI is trained on board exams and other high signal testing/examination materials? Surely it will become superhuman in its medical abilities?
Ive never experienced anything in my care with doctors that I couldn’t understand with a days worth of research into things like UpToDate. It isn’t complicated. It is largely memorization and application of an algorithm which is just borderline useless for complex conditions that are emerging more modernly.
Have you ever had anything besides a bad cold? Tell me you understand every acronym in this article, and the ability to explain it succinctly to a patient much less to be able to hand off a case to "another" physician https://www.nejm.org/doi/full/10.1056/NEJMoa2206714
Doctors and nurses have saved me many times from some very close calls because of decades of experience, training and intuition. That is of course not to mention the friend who beat a deep brain tumor on their brain stem that everyone else told them was inoperable, and now are in medical school themselves for neurology. No LLM is going to pull that out of itself, possibly ever, and certainly not GPT-4 (no one else had ever had the surgery done before, it was novel).
How do you know that you actually understand a topic as well as a doctor? How do you verify that? It's not unusual for people to think they comprehend a topic at expert level, when in fact they do not. The correlation between confidence and understanding is not a reliable measure. That's why doctors are trained by more expert colleagues who can judge their true understanding, have to take exams, etc.
I think most people with complicated chronic diseases for more than a few years end up knowing more than most doctors about their condition and related conditions. Doctors are more breadth than depth. But the problem is that depth is what is absolutely necessary in these situations. But there is a lack of that among specialists too, or at least they are not willing to go outside of insurance mandated covered procedures and testing and it creates a really useless and frustrating scenario for the patients.
Doesn't matter how much GI doctors know when all they do is scope you. Sure doc scoping is sure going to help people with atypical intolerances, IBS, and any number of modern chronic conditions for which treatments are inadequate. They have to do better!
For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.
So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.
Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.