Transcript

Harlan Krumholz: Welcome to Health & Veritas. We have a special bonus podcast today. Our guest is Dr. Robert Wachter who’s here to talk about his new book, A Giant Leap: How AI is Transforming Healthcare and What That Means for Our Future. It’s being published today, and we’re dropping this podcast. We recorded it a little while ago, but we were holding it for this special day. I’m Harlan Krumholz.

Howard Forman: I’m Howie Forman. We’re physicians and professors at Yale University trying to get close to the truth about health and healthcare, and we are excited about this special episode.

Harlan Krumholz: All right, let’s get to the interview. Today on Health & Veritas, we’re joined by Bob Wachter, a physician who’s repeatedly helped medicine make sense of the moments of major change. He’s also a friend and someone I deeply admire. We’ve known each other for almost 40 years, almost 40 years, and I’ve had the great pleasure of seeing his fantastic career take off. He’s a true connector. He really understands what’s going on in medicine. He’s, by the way, chair of medicine at UCSF. Widely regarded as the father of the hospitalist movement and long-standing leader in patient safety, quality and digital health. A decade ago, he wrote The Digital Doctor, a pretty clear-eyed account of how health IT promised transformation but often delivered frustration and harm. Now he’s back with a new book, A Giant Leap, which asks whether artificial intelligence represents another cycle of hype or something fundamentally different and whether healthcare is finally ready to get it right.

I found this book so interesting. He drew on, I don’t know, more than a hundred interviews and his own experience leading a major academic medical department. He really cuts through the hype. He’s much of a storyteller. He’s got some of the most amazing people giving him quotes for the book. I don’t know how you did that, Bob. You got all these folks to read it and endorse it. I wish you would’ve asked me. I would have loved to have done the same. It’s just a fantastic read, and I think it brings things together in the best possible way at the best possible moment, but I’m going to give you a chance to tell us. There have been several turning points in modern medicine. Explain to us why this book, why now. There is a lot out there. What makes your book different?

Robert Wachter: I think the moment is unlike anything I’ve seen, and I’m pretty old, so I’ve been around for 40 years in healthcare. I think it really is the combination of this dazzling new technology that has capabilities that no technology ever had, needing a healthcare system in desperate need of change. You start with the dazzling technology. It’s amazing, but when I sat in an Uber three years ago, I didn’t say, “Boy, do I need to be in a Waymo and I need to be a driverless car.” When I did a Google search three years ago, I didn’t say, “I can imagine something much better than this.” We already had systems that were working pretty well, and I think AI has expanded the capabilities. Maybe you guys know, but I don’t know anybody—patient, doctor, nurse, leader—who says, “The healthcare system’s perfect. Leave it alone. There’s no need to improve it.” It is so broken in so many fundamental ways, so it’s the combination of just a remarkable new set of technological capabilities meeting a system that really has to transform itself, that sort of creates the moment.

Why me? I got over my imposter syndrome on technology a long time ago. I really don’t know how this stuff works, but I actually am convinced that the people that have really deep understanding of what’s under the hood probably wouldn’t write the correct book because the book is a little bit less and the moment is a little bit less about the nuts and bolts of the technology and more about what happens when a technology like this, a thinking technology, technology that in many ways can replicate what doctors and nurses do and think. What happens when it meets a system like healthcare? What happens really has a lot to do with politics and policy and ethics and values and culture and history, and that’s stuff I know a lot about and I’ve thought a lot about and I’m an insider.

I turned out to be a good person to try to understand it, and as you mentioned, I did 110 interviews because once I decided to do it, it was clear to me that I needed to talk to people from all parts of the industry, from healthcare, from technology, to help educate me to find the major themes. That was just immense fun, just trying to sort out what does this really mean and where is it going.

Howard Forman: Healthcare does seem just so sclerotic to me. I’ve been in this just a little few years shorter than you, but pretty much many decades now. It just seems sclerotic. It’s hard to change healthcare. At the same time, every time we do introduce something, the unintended consequences are sort of disconcerting. The ACA information technology investments and things have had a lot of side effects that haven’t been great, and just last month before the book came out, Stat News in their predictions for the year talked about one prediction, which is that hospitals will figure out how to use AI to increase their revenue by three to 10% or some number like that. I worry, is the money being allocated to the right areas of healthcare right now, or are we going to see this also going to revenue enhancement and maybe some productivity enhancement, but not that the dramatic type of structural changes that you are talking about and hoping for?

Robert Wachter: Well, I guess first of all, I am capable of writing a pessimistic downer book because I wrote one 10 years ago. I came into this almost with equipoise. I mean, I didn’t know how I’d feel at the end. At the end, I actually came out of it quite optimistic, and the optimism, again, is in part because we’re starting from such a low floor. I mean, there’s so much room for improvement, and the tools really do address many of the fundamental problems, but they don’t change the payment system. The power of incumbency in healthcare for whether you’re talking about big health systems like yours and mine or doctors or nurses, the power of incumbency is really quite striking and the stakes are high, so people are a little risk-averse, and it’s a tight regulatory environment, so it doesn’t give the opportunity for entrepreneurial disruption that you see in lots of other industries.

The things I’m quite confident about are that I think care will get better, I think safer, I think more convenient, I think more satisfying. I think clinicians will be happier in their work. The thing I’m not at all confident about is, will this save money? How could it save money? It could save money in two main ways. I think if you look at the drivers of cost—you know this better than I do—it’s mostly labor, so it could save money if right now we have to hire boatloads of nurse practitioners or PAs to do certain work that maybe we wouldn’t. If you go to the billing department, there are a thousand people working there or you could just tick off all of people writing prior authorization requests and sticking them in fax machines. That could save money for labor displacement or labor changes. Second thing that theoretically could save money, and I’d be interested in your take on this, is as we get to real live, useful, computerized decision support, it could save money by saying, “Doctor, did you know that the best medicine for this thing is this one?” and it turns out to be one-tenth the price of that other thing and works just as well.

It could also, though, and this would depend on the values that are embedded in the AI, say, “You know what? We make more money if we do this procedure or we make more money if we give this drug and therefore that is going to be the one we recommend.” Talking to Harlan about this, I mean, just the idea of where does transparency fit in because it’s not like there is one right answer to any clinical question. There are probably a hundred right answers, and the incentive system, the ambient environment, is in large part going to determine, does it save money or does it not? I could see that going both ways.

Harlan Krumholz: Well, this leads me to, I think, a question a lot of people are asking and Howie and I have discussed before, to what degree do you think AI is going to actually replace doctors? Not in theory, but in a day-to-day clinical practice over the next decade, because it’s pretty easy to think of some use cases, certainly of tasks that doctors are doing. I’ll just say for me, I think in five years, why should any cardiologist be reading an ECG? AI is going to be doing a much better job and being able to extend far beyond what any cardiologist could do.

Robert Wachter: Sure.

Harlan Krumholz: What do you see as the future of this? I mean, do you see this as… people are afraid of this word “replacing” doctors, but conceptually, there are a lot of things that doctors do, a lot of pattern recognition that doctors do, that could be largely replaced by AI.

Robert Wachter: No, I think it’s a fundamental question. My last chapter in the book is kind of that question because it comes up all the time, and I have a daughter and a son-in-law who are pulmonary fellows, so I’ve got a vested interest in—

Harlan Krumholz: I was going to say, my daughter’s a medical student and the medical students are very anxious about—

Robert Wachter: They should be. I mean, it’s understandable. I ended up, whether this is a hopeful note or not a hopeful note kind of depends on your perspective, because you might say, “Who cares if it replaces it? If I’m a patient or I’m a payer, if it replaces doctors and the outcomes are as good or better and it’s more convenient and it’s less expensive, then the world’s a better place.” Obviously, if you’re a doctor or if you are someone who believes that doctors add something fundamental that can’t be replaced by AI, then that’s a scary scenario.

I have come to believe that for the foreseeable future, it will not replace doctors or nurses in part because of the extraordinary number of unmet needs in today’s system and the fact that doctors and nurses actually are begging for help from AI to take some of these tasks off their plate. Now, I mean, Howie’s here from radiology and I think the example is perfect. As you probably know, in 2016, Geoff Hinton, the founding father of generative AI, winner of Nobel Prize last year, gave a now famous or infamous speech where he said, “We should stop training radiologists today (2016) because there’s just no question that within five years they’re all toast and we don’t need any more than we already have.” What happened was, radiology residency applications tanked, which of course was amazing because it has always been the most popular specialty. That lasted about two or three years, and then the students woke up one day and saw that radiology incomes had gone up, that there were help wanted ads for radiologists everywhere, and they went to the radiology department and the radiologists looked busy as hell and all of a sudden the applications came back.

Radiology is almost without question the easiest field for AI to replace, maybe with the exception of pathology. They are about, in some ways, the recognition of a pattern of digital dots against the template of what do those things mean in terms of disease state. Now here we are 10 years after Hinton’s prediction and both at Yale and at UCSF, we can’t hire radiologists fast enough and radiologists are begging for help. What does that mean? It means it’s harder to replace doctors than people thought. It means that what we do has a lot of complexity built into it that is sort of beyond just pattern recognition. It means that the guilds are pretty powerful and the regulatory environment and the payment environment and the patient safety department and the malpractice lawyers will expect that there is a doctor signing off on everything, at least for the foreseeable future—although that’ll be tested—and the reason I think this is a good thing is not only for my daughter and yours, because I’d like them to have jobs, but I was an old poli sci major in college, and I look at this partly through a political lens.

If this was going to replace doctors or nurses, then you would see massive pushback, and as I’ve told lots of tech leaders over the years, doctors are better lobbyists than taxicab drivers, and when we push back, we will not say, “I’m worried about you taking my job.” We will say, “You’re going to kill somebody.” The first time someone is killed by AI, you believe me, it will be a headline, it will be a cause célèbre, it’ll be a lawsuit. I think that doctors are going to be safe for a while. I don’t know how long that is. I can’t imagine it’s 100 years, but I can’t imagine we’re gone in 10, and as a political matter, I actually think it’s quite important for AI’s growth and acceptance in medicine, because I think you’re going to see most doctors and nurses welcome it rather than fight it.

Harlan Krumholz: Just my quick thing on top of that is, just to say my view of this is, if you think that your job is going to be the same job that it has always been, then you’re wrong.

Robert Wachter: Yeah. You are wrong.

Harlan Krumholz: If doctors as a profession evolved and continue to be the advocates for patients, it’s a matter of how the profession can recognize the moment and figure out what is its next step. It’s sort of the same thing as when there were shaman and if they didn’t evolve into a profession that they were going to be left behind, not that we still don’t have shaman, but that’s my view is that this is about—

Robert Wachter: I couldn’t agree more. I think these are huge issues for the training environment, and it has to evolve. I couldn’t agree more. I mean, I was on the wards last week. Normally when I’m on the wards, I’m a generalist and two or three times in the morning rounds I would really want to get a curbside consult. It’s not a big enough deal that I’m going to call an ID consult or call a cardiology consult, but I have a question I just needed to answer. I’m hoping I run into Harlan in the cafeteria because I’ve got a tough cards question.

Harlan Krumholz: For people listening, curbside’s kind of this lingo we use when we say, “Can I find someone who’s got specialty in this so I can ask them a quick question in the hallway?”

Howard Forman: As opposed to a full consult, right?

Harlan Krumholz: As opposed to having somebody come and see the patient. I’ve just got a quick Bquestion, so we would call it a curbside consult.

Robert Wachter: I’d ask two or three of those. I probably asked 10 of them a day, but I now ask them of either GPT or Gemini or OpenEvidence. I’m using it. I use it as prescribing.

Harlan Krumholz: That’s right.

Robert Wachter: I use it for chart summarization. It’s not perfect, but as Biden liked to say, “Don’t compare me to the almighty, compare me to the alternative.” One out of five patients’ charts are longer than Moby-Dick. The idea that I’m going to be able to read a 600-page chart and summarize it in the two minutes I have to do it is ludicrous. It’s already, I think, becoming a thing and in some ways organic ways. My health system didn’t say “use OpenEvidence,” but everybody’s using it. I don’t want AI making a high-stakes decision like starting chemo or sending somewhere to the OR. I think there’s plenty for doctors to do over a 50- or 30-year time horizon. Who knows? I think things change a lot.

Howard Forman: You do talk a lot about the consumer side of things as well, not just the health system side. I’m curious to hear some of your thoughts or some of the anecdotes you even talk about, about how patients interact with AI now, how they might in the future, and how that might change what the sheer volume of care that our health system is counting on for now and that might shift outside of the system to AI products. I mean, again, one Stat prediction was 50% of all patients will have an encounter with AI before they see a physician.

Robert Wachter: Unquestionably. It’s already happening. Patients are using this all the time, and you might say, “Well, what’s the big change? They’ve always been Googling their symptoms,” but these tools are fundamentally different. I mean, try to put in to Google, “I am a 68-year-old man and I woke up this morning short of breath and my left knee hurts and I’ve got a fever.” And, you know, it sputters—and so will UpToDate, by the way. These tools have not been capable of ingesting the complexity of medicine, and the new ones do. That creates a remarkable set of capabilities. This is going to move very fast and two things happened the week we’re taping this that I think are really important moments to note.

One is that OpenAI and GPT just announced GPT for Health, which is rather than you just putting in a query into GPT, they created a space that at least they claim is going to be even more safe and more encrypted for you to load in your medical record, so that assuming you’ve done that and it works, you now put in “…and now today I have a fever.” It knows that you have rheumatoid arthritis and you’re on Humira. It’s in the system, and therefore its ability, hopefully, to give you better answers is growing. The second thing that happened that’s fascinating is the state of Utah just approved that AI can refill your prescriptions without a doctor. Beginning to encroach into the world of “Can it begin doing doctor-ish things?” and “Will the regulatory environment allow it to do so?” That’s going to happen more and more. Patients are going to have better information in a very short period of time, rather than you having to port your data out of Epic or Cerner or Oracle into GPT, it’ll be embedded in your electronic health record. It’ll be embedded in MyChart.

That of course is miraculously important because part of the problem here is for the last 10 years, patients have had complete access to their medical record. They can read my note, they can read your note, but it’s been in some ways almost a false promise of democratization because the patient gets a thing that says their EKG is abnormal and their magnesium is low. It’s like, what the hell does that mean? That seems scary. Now you’ll have AI tools embedded in these systems that say, “Here’s what that means” but not in general. “Here’s what it means for you,” knowing your history. This is exciting, I think. It’s a little scary because obviously you’re now creating systems where patients are going to get a lot of medical advice from digital tools. That’s scary because they can be wrong. It’s scary because there can be corporate interests that may influence the results, but I think it’s net good.

I guess the final thing is, I don’t know what it’s like in New Haven, but I’ll tell you in San Francisco, try to find a primary care doctor. Virtually impossible. I’m a general internist. I think primary care doctors are amazing and do a lot of amazing things. Do you need to see a primary care doctor to manage your hypertension? To manage your lipids? To say you’re obese, and I’m going to prescribe a GLP? I don’t think so. I mean, I think that there’s a fair amount of what we say requires the legacy medical system today that could be done through effective tools, and I think we’re going to move there pretty quickly.

Harlan Krumholz: Yeah. I’m going to give a quick plug again for your book. It’s kind of a public service announcement in the middle of this. I think that a lot of what you’re hearing from Bob reflects a lot of the wisdom they send in the book, even the Moby-Dick comment, Bob quotes a 2024 study that says one in five of the charts are longer than Moby-Dick. I mean, you’re going to see this kind of stuff if you read the book. It’s enjoyable, and it elaborates on a lot of the things that he’s talking about, so I just wanted to give you that plug. This thing about the patients though, the word people talk about when you know about this, it’s inevitable that people are now going to be in a position where they’re going to walk in with more knowledge. Howie, I don’t know if you’ve had a chance to see this thing that Bob’s talking about with the ChatGPT Health.

Howard Forman: I saw it in the news. I haven’t seen it, though.

Harlan Krumholz: Essentially what they’re doing is, there’s this company, B.Well, that’s just like Hugo. I had started this company, Hugo Health, which I no longer have, but it’s sort of like trying to help people get their personal health record together. Google Health had tried to do this a long time ago, but now with the AI, you can layer it on top of your own personal health record that’s pulling in your data in almost real time. Now these agents, which are really basically the foundational AI models with specific focused tasks associated with them, are going to help people in ways that are going to be extraordinary, and I do think that, in the same way you said, I’m looking for a curbside for someone in the cafeteria, but I think that this is what patients are going to do when they come in and see us.

They used to be equipped by the Google response, but now they’re going to be equipped with something much more sophisticated. We’re in for a wild time with regard to the balance of power that exists within because the power was always about the asymmetry of information and the asymmetry of knowledge. The thing that I think is so—and I’m interested in your reflection on this—but what’s so interesting to me and exciting is that you can ask the AI to speak to you at the level of health literacy that you have. It’s like saying, talk to me like it’s a sixth-grade level, talk to me at a fourth-grade level. No, talk to me at a graduate school level. People will get something that can take as much time as it takes to kind of explain something that is really kind of beyond what we’ve been able to do with humans.

Robert Wachter: All of that, I mean, incredibly exciting and where the risk… I mean, everything you said is true and I think in some ways almost hard to see the downside, although there will be doctors, of course, who object to the patient coming in. First of all, it may take a lot of time to go through all the stuff that they’ve printed out, but where I think the rubber really meets the road is this thing we were talking about previously, which is when they don’t go to see the doctor. When they’re using this essentially to be their doctor, and there I have some worries. I was asked by The Washington Post a couple months ago to look at 12 patient queries into GPT and look at the answers that GPT gave. I gave them an average rating of six on a scale of one through 10. 10 being perfect, one being awful. The results were average. Some of them were great, and one of them actually was far better than what I ever said. It knew something about the nuances of liver disease and a new guideline that I wasn’t aware of, but a couple of them were awful and quite scary.

A patient who I think had testicular cancer basically asked about ivermectin, and GPT said, “Oh, there are some rat studies that show it works and you should go out and try to find a doctor who will give it to you.” This is a cancer that’s 100% curable with appropriate therapy. To the extent that it’s democratizing things, but it’s giving you bad information, that’s scary, and I think the other thing that’s a little bit scary or at least has to get better is, I didn’t recognize until I was writing the book, the lens that an expert looks at the world through and the lens that a novice looks at the world through are very different.

We as experts can go into GPT and say, “I’ve got an 85-year-old patient with lymphomatoid granulomatosis who comes with a fever and a white count. What do you think’s going on?” A patient has no ability to do that, to synthesize the facts into the right package and then give it to the tool. Then the results come out, and I look at some of them and I say, “That’s great” and then others, “No, that doesn’t make any sense.” A novice has no ability to do that. To the extent that GPT right now is, “I’m going to put in a bunch of symptoms and it’s going to give me an answer,” and a novice—and this is not to denigrate patients, they’re not experts in this—will have limited ability to interpret it.

I think there’s going to be some mischief. I think there’s going to be some bad outcomes and bad results that come out of this. It doesn’t mean that’s going to be true forever. I just think the kind of tool that a patient needs, I don’t know is fully served by a generic chatbot kind of interface. I think it’s going to be much more interactive. I think it’s going to be, patient says, “I woke up this morning and my belly hurts,” and GPT, now knowing your past history because it’s in there, says, “Oh, okay. Do you have a fever? Are you nauseated?” and then you answer those. It will be much more replicating what a doctor does with a patient with a symptom. Right now, the tools don’t really do that very well, so I think they have to improve.

Howard Forman: You mentioned earlier that you yourself will look at, I think you said ChatGPT, Gemini, UpToDate, and OpenEvidence.

Robert Wachter: Well, I won’t look at UpToDate anymore, which is remarkable because UpToDate did to textbooks what GPT and OpenEvidence are doing to UpToDate. UpToDate doesn’t yet have the capability to do what we’re talking about. They have announced a month or two ago that they’re going to, and they better, because right now OpenEvidence is eating their lunch.

Howard Forman: That’s my question. To me, OpenEvidence is game-changer. I mean, as a radiologist even, there are times that I’m looking at a constellation of findings in a particular patient, and I just believe that this fits something, and I can use OpenEvidence, and it is game-changing for me.

Robert Wachter: Wow.

Howard Forman: Are you seeing that OpenEvidence is so far ahead of the others? Why are you still looking at Gemini and ChatGPT in medical settings? What are the relative advantages?

Robert Wachter: I’d say, frankly, the reason I will use ChatGPT on rounds is that I can talk to it and it can talk back to me. When OpenEvidence develops that capability, because sometimes I’ll be on rounds with my team and part of this is show and tell. I will pick up the phone and say, “Here’s this patient with blah, blah, blah. What do you think is going on?” We’ll do that five times during rounds, and right now you can’t do that with OpenEvidence. That’s probably the main reason, and it’s good enough, but if I really want, if it’s a high-stakes question, and particularly if I want to see the supported literature behind a recommendation, I will use OpenEvidence. I think the interesting question, UpToDate was a little bit asleep at the switch. Didn’t move quickly enough into the AI world. When I put in a complex query into UpToDate, a real patient with a whole bunch of things going on, UpToDate sputters. They’re building that capability, and when they do, there’s going to be a very interesting battle. In some ways, this will be studyable, hasn’t been studied yet.

The question will be, what is the best source of essentially ground truth? Is it what OpenEvidence does, which is basically troll around through the medical literature and guidelines and give you—the curation is basically in the materials that it’s looking at, but other than that it’s doing sort of a typical large language model search. Is it UpToDate where the literature that it will be going through is the curated chapters that have been written by experts who have the ability to look at study A and say, “Oh, that’s a crummy study even though it’s published in a prominent journal. It doesn’t change my thinking. Here’s study B that really does, and that’s the one I’m going to emphasize in what I write.” Is it Gemini or GPT, which are really trolling around the entire internet, but somehow coming up with answers that are generally pretty good?

I think that’s an open question. When UpToDate announced two or three months ago, “We’re getting into the AI game,” the quotes that they gave for the PR were, we will be looking through a curated, expert-driven database. I think that’s an open question, what gives you better answers? I’d say if it gives you better answers, people will toggle from OpenEvidence back to UpToDate, but when you ask my residents now what they use, they all use OpenEvidence, and that happened in three months. I mean, it was just, Boom. Amazing.

Harlan Krumholz: Just for context for people listening, it’s just so interesting the way in which the technology’s moving so quickly. UpToDate, for those not familiar, is a digital textbook in which what they did was instead of having somebody put out a textbook every few years, they were updating in real time your access to a digital textbook. That was, as Bob said, somebody was actually writing a chapter and continued to update it maybe every few months and if a new paper came out, they could add a few sentences or a paragraph to that chapter very quickly. Whereas OpenEvidence is a kind of a ChatGPT for doctors where people interact with it, much like they interact with ChatGPT, but it’s honed to answer medical questions, and it’s pretty good. My disclosure is, I’m actually an advisor to OpenEvidence, so I just want to disclose that.

Robert Wachter: Yeah, well, you’re doing a good job, I think. You’re doing a good job, (a) because it’s useful and (b) they’re now valued at $12 billion.

Harlan Krumholz: I’ve done almost nothing for them. It’s been remarkable how in such a short period of time—of course, the founder is a guy who, this is his second time around, he’d done something in finance for it. It’s really a remarkable story in OpenEvidence. Bob, I—

Robert Wachter: Harlan, I interviewed Daniel Nadler, the founder [of OpenEvidence], and one of the things, the points he made, was that the tool is so good, not just because of the data that we synthesize, but because their process of reviewing the answers, which they have to do to continuously tune the data, is really quite robust and because the tool is only available to licensed professionals. The feedback they’re getting in real time from doctors is something that of course GPT and at this point UpToDate doesn’t really have, and he makes it very clear that that’s a very important part of the system.

Harlan Krumholz: It’s a breathtaking performance. I mean the thing is, and as evidenced by maybe 70% of doctors and all the trainees—in an instant, people could feel it was different, and I thought it was the same way like Google. There were a whole bunch of search engines, but immediately when you started using Google, you said, “Well, this one’s different. It really does better.” Right now OpenEvidence has, like you said, has eaten the lunch of everyone else, and it’ll be an interesting journey.

I wanted to just pivot to one thing that you and I had done together and what you think about the future of it, which is we were on the American Board of Internal Medicine together. This is the organization that certifies internists and puts together the tests and criteria tasks that are necessary to do for people to stay certified, which is a signal that you are keeping up with the profession and have some degree of competency. You actually chaired the American Board of Internal Medicine and are a leader in thinking about this. What do you think AI means for certification and board exams? If AI can already outperform lots of trainees on parts of medical knowledge testing, what are we certifying in the future, and how should AI be part of that?

Robert Wachter: Yeah. Well, I think that question is profound, but it in some ways just is a slice of a larger question, which is what does it mean to be a doctor in an era that in your pocket is a tool that’s smarter than you are? I think for the foreseeable future, and I don’t know how long this is, pulling too much off the curriculum of doctors and therefore pulling too much of the assessment, whether it’s in med school or residency or for a practicing doctor, off the plate because this tool that we all have access to can replicate what you do, I think that’s a big mistake. The reason it’s a big mistake is we’re in this era now where these tools are good enough to be useful without question, but not perfect, and in order to make the most useful, you got to know a fair amount of stuff.

You got to know the right thing to put in, and you might need to be able to know how to interpret the answers it gives. For the time being, I wouldn’t be pulling a whole lot of the fact-based education off either tests or assessments or off at the educational plate. I would pull none of what we do to try to teach people clinical reasoning off the plate. I think we’ve got to stay open to the possibility that over time we will need to say, “I don’t know how to read a map anymore, but who cares? I don’t need to read maps anymore.”

Harlan Krumholz: Have you changed the residency at all?

Robert Wachter: Well, I’d say the biggest changes are teaching people about AI. How do you use it? How do you use it responsibly, the interface between that and HIPAA and not to put in your patients’—

Harlan Krumholz: You’ve integrated that into your training program.

Robert Wachter: Yes, and we’re growing it. We’re already doing some of it, but we’re building a curriculum. I think that’s an easy part. I think the hard part is the question you’re asking, which is what gets pulled out. I chaired a conference for the Macy Foundation about a year ago of really, really brilliant AI experts, educators, and this question came up a lot, and there was complete consensus on pulling out one thing out of the curriculum. You want to guess what it was? It’s the Krebs cycle. Everybody agrees. There’s the biochemical energy pathway that we all learn quite painfully. Everybody agreed that, “Nah, you don’t really need to learn that,” but I think there’s really a lot of disagreement as you go beyond that, and I think the disagreement is healthy.

Over time, if there’s a set of skills we’re not using, sure, pull it out. Make med school shorter, don’t assess it for practicing doctors, but I think in the short term…. Then there’s this other risk of, as long as we are looking over the shoulder of the AI and we’re the final arbiter, you could easily see a death spiral here, which if we get stupider, then why do have the doctor in the loop because we’re not adding anything and you probably saw that study a year ago where AI plus the doctors were better than the doctors alone, but the AI was better by itself, and so the doctor’s intervention merely screwed things up, and that’ll just get worse if we de-skill people. I think we got to be pretty careful here.

Howard Forman: As we get to the end, I’m just curious if you ever think about surprises to the upside that you sort of have in the back of your mind, you’re not expecting them to happen in the short run, but could happen. Are there anything related to AI that you think about and you’re like, “This could happen this year and could shock people?”

Robert Wachter: I think there’s a lot of that, that is in the world of robotics, that I don’t know about the pace there, but when you talk to people in the surgical world, so many surgeries now are done through a digital video interface. They’re done through a robot. The robot is watching every stroke of your hands and every move. Where we are now is the AI is getting good enough to draw a map for you of where you should cut or where you should suture. There are AI tools obviously for colonoscopy that says “This is a lesion that looks scary. You should clip there,” but the analogy I use sometimes is, I’m something of a golfer. Several years ago my game was going off the rails, and I went to take a lesson and the guy filmed my swing and he said, “Whose swing do you like?” I said, “Tiger Woods.” He pulls up Tiger Woods next to me and when I’m here with the club at a certain angle, Tiger’s club is at a completely different angle. If I had to guess who’s right, I’m guessing he’s right and I’m wrong.

Whereas in surgery you can imagine—and these tools are already being developed—a world where it knows what the hand position of a surgeon should be at a given moment of taking out your gallbladder or doing cancer surgery and signals you that, trains you on it, coaches you on it, maybe assesses you on it. As Harlan and I know from being on the board, that’s not going to be a technological problem. That’s a political problem. It’s like “Surgeon, in order to remain board-certified, we need to have AI watch the next 10 surgeries and assess whether your technique is appropriate, give you feedback, but if it’s not, you’re no longer a board-certified surgeon.” That’s a political question. That’s not a technological question. The technology will be good enough to do that.

Harlan Krumholz: That reminds me of Gawande’s article about the surgeons should have coaches, that New Yorker piece.

Howard Forman: I’m going to anchor to the optimism here and also just thank you so much for joining us. To remind our listeners, the book is A Giant Leap: How AI is Transforming Healthcare and What That Means for Our Future. It’s out now, and we’re really looking forward to seeing what other people have to say about it, and thank you so much for joining us today.

Harlan Krumholz: Bob, I got to say, it’s just been such a pleasure to know you all these years and yeah, it’s wonderful to have you on.

Robert Wachter: Thank you. Thank you. It’s great. Howie, it’s the first time I’ve met you. It’s a joy. I’m a longtime admirer, Harlan. I knew when you were a resident that you were going to be a superstar and I’m not right that often, but I got this one right. It was really a great pleasure to reconnect. Congratulations on doing this great show.

Harlan Krumholz: Thank you.

Howard Forman: Thank you.

Harlan Krumholz: Well, that was terrific, Howie. I’m so glad we had Bob on.

Howard Forman: Oh my God, he’s so great. I had not met him before. This was fantastic.

Harlan Krumholz: Yeah, these are fun. You’ve been listening to Health & Veritas with Harlan Krumholz and Howie Forman.

Howard Forman: How did we do? To give us your feedback, to keep the conversation going, email us at he************@**le.edu or follow us on any of social media, including our special Instagram account.

Harlan Krumholz: Yeah and give us feedback. We love it. Post it. Email us, whatever helps people find us.

Howard Forman: Health & Veritas is produced with the Yale School of Management and the Yale School of Public Health. To learn about Yale SOM’s MBA for Executives program, visit som.yale.edu/emba. To learn about the School of Public Health’s Executive Master of Public Health program, visit sph.yale.edu/emph.

Harlan Krumholz: We always like to acknowledge our superstar undergrads. Today we had Tobias Liu, we have Gloria Beck, who also helps us, another tremendous individual. Tremendous producer Miranda Shafer, and I’m always working with the best in the business, Howie Forman.

Howard Forman: I appreciate you, Harlan, and we are going to be back again at our usual time on Thursdays with our next episode.

Harlan Krumholz: Yep. Thanks. Talk to you soon, Howie.

Howard Forman: Thanks, Harlan. Talk to you soon.