Is “progress” in medical education backfiring?

by Karan Chhabra

A good bit of fuss surrounds the evolution of medical education. The way we’re taught sometimes seems as susceptible to trends than the clothes we wear—one decade trumpets the “integrated,” systems-based curriculum, the next decade moves to the “flipped classroom,” and the cycle continues. Of course, these changes are based on educational theory, and their outcomes are often studied rigorously. But every now and then, when I rotate with a doctor trained the old-fashioned way, I wonder what has really been gained—and whether something has been lost—since the days they were taught.

I’ve seen how older physicians recollect tidbits from their preclinical years (decades ago) that my classmates and I can’t seem to remember past a few months. I wonder how much of this is the result of the way they were taught. Of course, there are many confounders here: our own intelligence versus those physicians’, our generation’s perpetual state of distraction, and perhaps the volume of material we’re expected to retain. The “old way” of teaching by discipline (anatomy, pathology, pharmacology, etc.) seems far less intuitive than the way we’re currently taught, by organ system (cardiovascular, gastrointestinal, etc.). To mentally switch from psoriasis to anti-arrhythmics in the same day seems like work.

But that may in fact be the secret to its success. I’m referring to research on “massed practice,” rapidly learning subjects en bloc, as opposed to “interleaved practice,” switching between learning tasks rapidly and revisiting the same topics day after day. A recent article (thanks Skeptical Scalpel) shows a relevant example:


Consider this study of thirty-eight surgical residents. They took a series of four short lessons in microsurgery: how to reattach tiny vessels… Half the docs completed all four lessons in a single day, which is the normal in-service schedule. The others completed the same four lessons but with a week’s interval between them.


…The difference in performance between the two groups was impressive. The residents who had taken all four sessions in a single day not only scored lower on all measures, but 16 percent of them damaged the rats’ vessels beyond repair and were unable to complete their surgeries.


Why is spaced practice more effective than massed practice? … Rapid fire practice leans on short-term memory. Durable learning, however, requires time for mental rehearsal and the other processes of consolidation. Hence, spaced practice works better. The increased effort required to retrieve the learning after a little forgetting has the effect of retriggering consolidation, further strengthening memory.


This effect isn’t limited to technical skills. The article also references cognitive tasks, like geometry problems. I wonder if it’d extend to preclinical medical education as well. I can personally relate to how a disease seen in one organ system, say in November, may literally never be seen again under an “integrated” curriculum.  It’s far easier to learn the pathogenesis of strokes and their treatment in the same week—but easier is not always better. Perhaps a trickier, thornier learning process is also sticker in the long run.


Karan is a student at Rutgers Robert Wood Johnson Medical School and Duke graduate who previously worked in strategic research for hospital executives.

Follow him on Twitter @KRChhabra or subscribe to the blog.


Saving money in healthcare, PCMH edition

by Allan Joseph

I wanted to quickly follow up on Tom’s excellent post from yesterday on patient-centered medical homes (PCMHs), which nicely outlined some conflicting results from recent research on the model. (Edited to complete the sentence.)

It really shouldn’t surprise us that PCMHs only saved money in the 10% of patients with the highest risk. Why? Take a look at this chart from the NIHCM, which is one of my favorites in all of health policy:

Distribution of Healthcare Spending

Notice that the top 10% of spenders (not the same as the top 10% of risk scores, but pretty close) account for just about two-thirds of healthcare spending. The vast majority of patients account for very little spending — there’s no savings to be had there. Healthcare spending is highly concentrated at the top.

Now let’s look at spending from another angle. According to the Robert Wood Johnson Foundationtwo-thirds of healthcare spending is on patients with multiple chronic conditions. That means at minimum, roughly one-third of healthcare spending in America is spent on those patients in the top 10% with multiple chronic conditions. (To get even deeper, at least 16% of spending is on patients in the top 10% with three or more chronic conditions.) Of course, that’s only a lower bound — I’d be surprised if the number wasn’t much higher.

So what does this have to do with PCMHs? Well, the core idea behind a PCMH is greatly increased care coordination. That’s precisely the type of intervention that will help sick patients who have multiple chronic diseases — or the very group that accounts for a huge portion of our healthcare spending. No wonder the investment in PCMHs paid off for the sickest patients. They’re the ones where all the money to be saved is.

Given that the vast majority of patients who might use PCMH services account for little health spending, we should expect spending money to build a broad PCMH structure to save money on net. Nor should we be surprised that there’s money to be saved by better coordinating the care of the sickest patients. That’s the whole idea of the “hotspotting” movement. That’s also why Tom was spot-on to focus on the idea of “risk-adjusted population health,” such as focusing care managers on the sickest patients or designing separate clinics that focus exclusively on high-risk patients.

Sometimes it’s worth stepping back and taking stock of our intuitions about what might reduce healthcare costs. If an intervention isn’t aimed at the sickest patients, it’s probably not going to save a lot of money. Don’t be surprised when it doesn’t.


Allan Joseph is a first year medical student at the Warren Alpert Medical School of Brown University, where he is pursuing an MD/MPP. You can follow him on Twitter @allanmjoseph.

PCMHs Don’t Work—Or Do They? Insights from Two Recent Studies (Of the Same Program)

by Tom Liu

A month ago, a JAMA study rocked the health wonk world by showing provocative evidence that Patient-Centered Medical Homes do not work. Evaluating 32 practices in the PA Chronic Care Initiative over a three-year period (2008-2011), the authors found that achieving NCQA PCMH recognition did not statistically reduce utilization or costs, and only improved one of 11 quality measures (nephropathy screening for diabetes). Aaron Carroll summarized the study and accompanying editorial over at The Incidental Economist. Mainstream media and health wonk blogs alike declared the death of the “touted medical homes model”.

That’s why I was surprised to read this headline last week:

Study: Medical homes cut costs for chronically ill members

The punch line: these two studies evaluated the same PA pilot project over the same time period (albeit with different practices and patient populations).


Medical Homes Work—But Only for High-Risk Patients

A close read of the studies reveals that their conclusions are not incongruous. Indeed, the more recent AJMC study found no significant decrease in utilization or costs across all patients, just as the JAMA study did. However, when the authors limited their analysis to the top 10% highest risk patients (defined by DxCG risk scores), they found significant decreases in inpatient utilization in all three program years, and significant decreases in costs in the first two.

We can’t discern if the JAMA study would’ve found the same significant effects if they did a sub-analysis of the highest risk patients. (Interestingly, they state in the Methods section, “we repeated our utilization and cost models among only patients with diabetes,” but the results of that analysis are nowhere to be found.)

These results underscore an insight that’s becoming increasingly clear: cost savings from care management are concentrated in the highest risk individuals.

But we can go one step further.


Cost Savings Came ONLY From High-Risk Patients

Among the 654 high-risk patients, the PCMH produced adjusted savings of $107 PMPM in the first year. That roughly comes out to an estimated $69,978 in overall savings. Almost all of this (and then some) came from an estimated 40 avoided hospitalizations (654 patients x 61 adjusted avoided hospitalizations / 1000 patients).

Among 6940 patients overall, the PCMH produced (statistically insignificant) adjusted savings of $10 PMPM in the first year—an estimated $69,400 in overall savings. Across this entire patient group there were an estimated 41-42 avoided hospitalizations.

In other words, this study didn’t just find that savings are concentrated among high-risk patients. Essentially all of the cost savings and avoided hospitalizations came from the top 10% high-risk patient cohort.

This doesn’t mean that other PCMH models couldn’t squeeze savings out of lower risk patients. It just means that this and many existing models haven’t found out how to.


How to Achieve “Risk-Targeted Population Health”?

That finding raises a broader question that these studies can’t answer: What prevented the hospitalizations among the high-risk patients, and more importantly, were those key interventions limited to only the high-risk patients?

For example, were the crucial interventions ones that were only used for high-risk patients, such as a dedicated care manager, targeted outreach messages, and special appointments for high-risk patients?

Or were they interventions that were indiscriminately used on all patients, such as standard patient education or practice-level infrastructure that all patients enjoyed (even if only high-risk patients “benefited” in terms of reduced hospitalizations)?

This question is important because all interventions (and the infrastructure to support them) have a cost. Developing patient registries, expanding EHR capabilities, maintaining after-hours access, and investing in new training all represent substantial financial investments. Less than $70,000 in savings among high-risk patients—while extremely meaningful and significant—would be wiped out by the $20,000 “practice support” and average $92,000 bonuses paid out to each PCP by the medical home program.

If all of the benefits and savings are coming from the high-risk patients, we need to devise ways to concentrate our costs as well. Implementing such “risk-targeted population health” may be the only way to make the financials work.

Some practices are trying to do this by using dedicated care managers for high-risk patients within their existing patient panels. Others are trying to create separate clinics entirely dedicated to high-risk patients—which would allow them to limit fixed costs to high-risk patients as well. In fact, the NHS in England announced this January they are piloting this latter approach, creating “complex care practices” of 400-500 high-risk patients drawn from surrounding practices.

Whichever approach proves most effective, one thing is clear from these two studies: we need to rethink our current PCMH model.


Tom is a health care researcher with experience in public health and blindness prevention. Follow him on Twitter at @tliu14 or check out his blog.

Walking uphill both ways

by Allan Joseph and Emily Flower

Sometimes, good data visualization just has a way of grabbing your attention. That’s what the Providence Plan, a data-focused organization here in Rhode Island, was able to do last week with this stark map (click to see a larger version):

Current Busing Policy Maps

Some background: These maps represent the distances high school students have to walk before qualifying for access to free transportation services through the Providence Public School District (PPSD). Or in layman’s terms, if you live in a green area on your high school’s map, you get a free bus pass. If you live in the red area, you’re on your own. A monthly pass costs $62 a month, and that’s the cheapest option.

Middle school students in PPSD receive transportation services if they live more than 1.5 miles from their school. When students transition to high school (when they can pick their school based on their interests), they get a free bus pass only if they live more than 3 miles away — measured as the crow flies, not in walking distance.

It may be cliché, but more than a handful of students do actually walk uphill, both ways, during their often hour-long commute to and from school each day. The city is built primarily on two hills; it’s difficult to get anywhere here without traversing a highway, waterway, or climbing up or down a hill — often all three in the same walk.

For many students, this daily trek is enough of a reason to not attend school. Between middle school and high school, the transportation requirement doubles. So does the chronic absenteeism rate. The chronic absenteeism rate jumps too. (Edited for miscalculation.) PPSD middle schools see a 26% chronic absenteeism rate, defined by a student missing 18 instructional days (roughly the entire month of November). PPSD high schools reported a 36% chronic absenteeism rate in the 2012-2013 school year. Obviously that’s not exclusively due to the transportation issue, but it doesn’t seem like a stretch to say that buses are a major factor — especially since absence rates are sky-high in the winter, even compared to neighboring districts.

But this isn’t just about chronic absenteeism, nor is it limited to education policy. Schools are some of the most important forces for health in the life of poor children — and with 83% of students in PPSD qualifying for subsidized lunch (family income below 185% of the federal poverty level), the majority of students in PPSD are poor or near-poor. In the short term, many students depend on subsidized lunch for food security. Their most familiar and important healthcare provider? The school nurse. Their easiest access to health resources? The school nurse, again. When they don’t go to school, they’re cut off from all of those resources. That’s not to mention that if they do decide to walk to school from nearly 3 miles away as the crow flies, they could be placing their health at risk, as frostbite, heatstroke, and violence are all real risks at various times of the year.

Long-term, there’s a relationship between educational attainment and health outcomes. Though the literature isn’t clear on how much of that is causal, we think it’s a fair bet that students who do better in school (by attending more often) will have a better chance at attaining better health. (Here’s some tentative evidence.)

But there’s some movement towards changing the policy. Recently-introduced House Bill 7012 aims to change the policy in order to bring PPSD’s policy in line with the other school districts in the state: a 2-mile walk zone. That’s far from ideal, but it’s certainly an important step: about 2,100 students would gain access to busing, and it stands to reason that those are the students who would most benefit from expanded busing, as they live the farthest away. Here’s what the new maps would look like (again, click to enlarge):

New Policy Maps

As you can see, that one mile covers quite a bit of the city. But H.B. 7012 isn’t an easy sell, not by a long shot. For one, the policy would cost the city $1.35 million at a time of strict belt-tightening in the Ocean State. But look at it another way: Gov. Lincoln Chafee’s latest budget proposed a large cut to Medicaid — to $819 million in state spending, or $1.9 billion when federal matching funds are added. The policy would cost just 0.1% of the Rhode Island Medicaid program’s annual budget. And therein lies the key to understanding the problem.

Targeted, “low-hanging fruit” interventions on the social-service side can help promote health outcomes at relatively low cost. We can, and should, pursue those interventions — and it shouldn’t take a well-made map to draw our attention to them.

We’re not arguing that Rhode Island’s Medicaid program should pay for the busing solution, of course. That would be an admittedly silly ad hoc way to make policy. What we’re trying to say is that healthcare and medical care are not synonyms. What we talk about in “health policy” is often more properly termed “medical care” — how to finance health insurance, the difficulties of narrow networks, how to manage our medical-care costs as a country, etc. But health is affected by more than medical care — and it’s definitely affected by social services, school transportation included. Yet we’ve separated the ways we pay for social services and medical-care services in ways that don’t always make sense. Moreover, bad health limits the ability of poor Rhode Islanders — and Americans — to take advantage of social services designed to give them a leg up, while limited social services can make pursuing health a Herculean task.

Talk about climbing uphill both ways.

*Much more on this topic coming soon: a review of The American Health Care Paradox, a new book by Elizabeth H. Bradley and Lauren A. Taylor.


Allan is a first year medical student at the Warren Alpert Medical School of Brown University, where he is pursuing an MD/MPP. You can follow him on Twitter @allanmjoseph.

Emily works in civic education in Providence, Rhode Island. She graduated from The Ohio State University with a degree in Public Affairs. You can follow her on twitter @emilygflower.

The BMJ study isn’t the final word on mammograms

by Allan Joseph

By now, nearly everyone who’s interested in health policy has heard about the huge study published in the BMJ this week — a 25-year randomized, controlled trial of mammography that shows no effect on mortality. (If you haven’t, Aaron Carroll’s excellent summary at the Incidental Economist is a good place to start.) The study comes at a time when breast-cancer screening is hotly debated, as is other preventive screening for cancers. Much has been made over the difference between survival rates and mortality rates, and whether the study is actually an appropriate randomized controlled trial (RCT), but that’s not what I want to talk about.

Let’s just, for the moment, assume that we all agree that the BMJ is methodologically perfect and externally valid. (If you’re relying on survival rates, let’s set that aside for now and go with mortality rates as the best measure of patient death due to illness). If we assume all of that, we come to a pretty sobering conclusion: mammograms don’t save lives.

So is that it? Should women not be getting mammograms? No — that would be far too strong a conclusion to draw, for two reasons.

1. Don’t forget about high-risk women. The BMJ study examined results among the population at large. Women with a history of breast cancer (who have a higher risk) weren’t in the study, nor did they analyze the data for women with family histories of breast cancer, BRCA genes, or biological risk factors such as lobular carcinoma in situ (LCIS). These are the women at highest risk for developing breast cancer, and the research doesn’t really say anything about them — only about the population at large. Until we have better data on screening high-risk women, we should be very cautious about changing the way high-risk women are screened. (As an aside, it’ll be hard to get a good piece of data on high-risk women — how could you possibly run an experiment?)

2. Mortality isn’t everything. Mortality rates are really important for measuring whether an intervention actually saves lives. But we’re not only interested in whether women with breast cancer live, and for how long. We also want to know about the quality of life. Does early detection mean less chemotherapy? Does it mean fewer full mastectomies and invasive surgeries? The BMJ study reported that tumors in the non-screening group were bigger than those in the mammography group (2.1 cm v. 1.4 cm, p<0.001), suggesting that they were detected later. It’s plausible that the two groups could get the same results in terms of mortality, but that the control group had to undergo more aggressive treatment to get the same results. Mammograms might not save lives, but they might save breast tissue and reduce chemotherapy regimens. If that’s the case, they very well might still be worth the price of over-diagnosis.

This study isn’t the final word on mammography. It’s certainly impressive and eye-catching research, and it should be — it’s extraordinary on scale alone. But even with some strong assumptions (external validity being the biggest), we’re not yet able to make blanket statements on mammography. We’re getting closer to knowing more, but we don’t have a final answer yet.


Allan Joseph is a first year medical student at the Warren Alpert Medical School of Brown University, where he is pursuing an MD/MPP. You can follow him on Twitter @allanmjoseph.