Within hours of the U.S. Department of Health and Human Services' announcement that they planned to use a group of "mystery shoppers" to study access to primary care across the country, outcry erupted among physicians who felt the study was deceptive and unfair. "Snooping," one called it in the New York Times' coverage of the plan. A poor use of tax dollars, others said. Days later, the department announced they were putting the effort, which would have surveyed more than 4,000 physicians in nine states, on hold.
This week in the New England Journal of Medicine, Dr. Karin Rhodes, an emergency physician and health care policy researcher here at Penn's Perelman School of Medicine -- herself an expert in studies designed using the "secret shopper" method -- responds to the outcry in a "Perspective" piece aimed at taking the so-called "mystery" out of these studies. The technique is hardly new: It has a long history in other areas of the marketplace, where researchers have used it to tease out discrimination and disparities in employment, mortgage lending and housing access. Some critics of the plan felt it was unethical -- an entrapment of physicians, who would have no idea they were being audited. But when executed properly, Rhodes says, the studies employ concealment as a research tool in ways that are just as rigorous and ethical as the way in which drug trials are randomized and double-blinded to protect results from interpretation biases that may result when patients themselves or the doctors conducting the research know which patients are taking which drug.
"Just as well-designed clinical trials advance clinical care, well-designed audit studies -- the scientific name for mystery-shopper techniques -- are a powerful tool for understanding the experiences of patients as they seek needed health care," Rhodes writes. And, she notes, the primary care audit survey would have been much cheaper to conduct than the large, multi-year research studies of drugs and other treatments typically sponsored by the National Institutes of Health. Those projects can run well into the millions of dollars to follow a large enough sample of patients to obtain statistically significant results, and to follow them to long enough to properly assess safety, efficacy and long-term side effects.
Studies dating back almost 20 years have documented difficulties in the public's ability to access primary care services, and with only about half as many of today's medical students pursuing careers in family medicine and primary care as there were in the late 1990s, there's a looming shortage of these doctors in the pipeline to care for patients in the coming years. Access problems exist in other areas of medicine, too, and they're often hampered by insurance difficulties. Rhodes' own recent study, published last month in the New England Journal of Medicine, showed disturbing disparities in Medicaid-insured children's ability to get specialty care -- just a third of them were able to obtain appointments compared to nearly 90 percent of children with private insurance, and when they could get appointments, they waited twice as long for one. These studies helped put real, actionable data behind ideas that had previously only been known through unproven anecdotes.
But, Rhodes points out, many unknowns remain that are crucial to helping close gaps in access to care -- unknowns that a study like the one the government proposed could help identify. What, for instance, is the ideal patient-to-doctor ratio, and how does supply and demand for family physicians vary across the country? Identification of regional data could better allocate resources to address the problem. Expansion of loan forgiveness programs that send young doctors to underserved areas of the country might be one solution, perhaps, or incentivizing more family physicians to care for publicly insured patients by improving the sometimes difficult process associated with reimbursing them for care to this population.
Surveys of real patients -- one alternative suggested by critics of the federal plan -- couldn't come close to accurately capturing all factors involved in how and when patients seek medical care. For instance, they wouldn't be able to control for things like patients who have trouble speaking English and had difficulty communicating to appointment schedulers, or for those who called back multiple times rather than giving up after just one scheduling attempt. And those type of surveys would be subject to the so-called "recall bias" that dogs any retrospective look at a patient's experience -- if, for instance, a patient secured an appointment easily but disliked the doctor they saw, or had a long wait in the office before being cared for, it might color their response to questions in a survey about access.
For now, the DHHS study has been taken off the table indefinitely. Meanwhile, 33 states are considering cuts to their Medicaid-CHIP budgets, and the federal government may lower reimbursement rates that doctors receive to take care of patients on Medicare. More than than 100 million Americans rely on these programs for their health care, all of whom need some type of primary care. “We have determined that now is not the time to move forward with this research project,” the Department of Health and Human Services said just days after the study was announced. Neither, Rhodes says, is it the time for "mindless fear" to stand in the way of studying how to make sure those 100 million people can get help when they need it.