November 19, 2014
by Rob Mitchum, Computation Institute
Evaluating a new health care intervention can be a messy and costly process.
The gold standard for measuring the effectiveness of a new intervention is a randomized controlled trial, splitting patients into a treated group and an untreated group and comparing the results. But humans are not laboratory animals; they don’t exist in a perfectly isolated environment free from outside influences. Furthermore, because clinical trials are expensive and take years to run, investigators often only get one shot at an evaluation, making it difficult or impossible to learn from the results and continuously adjust the intervention to make it more effective.
Now, a collaboration between researchers at the Computation Institute and University of Chicago Medicine & Biological Sciences will create a new, information-rich tool for assessing and refining health care innovations and policies. With a $3 million grant from the National Institutes of Health, scientists from the Social and Behavioral Systems Group at Argonne National Laboratory will build an agent-based model to evaluate CommunityRx, a health information technology project underway in several South Side clinics.
Beyond providing valuable new information about the broader impact of CommunityRx, the researchers hope the model will lay the foundation for a new computational testing ground for health care and policy programs.
“It’s very costly to try something in the real world and have it fail or be ineffective,” said Charles Macal, PhD, CI senior fellow and Social and Behavioral Systems Group leader. “So why not devote what amounts to a tiny fraction of resources in advance to consider and identify what might be helpful interventions? In the end, this could save a lot of money and provide enormously better programs that are much more effective.”
CommunityRx, led by Stacy Tessler Lindau, MD, associate professor of obstetrics and gynecology and medicine at the University of Chicago, launched in 2012 with a $5.9 million Health Care Innovation award from the U.S. Centers for Medicare and Medicaid Services. The program provides patients with a printed HealtheRx report, a prescription of sorts for resources in the patient’s area to help manage a medical condition. That includes personalized and localized information about community health resources such as gyms, grocery stores, and pharmacies, to support self-care activities outside of the clinic and between doctor visits.
A sample HealtheRx document given to patients.
As it progresses, the CommunityRx team will use traditional methods to assess the effect of the program on health care costs. The researchers also would like to examine the broader impact of the intervention — how providing a patient with information about healthy, local resources ripples outward to potentially produce indirect health and economic benefits as well.
“When patients take this information home, they might influence or share it with family members or friends who then might also change their perceptions about what’s available in the community,” Lindau said. “We started looking for people with expertise to help us quantify and project the impact of a patient-level intervention on individuals, families, social networks, local communities, and even the broader economy.”
To achieve this goal, Lindau connected with Macal’s team through the Urban Sciences Research Coordination Network, a National Science Foundation-funded initiative led by the CI’s Urban Center for Computation and Data. Together, the teams discussed how agent-based modeling, a computational technique used to simulate the dynamics of complex social systems, could help researchers measure the full reach of CommunityRx, as well as potentially make the intervention more effective.
Agent-based models simulate social interactions by programming individual “agents” with decision-making rules that mimic real-world behavior. When millions of these agents are combined into a simulation, they produce complex, multi-scale dynamics that can be used to re-create and study financial markets, ecosystems, and transportation networks, among other areas. In the context of health care, agent-based models are typically used as epidemiological models simulating the spread of a disease.
In some regards, the CommunityRx model works similarly to those tracking diseases — “except,” Lindau said, “we’re trying to infect a community with information.” In this case, the agents of the model will be built to reflect the demographics of the patients, health care providers, and community services involved in the study, with their behavior driven by the results of surveys and data gathered during the intervention.
“The idea is to figure out how information flows through the system,” said Jonathan Ozik, PhD, CI senior fellow and computational scientist at Argonne. “Surveys of patients and surveys of providers will help us generate networks of who’s talking to who, how many people they regularly talk to about health-related topics, and if they have talked to people about this program.”
From there, simulations can be run to see how the information distributed by CommunityRx spreads out in the community. Researchers can take measurements on outcomes of interest, such as how many people receive the information, how it affects individuals’ behaviors, and changes in the usage of community resources. Using high-performance computing resources at the UChicago Research Computing Center and the Argonne Leadership Computing Facility, researchers can run the model tens of thousands of times with different starting parameters to estimate a range of possible outcomes.
“The nice thing about agent-based models is you basically have a virtual polling capability,” Ozik said. “At any point in time, we can create a snapshot and ask, ‘How are you feeling?’ or ‘How often have you gone to your local health club?’”
A previous example of an agent-based model built by Argonne researchers with Repast software.
The results of the simulations also may suggest unanticipated areas of interest to pursue in the real world: new questions to add to surveys or particular groups of people for CommunityRx investigators to contact. In addition, different strategies for the intervention can be tested in the virtual environment to inform how it is actually implemented. For example, researchers can study the differences in outcomes when patients are given HealtheRx prescriptions at every clinic visit or less frequently, to find the best balance of costs and benefits.
Once constructed for the South Side neighborhoods participating in the current study, the model can be expanded and adapted to test the impact of a CommunityRx-style intervention in other areas of Chicago or beyond.
“This approach allows us to move from understanding the mechanisms through which the CommunityRx intervention produces better health and health care to predicting the impact of the intervention in different settings and conditions,” Lindau said. “The simulation model will allow us to present CommunityRx to organizations and leaders in other cities and demonstrate quantitatively the potential impact of the intervention for their clients or residents. There’s just no way to do that with a traditional model.”
The researchers also hope that the model may be expanded to test the effects of other kinds of health programs or policies on a large scale. Agent-based models can help shape the design of these new programs before the study begins, and adjust its implementation as it is rolled out, saving valuable resources and time.
“I think this project could revolutionize the way people do program evaluation by putting it into the computer,” Macal said. “With this system, investigators can do all kinds of experiments on different policies or interventions and really gain insight into what is effective and what is not. We’re excited to be a part of what we think is a big wave that will really help people do this in the future routinely.”
In addition to Lindau, Macal, and Ozik, co-investigators on this project include UChicago Medicine’s David Beiser, MD, assistant professor of medicine, and Elbert Huang, MD, associate professor of medicine.
-See the original story at https://www.ci.uchicago.edu/blog/virtual-evaluation-modeling-ripples-health-care-information.