Aligning Forces for Quality Evaluation

Lessons learned from a decade long healthcare improvement program enrich the field of health services research

The Aligning Forces for Quality (AF4Q) initiative, launched in 2006, was the Robert Wood Johnson Foundation’s (RWJF) signature effort to improve the health and healthcare of 37 million people across the United States and to provide models for widespread national reform. In this nearly decade-long investment, RWJF provided unprecedented funding and expert technical assistance to 16 multi-stakeholder healthcare coalitions (“alliances”) composed of those who get care, give care, and pay for care, to work together to lift the overall healthcare of their communities. The alliances were specifically charged with aligning five key “forces” to improve healthcare and patient experiences, and reduce costs:

• Performance measurement and public reporting
• Quality improvement
• Engaging consumers in their health and healthcare
• Reducing healthcare disparities
• Reforming payment

In addition to providing funding to the AF4Q alliances, technical assistance vendors, and communications firms, RWJF contracted an independent evaluation led by Dennis Scanlon, director of the Penn State Center for Health Care and Policy Research, and researchers in multiple disciplines at Northwestern University, the University of Minnesota, the University of Michigan, and the George Washington University. This 10-year mixed methods evaluation examined the AF4Q program design and implementation as the national healthcare environment was undergoing tectonic change, including the passage of the Affordable Care Act (ACA) in 2010.

photo credit: Aligning Forces for Quality

Our evaluation design and work included continuously tracking the activities in the 16 AF4Q communities; collecting, synthesizing, and analyzing large volumes of program data; interviewing over 1100 stakeholders from varying perspectives; and administering national surveys to assess both the impact of the program interventions and to identify lessons learned about the multi-stakeholder approach to improving healthcare at the community level. Evaluating a program with the breadth and depth of AF4Q enabled our team to develop a variety of insights that will inform policymakers, program funders, and other stakeholders working to implement healthcare improvement interventions in communities. For example, we identified many barriers that healthcare collaboratives face and described some cultural shifts that have the potential to transform health systems. Furthermore, the lessons we extracted from the alliances’ experiences have helped to inform RWJF’s new strategy to create a national Culture of Health.

"Our findings from AF4Q provide insight into the important work happening locally to create better healthcare systems and the many challenges regions face when trying to make things better and more affordable."--Dennis Scanlon, Principal Investigator, Aligning Forces For Quality Evaluation

Finally, through the evaluation experience, we have expanded the health services research knowledge base with over 80 journal articles and research summaries including the August 2016 supplement of the American Journal of Managed Care (AJMC), The Aligning Forces for Quality Initiative: Summative Findings and Lessons Learned from Efforts to Improve Healthcare Quality at the Community Level. This issue provides in-depth information on AF4Q’s research design; background and evolution; interventions in the main programmatic areas; AF4Q’s longitudinal impact on measures of population health, quality and experience of care, and cost of care; and the sustainability of the AF4Q alliances. You may also read articles published as part of the 2012 AJMC supplement: The Aligning Forces for Quality Initiative: Early Lessons From Efforts to Improve Healthcare Quality at the Community Level, as well as a summary of our AF4Q early and midpoint findings in the November 2014 report, Early and Mid-Course Findings from the Evaluation of a National Health Improvement Program.

Key findings at the conclusion of AF4Q:

• Implementing and aligning activities across all five “forces” proved difficult for any one AF4Q community.
• While collaboration among stakeholders to improve systems is conceptually appealing, there are many challenges to sustaining regional relationships, though AF4Q highlighted some promising models.
• Over time, the AF4Q regions showed no major differences in improvement rates in quality and health measures examined, compared with control communities. However, all communities tended to trend towards improvement.
• Engaging consumers in reforming the system proved more difficult than expected, but the conversation has been elevated. The importance of transforming the system “by patients and for patients” is gaining traction.
• Sustaining quality and price transparency at the community level is challenging. Important questions moving forward include: Who will fund and produce this information, and who receives the most value from it?
• Disparities reduction activities proved challenging but happened with more intensity in communities that had long-established relationships with community groups representing diverse and underserved populations.