Reporting Our Results
CHMI’s evaluation and how we are responding
CHMI recently issued a call for health programs to report their results, good or bad, on key dimensions of impact like quality, efficiency, and outcomes. Today, we’re reporting the results of our own evaluation.
Earlier this year, the Bill & Melinda Gates Foundation commissioned a quasi-baseline survey of CHMI, with responses from nearly 600 program managers, donors, investors, researchers, policy makers, and other key health system stakeholders. Conducted by InterMedia, this evaluation aimed to set baseline performance data for CHMI as we enter our next phase of operation. In addition to the online survey, InterMedia conducted in-depth interviews with 30 people to assess CHMI’s use, usefulness, impact, and areas for future growth. The results provide insights about how CHMI is working. They will also help us improve and expand our offerings to provide information and connections for people who are working to improve health care for the poor.
I’d like to thank everyone who took the survey. We are already using your feedback to improve the way we identify, analyze, and scale up promising health innovations with potential to improve the private health sector.
People are using CHMI, and many feel they are getting value
Awareness of CHMI is high. 74% of those who took the survey (n=592) said they had heard about CHMI. Half of those aware of CHMI had used our information products, like the website, or had participated in CHMI activities. And of those who had used our products, 95% said they were satisfied.
Program managers said they used CHMI to attract donors and investors, and share their experiences with fellow implementers. Researchers interviewed identified CHMI as the connective tissue between academia, policy, and implementation. Many researchers said they frequently use CHMI to keep up-to-date about their area of interest. Similarly, investors felt CHMI is an important first source of information that can lead to deals, with one respondent noting, “there’s nothing out there like it.”
More than half of those who had used CHMI products agreed that CHMI contributed to an improved health status among the poor—speaking mostly to the potential of CHMI to achieve this impact in the future.
We are working to help the organizations we profile meet this goal, and survey feedback about how we can improve our information and networking offerings is helping us to foster scale up and adaptation of promising models. Here is what survey respondents told us they want from CHMI, and this is how we are responding.
Many respondents wanted more synthesis of “what works”
Sixty percent of respondents said they want to learn about good practices. Many program managers look to CHMI for lessons learned, good or bad: “I want to know ‘how do you achieve that?’ and ‘what were your learnings during the process’,” one manager said in our evaluation. So we are continuing our focus on collecting and reporting on program performance data through our Reported Results initiative.
Fifty-eight percent of respondents said they want to access synthesized information. So in our next phase we are also creating easy-to-access summaries of promising practices in key areas, like innovative ways that CHMI programs are delivering priority health services.
In addition, our community represents many diverse interest groups, and nearly all wish CHMI documented more and different information. Investors want financial data, researchers seek evaluation findings, and program managers look for niche information related to their diverse business models—such as call center scripts, quality assurance protocols for mobile health, or tips for creating government contracts in India. We may not be able to provide everything that our users want, but we are considering new ways to collect additional valuable information, and hope to augment our information offerings in the future.
But we will need your help to provide this kind of rich information. We currently profile more than 1200 nonprofits, social enterprises, public-private partnerships, and policies. To ensure our program profiles remain comprehensive and accurate, we will increasingly look to program managers to update us directly on their news.
Innovators wanted more connections that will foster scale-up and learning
In addition to seeking out lessons learned from other practitioners, many innovators are keen to connect with potential partners, including government leaders, to share their ideas. In our next phase, CHMI’s network of partners in low- and middle-income countries around the world will host competitions and other events to connect innovators with policymakers and other potential funders.
While nothing replaces the importance of in-person connections to foster partnerships, CHMI will increasingly connect people virtually, plugging into more networks and discussion platforms—places where users can get advice about important implementation challenges or connect with potential partners.
We know that many implementers may be reluctant to share their failures on a public website. One solution will be to facilitate discussions of lessons learned and failures in private forums, online as well as in person. In addition, CHMI can enable innovators to field diverse questions directly through online messaging. To ensure we effectively engage our diverse community online, we invite you to take a quick poll about your participation in social networks and online discussion platforms. Click here to take the survey.
Going into our next phase, CHMI will focus increasingly on fostering partnerships among program managers, donors, investors, researchers, and policy makers that lead to measurable scale-up or improvement of promising health care programs. Ultimately, we aspire to enable health systems around the world to better utilize private organizations to deliver quality, affordable and accessible care, especially for the poorest and most vulnerable.
Again, many thanks for your feedback and for helping us strengthen our offerings, so we can work together to improve health for the world’s poor.
Don’t forget to take CHMI’s quick poll on social networks and online discussion platforms! Everyone who takes the 5-question poll can download three new desktop wallpaper designs.
Pictured Above: A Bangladeshi family near a BRAC clinic, featured in the CHMI database.