Monitoring and Evaluation: CHMI Plus Gold Programs

Five years after identifying and sourcing health innovations on a global platform, the Center for Health Market Innovations (CHMI) team found that the database could be improved by providing more ways for programs to share information and impact. CHMI Plus was launched.in early 2015 as a way for users to see which programs are active in sharing detailed information about their program. This new feature increases the level of easily available information about programs’ approaches and monitoring and evaluation practices. It is also intended to improve transparency and accountability in global health, improve reporting practices, and, ultimately, enhance our collective understanding about what is producing promising results for the poor. With over 1,500 programs in the database, having a CHMI Plus rating of Bronze, Silver, or Gold can help programs stand out. Gold-rated programs are automatically featured on CHMI’s homepage.

116 programs in the CHMI database currently have Gold-status profiles. Many of these programs were included in this year’s annual Highlights report, are considered for funding, and are given chances to participate in opportunities like CHMI’s Learning Exchange. Many programs with a Gold status have attached monitoring and evaluation documents to their CHMI profiles, including impact evaluations, process evaluations, and/or data collection strategies. This openness helps other programs to understand what works (and what doesn't) in the field, and provides accountability for programs that claim success. Of the CHMI Plus Gold programs, 17 programs provide monitoring and evaluation documents. One such program, Safe Water and AIDS Project (SWAP), seeks to improve WASH indicators and reduce the incidence of HIV/AIDS in communities across Kenya. The program trains community health promoters to sell hygiene products, such as water filtration devices, condoms, sanitary pads, and soaps door-to-door.  SWAP collects data on the number of products sold through surveys that the community health promoters perform for their assigned residents. SWAP was a featured program in the 2015 CHMI-Bertha Center report on the Kenyan healthcare landscape, partially due to their thorough documentation of program M&E in the CHMI database. The CHMI Plus system enables programs to upload impact evaluations, processes evaluation, and data collection methods to their profile.

Impact Evaluations

Impact evaluations measure the burden of a health problem before and after a program is implemented, and is and essential part of good monitoring and evaluation. Eight programs in the CHMI database are Gold status and have attached impact evaluations to their profile.

D-Tree International, a multinational non-profit, specializes in providing mobile-phone-based clinical decision software for accurate diagnoses by Community Health Workers (CHWs). To evaluate the impact of their program, they measured the accuracy of diagnosis between paper-based Integrated Management of Childhood Illness (IMCI) protocol and their electronic IMCI (eIMCI) software. The study found that adherence to IMCI protocol was greater with the electronic system.¹ These reported results have the potential to help fundraising efforts for D-Tree International, as their program has proven to be effective. Another program evaluating the effectiveness of their electronic technologies in primary care is Baobob Health Trust in Malawi. For instance, in the dispensing of ART combination therapy to patients in healthcare facilities in Malawi, 70% of healthcare providers reported a preference and improvement in quality of care due to the electronic medical record initiative from Baobob Health Trust over traditional, paper-based methods.²

Process Evaluations

Process evaluations measure if programs met their goals over a certain period of time. One of the nine Gold programs in the CHMI database that have provided a process evaluation is FATA-KP Health Program-Pakistan, a USAID initiative that focuses on improving family services and improving maternal, newborn, and child health indicators. One of FATA-KP’s goals is to improve community-based access to integrated MNCH services in targeted communities. After one year, over 95,000 people were reached as a result of initiatives such as Child and Maternal Health Days, which fulfilled USAID’s goal of enhancing the ability of providers to manage MNCH illnesses throughout the entire popualtion.³  

Data Collection Strategies

Data collection strategies show outside audiences how data was collected to carry out impact or process evaluations. High quality data collection strategies signify accurate assessments, which lead to better interventions, and ultimately better outcomes. Common data collection strategies include conducting surveys, making field observations of health behaviors, putting together focus groups, carrying out randomized trials, and interviewing stakeholders. 

Healthy Entrepreneurs (HE), a social franchise based in Central Africa and Haiti, plans to begin collecting data this year to measure the impact of their program. HE has already identified key stakeholders, and plans on conducting customer surveys on the efficiency of the entrepreneurs’ ability to deliver medications and information. Prima Bina (Window of Opportunity), an initiative of CARE, also conducts household surveys, though on different topics, such as prevalence of stunting, malnutrition, and presence of infection. Prima Bina has taken this baseline data so that after the intervention is implemented, these measures can be taken again and the efficacy of the program can be assessed.⁴

LifeNet International provides sample questions that they ask their patients and add this questionnaire in their annual report so that funders and other stakeholders know exactly how the organization is collecting data and assessing needs. LifeNet additionally uses a Quality Score Card (QSC) to assess their partners’ performance of health impact over time. The QSC is a robust investigation of health interventions, and has helped LifeNet receive funding, obtain more partners, and, ultimately, do what the organization was aimed to do: improve the lives of people.

CHMI Plus Going Forward

Programs in the CHMI database can use their CHMI Plus status to easily display their impact on target populations. Outside vistors can learn from featured programs who share detailed M&E information, and other profiled programs can learn best practices from effective innovations. Going forward, CHMI encourages programs to achieve Gold status so they can reap the benefits of more funding, learning opportunities, and features on our website and blogs. Questions about how to become a part of CHMI Plus? Contact Lane Goodman at lgoodman@r4d.org or Jeff Arias at jarias@r4d.org. 

Photo ©LifeNet International.