Programmes that aim to target disease control and improve health outcomes for populations in low- and middle-income countries (LMICs) are often complex and multi-faceted and, as a result, it can prove challenging to track effectiveness and measure impact against an intended goal. Limited and timely access to quality data is often at the root of these challenges and can have knock-on consequences on, for example, supply chains, resource allocation and quality of an intervention, whether it is being delivered to a high standard and reaching the people that need it. Digital tools that support routine monitoring and data collection, and their use by ministries of health and partner organisations, can help address these challenges by providing evidence in real-time, supporting iterative decision-making and ensuring there is adequate data to measure overall impact.
Tracking results for resilient health systems
I joined Malaria Consortium’s Global Monitoring and Evaluation team at a pivotal moment. The organisation’s five-year strategy had just been launched, with a focus on supporting countries to progress towards universal health coverage align with this, and a major emphasis on making effective use of digital tools and approaches that improve data access and quality, enhance skills development and support disease surveillance and outbreak response.
One of the big draws for me – coming from a background in data analysis, strategy specialisations and multi-sectoral work experience – was the team’s emphasis on holistic approaches. Malaria Consortium implements evidence-based programmes to improve health outcomes in multiple geographies. We seek to embed a culture of data-informed decision-making across both our programmes and those within national programmes so we are collectively better able to prioritise and target interventions to where they are needed most.
Utilising LogAlto (a software-as-a-service provider), the team designed an online platform to help the organisation holistically track and effectively assess progress towards planned objectives – the tool was named PReS (Project Results System). The platform incorporated results frameworks and logframes, helping programme implementers keep sight of a project’s overarching goal, as well as the short-term effects of project outputs and measure longer term success against Strategic Objective Indicators (SOIs). Allowing users to zoom in or out to view a project, country, regional and global levels has also helped programme managers to ask the right questions: Has a programme reached its targets? Do programme implementers have effective resource to reach the targeted community?
The platform allows Malaria Consortium to track contributive results aligned to focus areas across our strategy, including working towards strategic goals around malaria elimination, resilience at all levels of the health system, pathways to achieving universal health coverage, as well as generating the necessary evidence to influence policy and practice at national and global levels.
What does PReS look like in practice?
In 2022, Malaria Consortium’s seasonal malaria chemoprevention (SMC) programme reached 24 million children with lifesaving antimalarial treatment across seven countries: Burkina Faso, Chad, Mozambique, Nigeria, South Sudan, Togo and Uganda. The significant scale of this work presents unique and complex implementation challenges, particularly when it comes to using data for decision-making.
Working with national governments to deliver health interventions is a critical part of ensuring activities are collaborative and sustainable. However, across SMC implementing countries, National Malaria Control Programmes have differing levels of data collection, data quality and data use that can impact on the data available for monitoring and evaluation activities relating to Malaria Consortium’s SMC programme. Countries also differ in their implementation strategies, such as the number of SMC cycles and the times of year in which they take place. These differences can make it hard to get a comprehensive picture across all of settings.
Specific challenges were identified by Malaria Consortium’s SMC team and, following the programme’s integration into PReS, a monitoring and evaluation (M&E) framework with specific, measurable, achievable, relevant and time-bound indicators was developed that allowed the SMC programme, across all implementing countries to track acceptability, decision-making, fidelity, supply and demand, quality, and safety of SMC. These indicators cut across different stages of the annual SMC round using four main routine data-informed decision-making instruments:
- microplanning
- end-of-cycle reports,
- lot quality assurance sampling (LQAS)
- end-of-round surveys, to enhance the uptake of data.
PreS also provides a work breakdown structure to help track the timeliness of these surveys.
The screenshots below are a snapshot of two indicators tracked using the SMC M&E framework where at a glance, users can see the number of courses of SMC drugs administered in the cycle and whether targets have been met.
The SMC M&E framework has helped the programme to keep track of quality indicators; addressing quality, acceptability and other issues of programme implementation, since data use is embedded into routine practice to track and assure the quality delivery of the various components of SMC.
Setting health systems up for long-term success
LMICs typically have some of the highest burdens of endemic disease and limited resources to tackle them. Strong and consistent data capture and evaluation is a critical component of building resilient and responsive health systems, making it easier to identify weaknesses or inefficiencies in interventions, delivery models or areas of a health system and, in turn, helping to identify the most effective interventions for a specific area or group of people and optimising the use of available resources to provide better health outcomes. Data collection and analysis across different fields will be central to predicting and responding to climate-related issues.
Working alongside ministries of health to strengthen their own health management information systems is a key part of this. Learnings from a system such as PReS, enables Malaria Consortium to share reliable, actionable, and evidence-based results with national governments, who can pool this with other data sources to formulate a better understanding of determinants and contributory factors and contribute to data-driven decision-making and more accurate strategic planning.
Joshua is Malaria Consortium’s Monitoring and Results Measurement Manager, based in Abuja.