https://orcid.org/0000-0002-5531-0220
Figures
Abstract
Background
The majority of post-neonatal deaths in children under 5 are due to malaria, diarrhoea and pneumonia (MDP). The WHO recommends integrated community case management (iCCM) of these conditions using community-based health workers (CHW). However iCCM programmes have suffered from poor implementation and mixed outcomes. We designed and evaluated a technology-based (mHealth) intervention package ‘inSCALE’ (Innovations At Scale For Community Access and Lasting Effects) to support iCCM programmes and increase appropriate treatment coverage for children with MDP.
Methods
This superiority cluster randomised controlled trial allocated all 12 districts in Inhambane Province in Mozambique to receive iCCM only (control) or iCCM plus the inSCALE technology intervention. Population cross-sectional surveys were conducted at baseline and after 18 months of intervention implementation in approximately 500 eligible households in randomly selected communities in all districts including at least one child less than 60 months of age where the main caregiver was available to assess the impact of the intervention on the primary outcome, the coverage of appropriate treatment for malaria, diarrhoea and pneumonia in children 2-59months of age. Secondary outcomes included the proportion of sick children who were taken to the CHW for treatment, validated tool-based CHW motivation and performance scores, prevalence of cases of illness, and a range of secondary household and health worker level outcomes. All statistical models accounted for the clustered study design and variables used to constrain the randomisation. A meta-analysis of the estimated pooled impact of the technology intervention was conducted including results from a sister trial (inSCALE-Uganda).
Findings
The study included 2740 eligible children in control arm districts and 2863 children in intervention districts. After 18 months of intervention implementation 68% (69/101) CHWs still had a working inSCALE smartphone and app and 45% (44/101) had uploaded at least one report to their supervising health facility in the last 4 weeks. Coverage of the appropriate treatment of cases of MDP increased by 26% in the intervention arm (adjusted RR 1.26 95% CI 1.12–1.42, p<0.001). The rate of care seeking to the iCCM-trained community health worker increased in the intervention arm (14.4% vs 15.9% in control and intervention arms respectively) but fell short of the significance threshold (adjusted RR 1.63, 95% CI 0.93–2.85, p = 0.085). The prevalence of cases of MDP was 53.5% (1467) and 43.7% (1251) in the control and intervention arms respectively (risk ratio 0.82, 95% CI 0.78–0.87, p<0.001). CHW motivation and knowledge scores did not differ between intervention arms. Across two country trials, the estimated pooled effect of the inSCALE intervention on coverage of appropriate treatment for MDP was RR 1.15 (95% CI 1.08–1.24, p <0.001).
Interpretation
The inSCALE intervention led to an improvement in appropriate treatment of common childhood illnesses when delivered at scale in Mozambique. The programme will be rolled out by the ministry of health to the entire national CHW and primary care network in 2022–2023. This study highlights the potential value of a technology intervention aimed at strengthening iCCM systems to address the largest causes of childhood morbidity and mortality in sub-Saharan Africa.
Author summary
The inSCALE cluster-randomised trial in Mozambique was part of a $10million project funded by the Bill and Melinda Gates Foundation to design and test innovative primary care interventions to improve health outcomes for children with malaria, diarrhoea and pneumonia (MDP), which together are the largest killers of children aged <5yrs. The study aimed to strengthen the primary health care system with a focus on community health workers, representing the most accessible level of care for many underserved populations. We designed a technology-based intervention delivered using low-end smartphones. This intervention was based on mHealth principles and included electronic WHO algorithms with basic AI functions for community health workers (CHWs) to guide correct diagnosis and treatment of MDP, provided personalised feedback to health workers, and alerts to supervising health facilities on stock outs, data tracking and CHW competencies. The study was implemented within the entire province of Inhambane, and districts were randomly assigned to the intervention or to continue with standard care (control). Compared to control districts, we observed significant reductions in the prevalences of MDP in children under 5 years (reductions of 20% for malaria, 34% for pneumonia, and 45% for diarrhoea) and an increase in appropriate treatment of any cases of MDP by 26% (of all cases MDP) and 40% (of all children) in the intervention districts. As a result of this trial, the government of Mozambique incorporated the inSCALE intervention into its policy for child health services, and is in the process of scaling up the programme to all 8000+ CHWs across the country (2022–2023).
Citation: Soremekun S, Källander K, Lingam R, Branco A-CC, Batura N, Strachan DL, et al. (2023) Improving outcomes for children with malaria, diarrhoea and pneumonia in Mozambique: A cluster randomised controlled trial of the inSCALE technology innovation. PLOS Digit Health 2(6): e0000235. https://doi.org/10.1371/journal.pdig.0000235
Editor: Dylan A. Mordaunt, Flinders University, AUSTRALIA
Received: July 12, 2022; Accepted: March 20, 2023; Published: June 12, 2023
Copyright: © 2023 Soremekun et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: In line with the original consent criteria for the trial, the data for the inSCALE project can be available upon reasonable request from the data manager of the public repository LSHTM Data Compass under DOI https://doi.org/10.17037/DATA.00002559. Researchers wishing to access study data must perform the following actions: 1. Review the data codebook 2. Complete and submit a data request form (available at 10.17037/DATA.00002559) that outlines the purpose of their research, data variables/measurements that they wish to access, and other information necessary to support their request. Queries about the data transfer agreement/access process can be sent to researchdatamanagement@lshtm.ac.uk.
Funding: The inSCALE study was supported by grants to JT and BK from The Bill and Melinda Gates Foundation, grant number OPP1002407. The BMGF played no role in the design, analysis, decision to publish or preparation of this manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Globally more than 5 million mostly preventable deaths occur in children under 5 years of age every year [1]. The vast majority of post neonatal deaths in this group are due to pneumonia, diarrhoea, malaria or malnutrition [2]. It is acknowledged that urgent work is required to address the causes and more widely, the systems within which such deaths occur. However progress has been uneven; under 5 mortality rates in sub Saharan Africa today remain similar to or higher than 1990 mortality rates in most high income countries with an average rate of 78 deaths per 1000 live births across the continent in 2018 [2]. Lack of access to timely and appropriate clinical management of sick children is a key factor driving poor health outcomes [3,4] prompting a World Health Organisation/UNICEF joint statement in 2012 advocating for integrated community case management (iCCM) of malaria, diarrhoea and pneumonia delivered by community-based health workers (CHW), who would be trained to diagnose and treat key childhood conditions in areas with poor access to facility-based services [3]. In the decade following this statement, iCCM programmes have been introduced into more than 50 low- and middle-income countries (LMICs) [5–7]. Whilst a recent multi-country modelling study in DRC, Malawi, Niger, and Nigeria [8] estimated that iCCM may have been responsible for averting nearly 5000 deaths per year in these countries combined due to malaria, diarrhoea and pneumonia, this estimate varied considerably between countries, and was in contrast to a Cochrane systematic review which found no clear impacts of iCCM on morbidity and mortality [9]. Both studies reported poor or uneven implementation of iCCM programmes and a need for further studies that addressed structural deficiencies in health systems and processes that hamper success and scale-up.
Central to the challenge in implementation of iCCM has been the lack of sufficient support for CHW networks, high rates of CHW attrition, low rates of reported motivation and poor quality of disease management practices [10]. The WHO calls for “innovative and tailored approaches” to strengthening child health services [1], yet there are few programmes or even research studies which have explicitly addressed structural barriers in the context of scaling up iCCM effectively.
Digital technologies, and specifically mHealth (the use of mobile technologies in medical and public health practice [11]) is an area of growing interest for the delivery of health services in sub-Saharan Africa [12,13] particularly given the rapid expansion of mobile phone networks and usage in this setting [14]. We designed a systems-strengthening package of programmes collectively called”inSCALE” (interventions at scale for community access and lasting effects). inSCALE was a multi-country study of innovative interventions implemented by the Malaria Consortium in partnership with the governments of Mozambique and Uganda and supported by international funders (the Bill and Melinda Gates Foundation, UNICEF, and the Canadian international Development Agency now Global Affairs Canada (GAC)) and co-designed with academic partners (The London School of Hygiene and Tropical Medicine and University College, London). Reported in this paper are the findings of the inSCALE technology intervention, a disease management support package developed following extensive formative research and building on the study conceptual framework [15]. The package was primarily delivered on Samsung Galaxy smartphones (“the inSCALE app”) that included electronic clinical decision algorithms, personalised reports, messaging, and feedback to CHWs and health facility staff. The Mozambique site tested the inSCALE technology intervention only; the Uganda site tested both the technology intervention and a community-based intervention described elsewhere [16]. The technology interventions in both countries sought to specifically address primary health system challenges that could lead to improved management of key childhood illnesses through first improving CHW performance, motivation and reducing attrition. A detailed description of the development of the inSCALE technology intervention, rationale and conceptual framework is provided elsewhere [15]. The primary aim of the trial was therefore to assess the impact of the inSCALE technology intervention on the coverage of the appropriate treatment of childhood malaria, diarrhoea and pneumonia. This paper reports on the impacts of the technology intervention in Mozambique to accompany the separate impact analysis for the Uganda inSCALE trial [16] and conducts a meta-analysis of pooled impact of the technology intervention across its sites in Uganda and Mozambique.
Results
In total 9231 households in the 12 districts were assessed for eligibility and of these, 495 were empty or no one was home, and 4483 did not contain children under 5 years of age. Overall, the main caretaker was available in 92% (3920) of houses with children under 5 years of age. At the endline survey, a total of 2718 children had had symptoms of suspected malaria, diarrhoea or pneumonia (MDP) in the past month (the impact analysis sample), of which 1467 were in the control arm and 1251 in the intervention arm (Fig 1 flow chart). Household characteristics of sick children were fairly well balanced between arms with small variations in some parameters–Table 1.
Final analysis sample: children aged 2 months to 5 years with suspected malaria, diarrhoea or suspected pneumonia (MDP). * Maxixe and Inhambane, the economic and provincial capitals were excluded from the study (do not implement iCCM).
The restricted randomisation method resulted in good balance between intervention arms for balance variables as described in the protocol paper [15], which were care seeking rates, CHW motivation, and cost of seeking care (Table 2). Differences for all variables was either less than 5% (percentages), or less than 0.1 (means).
Intervention coverage, sickness and care seeking
By the end of the 18-month implementation period, 68% of CHWs reported having a working inSCALE phone and app, and 45% had sent a report via the app in the last 4 weeks (Table 3). Whilst overall prevalence of MDP was similar between arms at baseline, there were significant reductions in the prevalence of suspected malaria, diarrhoea and pneumonia (reductions ranging from 18% to 45%), and a 16% increase in the prevalence of diagnostic testing for malaria in the intervention arm at endline (Table 4). Care seeking to the CHW as the first port of call was 14.4% in the control arm and 15.9% in the intervention arm. After adjusting for the restricted randomisation and clustered design this represented a 63% increase in care-seeking to the CHW, a result that just fell short of significance (Table 5, p = 0.085). There were no significant differences in care seeking rates to other providers or in the prevalence of children for whom no care was sought (Table 5).
Appropriate treatment
Baseline levels of appropriate treatment were similar for the combined MDP endpoint but varied by provider and for individual conditions between intervention arms (Table 6). At endline there was an 26% increase in appropriate treatment of cases of MDP (the primary outcome (Table 7) in the inSCALE arm compared to the control arm (95% confidence interval: 12%-42%, p<0.001). This rose to 30% when only episodes treated with recommended first line treatments were defined as appropriately treated (Table 7 95% CI: 12%-52%, p = 0.001). A breakdown of appropriate treatment by provider suggests this impact was driven by significant improvements in appropriate treatment coverage in public health facilities (Table 7) which were the first port of call for caregivers for more than 60% of sick children in both arms (Table 5), and in the appropriate treatment of diarrhoea, where a 76% increase (95% CI 20%-256% p = 0.003) in the use of oral rehydration salts was observed in the intervention arm (Table 8). Note it was not possible to estimate a confidence interval for use of ORS plus zinc due to small cell sizes. Additional improvements came in the use of the recommended first line antibiotic for pneumonia (50% increase in prescribing of amoxicillin in the intervention arm 95% CI: 3%-217%). A whole-child analysis (Table C in S1 Text) estimated an improvement of 40% in the proportion of children appropriately treated for all MDP condition(s) reported in the last 4 weeks in the intervention arm (95% CI: 9% - 79% p = 0.009).
CHW motivation, identification, clinical knowledge, attrition, and stockouts
The mean motivation and the mean social identification scores were relatively high and did not differ between arms (difference in motivation factors score: -0.03 95% CI -0.57, 0.51; difference in identification factor score: 0.01 95% CI -0.42, 0.43). CHW attrition rates were approximately 18%-19% in both intervention arms with no detected difference (Table 9). Overall scores for CHW clinical knowledge of the signs and management of MDP, also did not differ significantly between arms (Table 9). A breakdown of CHW knowledge scores by condition suggests a trend of improved knowledge in technology arm CHWs particularly in diagnosing pneumonia and diarrhoea—though none reached the threshold for statistical significance and may be chance findings (Table F in S1 Text). There were no between-arm differences in the rate of stockouts for key MDP treatments in the previous 3 months, though overall rates of stockouts were fairly high at >50% (Table 9).
Sensitivity analyses and robustness tests
Sensitivity tests were conducted with two alternative definitions of appropriate treatment: i) only including children with a positive malaria diagnostic test, and ii) including quinine as an appropriate treatment for malaria to account for its use as a second line treatment or during in-patient care (Table B in S1 Text). Both showed a trend of improved coverage of overall appropriate treatment (20% an 14% improvement, p values 0.006 and 0.073 respectively for i) and ii)). We performed robustness tests of the key primary and secondary impact outcomes appropriate treatment, care seeking, CHW motivation, social identity, clinical knowledge, stockouts, and attrition by repeating the analyses using a comparison of cluster-level mean outcomes (Tables D1-E2 in S1 Text). This approach estimated similar significant improvements in appropriate treatment as the random effects model with the exception of the increase in care seeking to CHWs which did not reach significance.
Meta-analysis of the impact of the inSCALE technology intervention on appropriate treatment for MDP: Uganda and Mozambique
Fig 2 shows the results of a fixed effects meta-analysis of the risk ratios (RRs) comparing coverage of appropriate treatment in control and intervention arms in Uganda and Mozambique. The analysis suggests implementation of the inSCALE technology model could improve appropriate treatment by 15% (8% - 24%) overall.
Effect size (ES) is the risk ratio for appropriate treatment. Fixed effects meta analysis (country = fixed effect) Appr. Treat = appropriate treatment of illness episodes (suspected malaria, diarrhoea or suspecter pneumonia). I = intervention arm, C = control arm. ES = baseline-adjusted risk ratio (effect size) of % episodes MDP appropriately treated in intervention versus control arm.
Discussion
We have demonstrated that a scalable, innovative technology-based intervention implemented in Mozambique can meaningfully improve the coverage of appropriate treatment for common childhood illnesses covered by the iCCM programme by 26% (any guideline recommended treatment) and 30% (first-line treatments). These changes were mostly driven by significant improvements in the correct treatment of diarrhoea and pneumonia, and in increased appropriate treatment in primary care facilities. We did not observe between arm differences in CHW outcomes (motivation, social identity, knowledge, and attrition). We finally showed in a meta-analysis of two countries (Mozambique and Uganda) implementing similar technology-based interventions that an 8–24% improvement in appropriate treatment is likely achievable elsewhere. Our results suggest that iCCM programmes can contribute to the quality of care provided by primary health services by the use of relatively inexpensive technology that prioritises support, efficiency and performance. As a result of this work, the government of Mozambique has expanded the inSCALE technology intervention and to date seven out of eleven provinces with over 2,500 CHWs and 299 supervisors are using an expanded version of the app. The platform is being rolled out to all 8,800 CHWs and 1,300 supervisors nationally by mid-2023 and a full data integration plan with DHIS2 is ongoing to optimize strategic decision making[18, 19]. The expanded programme name, upSCALE, which includes not only the IMCI sick child module, but also growth monitoring, family planning, adult illness, HIV/TB treatment follow-up and COVID-19, reflects the successful and rapid uptake of our research into national policy and the creation of a national CHW mHealth system[19].
Our conceptual framework for the design of the inSCALE innovation expected that differences in CHW motivation, attrition and social identification would be one of the drivers of improvement in appropriate treatment[15, 20]. As noted in our formative research and in other settings, ownership of local programmes can be fragmented, CHW interventions are often only tenuously linked with the wider health system and therefore poorly supervised, and their contribution under-recognised[21]. The CHWs we surveyed however were in general highly motivated and reported a strong degree of connectedness to the wider health system in both arms at baseline and at the end of follow-up. Control arm CHWs will have received some support offered as standard practice—supervision, review of consultation records and refresher training as required. Local facility records estimated CHW attrition was moderate at 18% over the implementation period and did not differ by intervention arm. Our findings suggest that in contrast to our conceptual framework, alternative routes to impact may be more important in this setting. Though coverage of appropriate treatment in children taken to the CHW was similar across arms, we observed the largest overall increase in coverage of appropriate treatment occurred at public health facilities and in children with diarrhoea and in first line treatments for pneumonia, very similar to the pattern of impact observed in the sister inSCALE trial in Uganda [16]. That these patterns were largely consistent across two countries with separate intervention designs and implementation teams is a persuasive indicator of the alternative impact pathways and mechanisms of impact through which inSCALE operated—and warrants further exploration in future studies. This may highlight the integrated nature of inSCALE intervention where training and usage of the app by both CHWs and supervising staff at primary care facilities will have increased facility-based health worker exposure to guideline-led recommendations for disease management, alerts for low CHW stock for key commodities, motivational messaging, and reporting on prevalence of key childhood conditions. Diarrhoea in particular is often perceived simply as a symptom of a more highly prioritised condition on which treatment is focused [22] and may therefore be overlooked by health workers even if ORS is available [22–24]. These findings point to the gains that may be made by adopting an approach that not only strengthens community based iCCM programmes, but can also positively impact more broadly on the primary health care system.
Coverage of the inSCALE innovation after 18 months of follow-up was moderate to good (68% CHWs had a working phone and inSCALE app and 45% had sent one or more reports in the last month). It is difficult to assess this intervention coverage rate in context of similar interventions due to the variability in intervention components, and in endpoint definitions. A SMS (short message service) message-based randomised controlled trial in Tanzania delivered 7 messages to drug shop vendors with recommendations for correct treatment of malaria and reported 70% of dispensers received at least 75% of the intended SMS messages. Whilst this appears comparable to our reported rates, the difference in the health provider type and in the content of the intervention precludes a direct comparison [25]. Nonetheless a review of a selection of trials of child health-based mHealth interventions in low and middle income country settings (LMICs) report widely varying intervention coverage rates between 24% and 85% [26, 27].
Lack of medicinal stock for common childhood conditions has been a barrier to prompt and appropriate treatment in other settings [28], and affected our study sites in Mozambique during and after the period of implementation [8, 21]. Key issues were the national stock outs of key malaria and antibiotic medications and diagnostic tests within the last 6 months of the study, which may have supressed the full potential impact of the technology intervention for these outcomes. Research by Save the Children in this setting suggests that households who were aware of stockouts at the iCCM CHW may go elsewhere for treatment [21]. More than 50% of CHWs across both arms reporting at least one stockout of supplies for the management of MDP in the previous 3 months; such commodity supply barriers were outside the control of our intervention nonetheless it is encouraging that the intervention remained impactful in the context of these persistent issues and moderate intervention coverage.
Wider implications for case prevalence, fatality, productivity, user costs and transmission
The highest case fatality rates for pneumonia, malaria and diarrhoea occur in sub-Saharan Africa [29, 30]; and important proximal outcomes of community-based interventions like inSCALE may include a potential to reduce community transmission of causative pathogens and/or the risk of recurrent MDP infections [31] and lowering cost of care seeking and increasing equitable access to care [24] through the strengthening and raised profile of iCCM-trained CHWs. There was a decrease in prevalence of MDP over time across the study site in Mozambique, and the additional significantly faster fall in prevalence in intervention arm districts. The primary aim of inSCALE was to deliver more appropriate treatment for MDP through a strengthening of the health system. A secondary aim of the intervention was to increase community awareness of MDP through the community-based actions of the health workers. Decreased prevalence of illness may arise through the timely use of appropriate medications, and increased awareness of preventative measures; however we cannot rule out the presence of unmeasured confounding including the influence of secular trends.
Our study had several limitations. Our estimates of prevalence of MDP and appropriate treatment were based on caregiver reports which may be subject to misclassification. Studies of caregiver recall for pneumonia and malaria symptoms have sensitivities and specificities in the range of 31%-91% depending on the setting [17, 32]. Recommended strategies to improve the accuracy of disease and treatment estimates include setting the length of the recall period to four weeks (versus the 2 week recall period used in the inSCALE baseline survey), the use of pictures of available treatments to respondents and the development of survey questions which prioritise local terms and are based on well-validated survey instruments [17, 32, 33] including the United Nations- and USAID-supported Multiple Indicator Cluster Survey and Demographic Health Surveys [34, 35]. We incorporated all these strategies in our surveys and used tailored definitions of disease and treatments in line with national guidelines, including the presumptive use of ACT to treat fever, a practice compounded by nationwide stockouts of iCCM drugs and tests at the time. Our sensitivity analyses of the main outcome with alternative definitions of appropriate treatment [see Table B in S1 Text] nonetheless produced similar results–of note, restricting cases of fever to those with a confirmed malaria test estimated a 20% improvement in appropriate treatment which remained highly significant. The inSCALE study evaluated the impact of strategies to improve scale-up and coverage of iCCM programmes using a randomised controlled design as we believed this method would provide the most convincing effectiveness estimates for stakeholders. The number of district clusters available for randomisation was however determined by iCCM programme readiness (iCCM-trained CHWs in place) and lack of concurrent interventions in the area at baseline. This resulted in twelve districts available for randomisation. We tested the robustness of our random-effects models with cluster-averaged t-tests of our primary outcomes as recommended by Hayes and Moulton [36], and whilst less powerful, we observed similar estimates of impact on appropriate treatment (See Table D1 in S1 Text). We are missing information for 495 households (250 and 245 households where no one was at home in the control and intervention arms respectively), which if we assume a similar proportion contain eligible children to the proportion observed in visited household, about half of the 495 may have contained one or more eligible children under 60 months. Any impacts on generalisability are difficult to quantify however these are likely to be minimal given the missing represents 6% of all households/eligible households.
Conclusions
The body of research evidence for digital and mHealth programmes is increasing, however there remains a significant gap for evidence from good quality studies of the potential for available and emerging technologies to transform health outcome in LMICs for children, particularly when delivered via community-based health workers [37]. Mobile phone penetration is predicted to increase in sub-Saharan Africa [38] and is estimated to be a key medium by which health care access and service delivery can be improved for the underserved. The WHO’s recently published set of global recommendations for digital interventions for health services delivery highlight the potential of mHealth interventions for a range of uses and includes the key components of the inSCALE technology intervention–clinical decision support, stock tracking and provider-to-provider telemedicine [39]. Significantly, the 2019 WHO/UNICEF iCCM strategy meeting in Addis Ababa recommended “Interventions to improve quality, including supportive supervision and mentoring of CHWs in designated health facilities, are essential to ensure high-quality iCCM and should be budgeted for and included in district plans”[40]. This renewed encouragement for implementation of iCCM this decade may encourage relevant stakeholders looking to efficiently scale up programmes to explore digital solutions for their context. Our study and meta-analysis together with the expansion of the inSCALE programme in Mozambique [18, 19] highlight the potential scalability of an mHealth intervention aimed at strengthening iCCM systems to address the largest causes of childhood morbidity and mortality in sub-Saharan Africa.
Materials and Methods
Setting
The study took place in the Inhambane Province of southern Mozambique between November 2013 and July 2016. The province covers a geographical area of 68,775km2 with a population of approximately 1.4 million of which 17% are under 5 years of age [41]. The Inhambane mortality rate in this age group at the start of the study was 76.4 deaths per 1000 livebirths [42], with more than 60% of all deaths in the province caused by a combination of malaria, diarrhoeal diseases and pneumonia [43, 44]. Since 2011, iCCM had been included in the Inhambane programme of work for Agentes Polivalentes Elementares (APE), the name for the local cadre of community health worker and referred to as CHWs in this paper. Government and donor funding for the programme supported the training, supervision and provision of iCCM commodities for all CHWs in the province throughout the study period. A detailed study protocol for the inSCALE trial in Mozambique has been published elsewhere [15]. Details of the setting and study design for the inSCALE study in Uganda, used for the metanalysis presented in this paper, is published elsewhere [16].
Study design and participants
We conducted a superiority open label two-arm cluster-randomised controlled trial in 12 districts in Inhambane Province in southern Mozambique. Six districts (Funhalouro, Govuro, Jangamo, Panda, Vilankulo, and Zavala) were randomised to the inSCALE intervention and six (Homoine, Inharrime, Inhassouro, Mabote, Massinga, and Morrumbene) to the control arm. The provincial and economic capitals of Inhambane: Inhambane City and Maxixe respectively, was excluded from the randomisation and Maxixe was instead used for piloting of the intervention in its rural areas during the design stage. Each district was defined as a cluster and randomly allocated to either intervention or control; the rationale being the district was the lowest unit of administration and training for the CHW iCCM programme and therefore for implementation of the inSCALE intervention. Eligible households within the district were defined as those with at least one resident child under five years of age at any point during the study in a community served by at least one iCCM-trained CHW. Local data collected by Malaria Consortium in the area in 2011 suggested that 45% of households within each community fit the eligibility criteria.
Ethical approval for the study was obtained from the Mozambique National Bioethics Committee for Health (reference 345-CNBS-10) and the London School of Hygiene & Tropical Medicine Ethics Committee in the UK (reference 5762). Approval for the random allocation of districts to intervention or control groups and implementation of the interventions was obtained from the Inhambane Provincial Office responsible for health administration for the province. Written informed consent was obtained from the main caregiver of each child under 5 years of age for baseline and follow up data collection. Consent forms included an explanation of the purpose of the data collection and confirmed participants were free to withdraw at any time. The inSCALE trial is registered on clinicaltrials.gov with identifier NCT01972321.
Randomisation and blinding
In order to reduce the likelihood of chance imbalances between intervention arms at baseline given the number of clusters available for randomisation [36, 45]., we performed a randomisation that was restricted with respect to the percentages of households with sick children seeking care at a public health facility, the percentage seeking care to a CHW, average CHW motivation (composite) score and the log cost of care seeking for sick children. Data for these parameters was obtained from a baseline survey conducted between November- December 2012. We used the statistical software Stata (version 13) to generate the 924 possible randomisation schemes for the 12 districts and picked one at random from the subsection of 84 schemes that fitted our balance criteria [46] [see online S3 Text]. Due to the nature of the technology intervention, it was not possible to blind participants or fieldworkers to the intervention allocation, however allocation was masked in the database and all analyses were initially performed on data with intervention allocation removed and repeated on unblinded data after agreement on the final analysis methods with study partners.
Study procedures
Sample size calculations.
The baseline survey was conducted in all households in 2–8 randomly-selected communities (known as enumeration areas or EAs) in each district in order to cover approximately 500 households and 350 children per district (mean of 91 households per village). In the absence of site-specific data, these estimates were based on equivalent baseline data on prevalence of illness and between-cluster correlation from the sister inSCALE trial in Uganda. Following the baseline survey in Mozambique, sample size calculations for the impact analysis were updated using local data on MDP prevalence and the proportion of households with eligible children; indicating that by sampling 530 households with a rate of 48% containing eligible children we would have 90% power to detect an impact of the intervention of a 20% or higher increase in appropriate treatment given an estimated coefficient of variation in the outcome between clusters of 0.16 and a rate of 40% of children with MDP receiving appropriate treatment in the control group. Parameters and calculations are described in more detail in the inSCALE protocol paper [15]. Approximately 21 CHWs were also surveyed per district at baseline (1–2 per village). Community sampling frame data were based on complete enumeration information provided by the District Offices and EA location, size and households were verified by field workers.
Data collection.
The baseline survey was administered to caregivers of children under 5 years of age, and collected information on symptoms of malaria, diarrhoea, and pneumonia (MDP) in these children in the last 2 weeks, care seeking for these illnesses, costs of care, socioeconomic data at the household level. Household questionnaires were based on standard health survey instruments used in the country (Mozambique DHS [34] and Malaria Indicator Cluster Surveys [35]). The CHW survey included data on CHWs’ clinical knowledge, time use, motivation and ‘social identity’–or sense and degree of connectedness to the local and national CHW and health services systems. The CHW clinical knowledge instrument was developed based on the format and example clinical event scenarios in the WHO Health Facility Survey Tool template [47]. The likert scale-based CHW motivation and identity tool used a pictorial scale indicating degree of agreement with statements about enthusiasm for the job, motivation and identification with the local and national iCCM programme, and was developed based on extensive formative research by the study team on health worker motivation and social identity [48, 49]. Both tools were adapted to the Mozambique setting prior to usage with the input of facility staff and CHWs in an iterative process that developed wording, accompanying graphics and reference to local health system structures and clinical guidelines.
Household and CHW surveys were also extensively field piloted with field staff including workshops role playing and refining of translations, and all tools were delivered in Portuguese. A quality control check of 10% of the weekly survey households was undertaken by field worker supervisors. Pictures of locally available medications were used in the household surveys to improve the accuracy of reports of treatment for MDP [17].
At the end of the follow-up period an endline impact survey was conducted in May-June 2015 in the same villages visited at baseline in households with children meeting inclusion criteria at endline, and again covering all active CHWs serving the selected villages. The survey collected data across the same themes to estimate study impact and included additional questions to CHWs and caregivers on coverage of the inSCALE innovation components in the intervention arm. The recall period for recent illness in children (caregiver reported) was extended to 4 weeks in the endline survey following the publication of a series of seminal papers recommending increasing the exposure period to 28 days to improve accuracy of reports of suspected pneumonia [17] household and CHW data collection tools are available in S2 Text.
Intervention
The inSCALE technology intervention was developed in collaboration with Dimagi a social enterprise corporation focused on the development of digital solutions for health workers in low and middle income settings [50]. Steering and input was received from the Mozambique ministry of health, community and facility-based stakeholders, funding bodies and the inSCALE trial steering committee [15, 51]. The aim of the innovation was to leverage technological strategies to strengthen links between fragmented CHW programmes and the primary care health services including links between CHW peers and health facility staff, and by doing so to improve CHW outcomes and child health outcomes for children. The inSCALE technology intervention development process and conceptual framework linking intervention components, outputs and outcomes is described in detail in the protocol paper [15].
The technology intervention was implemented in July 2013, and after a 6 month roll-out and embedding period, follow-up to impact analysis continued to July 2015 (18 months). iCCM-trained CHWs and their health facility supervisors in the intervention arm were provided with a Samsung Galaxy smartphone installed with the inSCALE mHealth application (with CHW and facility supervisor versions), a solar charger with a number of pin adapters, and a solar lamp for night-time consultations. Intervention arm CHWs and health facility supervising staff were trained in the use of the mHealth tools. Training was conducted using a cascade approach where 7 master trainers trained 18 facility- and community-based trainers who in turn trained 47 health facility supervisors and 132 CHWs in the technology arm over 4 days with follow-up support provided by the project team during the implementation period. The inSCALE mHealth mobile app consisted of a sick child decision support algorithm in a user-friendly interface including multimedia, audio and images to improve adherence to iCCM treatment protocols. CHWs were additionally able to use the phones to track consultations with sick children, make diagnoses and management choices, report key statistics to a central server, request stock from their supervising facilities, and maintain a register of patients seen. These reports were forwarded on a weekly basis to health facility supervising staff who received tailored follow-up supervision prompts. The inSCALE app also worked offline, therefore where there were network coverage issues, CHWs and health facility supervising staff had the option to delay sending of reports until they were in a location with better coverage. CHWs and supervisors were allocated to local closed user groups which enabled complimentary calls with the inSCALE smartphones to be made without charge to fellow members in the group. Finally, CHWs and their health facility supervisors received weekly (CHWs only) and monthly (CHWs and facility supervisors) automated motivational messages tailored to their report results with the tone and emphasis based on formative work. In brief this included pretesting of messages with subgroups of CHWs and supervisors where versions of the app were evaluated on acceptability, comprehension, and usefulness, and phone functionalities were adapted accordingly. Email report formats were pretested with supervisors and iteratively updated. A series of action-oriented tips were developed based on a range of common performance-related scenarios, and all materials were adapted to Mozambique’s context and android phone software, and translated into Portuguese.
After an initial 6-month roll-out and embedding phase, the technology intervention continued to be implemented in intervention arm districts for an additional 18 months. Standard care during this period was available in both intervention and control arm districts, where CHWs and health facilities provided guideline-led management for childhood illnesses via the underlying iCCM programme and integrated management of neonatal and childhood illnesses (IMNCI) services [9, 52] respectively.
Data on CHW recruitment, training, resignations, and replacements were tracked by supervising health facilities at district level and this data was regularly extracted into the inSCALE database during the follow-up period by the study team in order to estimate attrition rates.
Standard practice in control arm
Control arm CHWs did not receive the inSCALE package and accessories. CHWs in both arms received training in iCCM, health promotion and education, supplies of commodities including rapid diagnostic tests and drugs for common childhood illnesses, monthly supervision, and a $40 monthly subsidy. CHWs in both arms maintained paper based registers. Additional details of the components of the intervention arms is provide in Table I in S1 Text
Outcomes
All study outcomes were based on data collected on questionnaires during the cross sectional survey at the end of the intervention follow-up period as described above (Data Collection). The primary outcome for the trial was the percentage of reported recent cases of MDP in the last 4 weeks in children 2 months to 5 years of age that were treated using appropriate medication [15]. Secondary outcomes were the prevalence of MDP in children at endline, the percentage of cases of MDP appropriately treated with recommended first line medications, the percentage of children who sought care for MDP by provider type, the percentage of CHWs who had had a stock-out of iCCM medications in the past 3 months, CHW attrition rate calculated as the proportion of CHWs who left their post during the follow-up period out of the total CHWs trained at baseline, CHW clinical knowledge scores (overall and by condition subgroup), and composite scores describing CHW enthusiasm and drive for the job (motivation) and sense of connectedness to the local and national iCCM programme (social identification). CHW knowledge scores were calculated as the sum of correct clinical decisions mentioned by the CHW in response to a series of scenarios involving children with symptoms of mild and severe disease. Points for correct items were weighted according to an algorithm defined in Tables G1-G6 in S1 Text. CHW motivation and social identification scores were calculated by a factor analysis and empirical item reduction process applied to original field tool responses. The methods have been detailed previously [53], and resulted in a three-factor CHW motivation scale of 17 items measuring general motivation, feelings of reward for effort, and retention in role. An additional single factor CHW scale of 4 items was identified which explained the majority of the variation in social identification between CHWs. Retained items from both the motivation and social identification tools showed good internal consistency (alpha = 0.67 for CHW motivation and alpha = 0.69 for CHW social identification). The final motivation scores were calculated as the product of the rotated factor analysis regression coefficients and the Likert value for each original item. The final social identification scores were calculated as the sum of the original 4 items (after reverse coding as appropriate).
Additional impact measures included the percentage of children who were appropriately treated overall whether they had symptoms of one or more of MDP in the last 4 weeks (i.e. a per-child analysis), and two sensitivity analyses of the main impact outcome using alternative definitions of appropriate treatment. All definitions of appropriate treatment and medications used in primary secondary and additional endpoints are provided in Table H in S1 Text.
Statistical analysis
All analyses were based on intention-to-treat principles and accounted for the cluster-randomised design. Multivariate mixed-effects models were specified for all impact outcomes, which account for the clustering of outcomes by inclusion of a random intercept at the district level. Intervention arm, the cluster mean percentage of children who were appropriately treated at baseline, and the parameters used to balance intervention arms during randomisation were included as fixed effects. Unadjusted analyses of impact were also performed, and the results included in Table A in S1 Text. For the primary outcome and other binary outcomes, odds ratios were converted to risk ratios using the marginal standardisation technique and 95% CIs estimated using the delta method [54]. Additional sensitivity analyses of the primary outcome were conducted to account for alternative treatments available for MDP [see Table H in S1 Text for definitions]. Robustness checks of the primary outcomes were conducted by estimating cluster averaged results using t-tests following the procedures of Hayes and Moulton [36], recommended when the number of clusters per arm is less than 10[36] [see Tables D1-E2 in S1 Text]. CHW attrition was estimated using cluster-averaged mean rates as human resources information was provided at this level only from district health offices. In order to estimate a pooled effect of the inSCALE technology intervention on appropriate treatment across both Uganda and Mozambique, we performed a meta-analysis of effect sizes by combining the results of the Mozambique (this study) and Uganda [16] trials of the inSCALE intervention. As there was no significant heterogeneity (<50% estimated from exploratory meta-analysis models) between studies, we present the pooled result from a fixed effects meta-analysis.
Supporting information
S1 Text. Table A. Unadjusted analyses of primary outcomes.
Table B. Sensitivity analyses for primary outcome at endline (any episode of MDP): Alternative definitions of appropriate malaria treatment. Table C. Whole Child Analysis–Appropriate treatment. Table D1. Robustness tests (cluster-averaged outcomes)–Primary Outcomes. Table D2. Robustness tests (cluster-averaged outcomes)–Secondary Outcomes: appropriate treatment stratified by condition or provider Table E1. Robustness tests (cluster-averaged outcomes)–Secondary outcomes: prevalence of suspected and confirmed malaria, malaria blood testing, diarrhoea, and pneumonia. Table E2. Robustness tests (cluster-averaged outcomes)–Secondary outcomes: prevalence of care-seeking for suspected malaria, diarrhoea and pneumonia. Table F. CHW clinical knowledge by subgroup (diagnosis of MDP, treatment of MDP). Table G1. Item weighting–Case study: Francis/Alberte (pneumonia and diarrhoea). Table G2. Item weighting–Case study: Hope/Tina (malaria). Table G3. Item weighting–Case study: Beatrice/Janete (severe pneumonia). Table G4. Item weighting–Case study: Muteesa/Kizito (severe disease/malaria). Table G5. Item weighting–Case study: James (severe disease/malaria). Table G6. CHW Knowledge scoring for subcategories: diagnosis and treatment of suspected malaria, diarrhoea and pneumonia. Table H. Definitions of appropriate treatment and other key child health outcomes. Table I. Intervention and control arm components—inSCALE Mozambique.
https://doi.org/10.1371/journal.pdig.0000235.s001
(DOCX)
S3 Text. Restricted Randomisation Procedure and codes–inSCALE Mozambique.
https://doi.org/10.1371/journal.pdig.0000235.s003
(DOCX)
Acknowledgments
We wish to thank the local research team who collected and managed the data, diMaggi for support in designing the app software, the CHWs and supervisors who delivered the intervention, and the participants of the study for their time and insight. SS, KK, RL, NB, FK and BK had full access to all the data and SS had final responsibility for the decision to submit for publication.
References
- 1. WHO. Children: improving survival and well-being. Sep 2020 [cited 29 Sep 2021]. Available: https://www.who.int/news-room/fact-sheets/detail/children-reducing-mortality
- 2. UNICEF. Levels and trends in child mortality 2019: Estimates developed by the UN Inter-agency group for child mortality estimation. Sep 2019. Available: https://www.unicef.org/reports/levels-and-trends-child-mortality-report-2019
- 3. WHO, UNICEF. WHO/UNICEF JOINT STATEMENT: Integrated Community Case Management (iCCM). 2012. Available: https://www.who.int/maternal_child_adolescent/documents/statement_child_services_access_whounicef.pdf
- 4. Kallander K, Burgess DH, Qazi SA. Early identification and treatment of pneumonia: a call to action. Lancet Glob Health. 2016;4: e12–e13. pmid:26577842
- 5. Diaz T, Aboubaker S, Young M. Current scientific evidence for integrated community case management (iCCM) in Africa: Findings from the iCCM Evidence Symposium. J Glob Health. 2014;4: 020101. pmid:25520783
- 6. Boschi-Pinto C, Labadie G, Dilip TR, Oliphant N, Dalglish SL, Aboubaker S, et al. Global implementation survey of Integrated Management of Childhood Illness (IMCI): 20 years on. BMJ Open. 2018;8: e019079. pmid:30061428
- 7. Allen KC, Whitfield K, Rabinovich R, Sadruddin S. The role of governance in implementing sustainable global health interventions: review of health system integration for integrated community case management (iCCM) of childhood illnesses. BMJ Glob Health. 2021;6: e003257. pmid:33789866
- 8. Prosnitz D, Herrera S, Coelho H, Moonzwe Davis L, Zalisk K, Yourkavitch J. Evidence of Impact: iCCM as a strategy to save lives of children under five. J Glob Health. 2019;9: 010801. pmid:31263547
- 9. Oliphant NP, Manda S, Daniels K, Odendaal WA, Besada D, Kinney M, et al. Integrated community case management of childhood illness in low- and middle-income countries. Cochrane Database Syst Rev. 2021 [cited 22 Sep 2021]. pmid:33565123
- 10.
The Global Fund for HIV TB and Malaria, WHO, UNICEF. Integrated Community Case Management (iCCM) in Sub-Saharan Africa: Successes & Challenges with Access, Speed & Quality. Nairobi, Kenya; 2018 Aug p. 145. Available: https://www.theglobalfund.org/media/9754/core_integratedcommunitycasemanagementsub-saharanafrica_review_en.pdf
- 11. WHO Global Observatory for eHealth. mHealth: new horizons for health through mobile technologies: second global survey on eHealth. 2011 [cited 5 Feb 2022]. Available: https://apps.who.int/iris/handle/10665/44607
- 12. Braun R, Catalani C, Wimbush J, Israelski D. Community Health Workers and Mobile Technology: A Systematic Review of the Literature. Bullen C, editor. PLoS ONE. 2013;8: e65772. pmid:23776544
- 13. Källander K, Tibenderana JK, Akpogheneta OJ, Strachan DL, Hill Z, ten Asbroek AHA, et al. Mobile Health (mHealth) Approaches and Lessons for Increased Performance and Retention of Community Health Workers in Low- and Middle-Income Countries: A Review. J Med Internet Res. 2013;15: e17. pmid:23353680
- 14. World Bank. Mobile cellular subscriptions (per 100 people)—Sub-Saharan Africa. 2020 [cited 5 Feb 2022]. Available: https://data.worldbank.org/indicator/IT.CEL.SETS.P2?locations=ZG
- 15. Källander K, Strachan D, Soremekun S, Hill Z, Lingam R, Tibenderana J, et al. Evaluating the effect of innovative motivation and supervision approaches on community health worker performance and retention in Uganda and Mozambique: study protocol for a randomised controlled trial. 2015;16: 157. Trials. pmid:25873093
- 16. Källander K, Soremekun S, Strachan D, Hill Z, Kasteng F, Kertho E, et al. Improving community health worker treatment for malaria, diarrhoea and pneumonia in Uganda through inSCALE community and mHealth innovations: A cluster randomised controlled trial. PLOS Digit Health. 2023.
- 17. Campbell H, el Arifeen S, Hazir T, O’Kelly J, Bryce J, Rudan I, et al. Measuring Coverage in MNCH: Challenges in Monitoring the Proportion of Young Children with Pneumonia Who Receive Antibiotic Treatment. PLOS Med. 2013;10: e1001421. pmid:23667338
- 18. Feldman M, Krylova VL, Farrow P, Donovan L, Zandamela E, Rebelo J, et al. Community health worker knowledge, attitudes and practices towards COVID-19: Learnings from an online cross-sectional survey using a digital health platform, UpSCALE, in Mozambique. PLOS ONE. 2021;16: e0244924. pmid:33566850
- 19. Malaria Consortium. upSCALE mHealth system strengthening for case management and disease surveillance. 2018. Available: https://www.malariaconsortium.org/upscale/. Accessed 17 Sep 2021.
- 20. Global Fund to Fight AIDS, Tuberculosis and Malaria. Global Fund Thematic Review on Community Health. 2020 Jun p. 46. Available: https://www.theglobalfund.org/media/10128/core_endasantecommunityhealthsynthesis_review_en.pdf
- 21.
Save the Children. Systems Effects of Integrated Community Case Management Projects: Mozambique Country Report. In: Resource Centre [Internet]. 8 Apr 2019 [cited 11 Jun 2021]. Available: https://resourcecentre.savethechildren.net/library/systems-effects-integrated-community-case-management-projects-mozambique-country-report
- 22. Unger CC, Salam SS, Sarker MSA, Black R, Cravioto A, Arifeen SE. Treating diarrhoeal disease in children under five: the global picture. Arch Dis Child. 2014;99: 273–278. pmid:24197873
- 23. Nsabagasani X, Ogwal-Okeng J, Mbonye A, Ssengooba F, Muhumuza S, Hansen EH. Availability and utilization of the WHO recommended priority lifesaving medicines for under five-year old children in public health facilities in Uganda: a cross-sectional survey. J Pharm Policy Pract. 2015;8: 18. pmid:25995847
- 24. Soremekun S, Kasteng F, Lingam R, Vassall A, Kertho E, Settumba S, et al. Variation in the quality and out-of-pocket cost of treatment for childhood malaria, diarrhoea, and pneumonia: Community and facility based care in rural Uganda. PLOS ONE. 2018;13: e0200543. pmid:30475808
- 25. Bruxvoort K, Festo C, Kalolella A, Cairns M, Lyaruu P, Kenani M, et al. Cluster Randomized Trial of Text Message Reminders to Retail Staff in Tanzanian Drug Shops Dispensing Artemether-Lumefantrine: Effect on Dispenser Knowledge and Patient Adherence. Am J Trop Med Hyg. 2014;91: 844–853. pmid:25002300
- 26. Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, Rowe AK, et al. The effect of mobile phone text-message reminders on Kenyan health workers’ adherence to malaria treatment guidelines: a cluster randomised trial. Lancet Lond Engl. 2011;378: 795–803. pmid:21820166
- 27. Friedman W, Woodman B, Chatterji M. Can mobile phone messages to drug sellers improve treatment of childhood diarrhoea?—A randomized controlled trial in Ghana. Health Policy Plan. 2015;30 Suppl 1: i82–92. pmid:25759456
- 28. Mubiru D, Byabasheija R, Bwanika JB, Meier JE, Magumba G, Kaggwa FM, et al. Evaluation of Integrated Community Case Management in Eight Districts of Central Uganda. PLOS ONE. 2015;10: e0134767. pmid:26267141
- 29. Marangu D, Zar HJ. Childhood pneumonia in low-and-middle-income countries: An update. Paediatr Respir Rev. 2019;32: 3–9. pmid:31422032
- 30.
WHO. Child mortality and causes of death. Geneva, Switzerland; 2022. Available: https://www.who.int/data/gho/data/themes/topics/topic-details/GHO/child-mortality-and-causes-of-death
- 31. Information NC for B, Pike USNL of M 8600 R, MD B, Usa 20894. Malaria transmission and antimalarial medicines. Guidelines for the Treatment of Malaria. 3rd edition. World Health Organization; 2015. Available: https://www.ncbi.nlm.nih.gov/books/NBK294437/
- 32. Eisele TP, Silumbe K, Yukich J, Hamainza B, Keating J, Bennett A, et al. Measuring coverage in MNCH: accuracy of measuring diagnosis and treatment of childhood malaria from household surveys in Zambia. PLoS Med. 2013;10: e1001417. pmid:23667337
- 33. Walker CLF, Fontaine O, Black RE. Measuring Coverage in MNCH: Current Indicators for Measuring Coverage of Diarrhea Treatment Interventions and Opportunities for Improvement. PLOS Med. 2013;10: e1001385. pmid:23667330
- 34.
Demographic Health Surveys. The DHS Program—Questionnaires and Manuals. In: USAID [Internet]. [cited 11 Jun 2021]. Available: https://dhsprogram.com/Methodology/Questionnaires.cfm
- 35.
UNICEF. Tools—UNICEF MICS. [cited 11 Jun 2021]. Available: https://mics.unicef.org/tools
- 36.
Hayes R, Moulton L. Cluster Randomised Trials. 2nd ed. Chapman and Hall/CRC; Available: https://www.routledge.com/Cluster-Randomised-Trials/Hayes-Moulton/p/book/9781498728225
- 37. Feroz A, Jabeen R, Saleem S. Using mobile phones to improve community health workers performance in low-and-middle-income countries. BMC Public Health. 2020;20: 49. pmid:31931773
- 38. Crul S, Diop J. The mHealth opportunity in Sub-Sahara Africa: The path towards practical application. Deloitte; 2014. Available: https://www2.deloitte.com/content/dam/Deloitte/nl/Documents/technology-media-telecommunications/deloitte-nl-mhealth.pdf
- 39.
WHO. WHO Guideline: recommendations on digital interventions for health system strengthening. World Health Organization; 2019. Available: http://www.who.int/reproductivehealth/publications/digital-interventions-health-system-strengthening/en/
- 40. WHO, UNICEF. Institutionalising integrated community case management (iCCM) to end preventable child deaths: Technical consultation and country action planning 22–26 July 2019, Elilly International Hotel, Addis Ababa, Ethiopia.
- 41. National Institute of Statistics Mozambique. Inhambane—Instituto Nacional de Estatistica. 2013 [cited 11 Jun 2021]. Available: http://www.ine.gov.mz/estatisticas/estatisticas-territorias-distritais/inhambane
- 42.
UN Inter-agency Group for Child Mortality Estimation. Under five Mortality Rate—Mozambique. [cited 11 Jun 2021]. Available: https://childmortality.org/data/Mozambique
- 43. Fernandes QF, Wagenaar BH, Anselmi L, Pfeiffer J, Gloyd S, Sherr K. Effects of health-system strengthening on under-5, infant, and neonatal mortality: 11-year provincial-level time-series analyses in Mozambique. Lancet Glob Health. 2014;2: e468–e477. pmid:25103520
- 44. Källander K, Counihan H, Cerveau T, Mbofana F. Barriers on the pathway to survival for children dying from treatable illnesses in Inhambane province, Mozambique. J Glob Health. 9. pmid:31275569
- 45. Ivers NM, Halperin IJ, Barnsley J, Grimshaw JM, Shah BR, Tu K, et al. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials. 2012;13: 120. pmid:22853820
- 46. Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clin Trials Lond Engl. 2004;1: 297–305. pmid:16279255
- 47. WHO. Health facility survey: tool to evaluate the quality of care delivered to sick children attending outpatient facilities. World Health Organization; 2003. Available: https://apps.who.int/iris/handle/10665/42643
- 48. Chandler CIR, Chonya S, Mtei F, Reyburn H, Whitty CJM. Motivation, money and respect: a mixed-method study of Tanzanian non-physician clinicians. Soc Sci Med 1982. 2009;68: 2078–2088. pmid:19328607
- 49. Franco LM, Bennett S, Kanfer R, Stubblebine P. Determinants and consequences of health worker motivation in hospitals in Jordan and Georgia. Soc Sci Med 1982. 2004;58: 343–355. pmid:14604620
- 50.
DIMAGI. Dimagi | Technology for Social Impact. In: Dimagi [Internet]. [cited 9 May 2022]. Available: https://www.dimagi.com/
- 51. Strachan DL, Källander K, Nakirunda M, Ndima S, Muiambo A, Hill Z, et al. Using theory and formative research to design interventions to improve community health worker motivation, retention and performance in Mozambique and Uganda. Hum Resour Health. 2015;13: 25. pmid:25925007
- 52.
WHO UNICEF, editors. Handbook IMCI: integrated management of childhood illness. Geneva: World Health Organization; 2005.
- 53. Strachan DL. Ugandan community health worker motivation: using the Social Identity Approach to explore an accepted constraint to scaled up health strategies. Doctoral thesis, UCL (University College London). Doctoral, UCL (University College London). 2019. pp. 1–410. Available: https://discovery.ucl.ac.uk/id/eprint/10073256/
- 54. Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007;60: 874–882. pmid:17689803