https://orcid.org/0000-0002-5778-5780
Figures
Abstract
The inSCALE cluster randomised controlled trial in Uganda evaluated two interventions, mHealth and Village Health Clubs (VHCs) which aimed to improve Community Health Worker (CHW) treatment for malaria, diarrhoea, and pneumonia within the national Integrated Community Case Management (iCCM) programme. The interventions were compared with standard care in a control arm. In a cluster randomised trial, 39 sub-counties in Midwest Uganda, covering 3167 CHWs, were randomly allocated to mHealth; VHC or usual care (control) arms. Household surveys captured parent-reported child illness, care seeking and treatment practices. Intention-to-treat analysis estimated the proportion of appropriately treated children with malaria, diarrhoea, and pneumonia according to WHO informed national guidelines. The trial was registered at ClinicalTrials.gov (NCT01972321). Between April-June 2014, 7679 households were surveyed; 2806 children were found with malaria, diarrhoea, or pneumonia symptoms in the last one month. Appropriate treatment was 11% higher in the mHealth compared to the control arm (risk ratio [RR] 1.11, 95% CI 1.02, 1.21; p = 0.018). The largest effect was on appropriate treatment for diarrhoea (RR 1.39; 95% CI 0.90, 2.15; p = 0.134). The VHC intervention increased appropriate treatment by 9% (RR 1.09; 95% CI 1.01, 1.18; p = 0.059), again with largest effect on treatment of diarrhoea (RR 1.56, 95% CI 1.04, 2.34, p = 0.030). CHWs provided the highest levels of appropriate treatment compared to other providers. However, improvements in appropriate treatment were observed at health facilities and pharmacies, with CHW appropriate treatment the same across the arms. The rate of CHW attrition in both intervention arms was less than half that of the control arm; adjusted risk difference mHealth arm -4.42% (95% CI -8.54, -0.29, p = 0.037) and VHC arm -4.75% (95% CI -8.74, -0.76, p = 0.021). Appropriate treatment by CHWs was encouragingly high across arms. The inSCALE mHealth and VHC interventions have the potential to reduce CHW attrition and improve the care quality for sick children, but not through improved CHW management as we had hypothesised.
Trial Registration:ClinicalTrials.gov (NCT01972321).
Author summary
There is global consensus that trained and equipped community health workers (CHWs) are a cornerstone of primary health care and pandemic preparedness. Yet, efforts to improve their motivation and quality of care delivered, in particularly to children with potentially life-threatening conditions like malaria, pneumonia and diarrhoea, are still largely undocumented. This is the first study that has implemented mHealth and community engagement interventions at scale to support community health workers and appropriate treatment of sick children. Our results suggest that interventions that promote learning and increase CHW’s confidence and connectedness to the wider health system and community, be it by using mHealth or community engagement, can increase appropriate treatment for malaria, diarrhoea, and pneumonia. However, sub-analysis showed between arm effects were at the level of the health facility and pharmacy with CHWs offering elevated levels of appropriate treatment across arms; possibly due to increased awareness of appropriate treatment among both caregivers and health providers. While the body of evidence on mHealth is increasing, few studies have to date documented the effect on health outcomes. Our robust evaluation, using a cluster randomised controlled trial, is therefore one of the very few that have shown a statistically significant impact of an mHealth intervention on appropriate treatment of sick children. The second intervention, village health clubs (VHCs) also increased appropriate treatment of sick children, in particular for diarrhoea.
Citation: Källander K, Soremekun S, Strachan DL, Hill Z, Kasteng F, Kertho E, et al. (2023) Improving community health worker treatment for malaria, diarrhoea, and pneumonia in Uganda through inSCALE community and mHealth innovations: A cluster randomised controlled trial. PLOS Digit Health 2(6): e0000217. https://doi.org/10.1371/journal.pdig.0000217
Editor: Laura M. König, University of Bayreuth: Universitat Bayreuth, GERMANY
Received: April 7, 2022; Accepted: February 17, 2023; Published: June 12, 2023
Copyright: © 2023 Källander et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data is available at: Soremekun, S (2022). inSCALE Uganda Impact Evaluation Dataset. [Data Collection]. London School of Hygiene & Tropical Medicine, London, United Kingdom. https://doi.org/10.17037/DATA.00002559.
Funding: The study was funded by Bill & Melinda Gates Foundation (https://www.gatesfoundation.org/) and awarded to co-author SM (Grant number OPP1002407). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Every year 1.8 million children under the age of 5 years die from malaria, diarrhoea, and pneumonia (MDP), accounting for 30% of all-cause mortality in this age group. In 2015 it was projected that a further 21 million children will die from these three treatable conditions by 2030 [1]; a number likely to be further augmented by the COVID-19 pandemic [2]. The ability of many low-income countries to meet the Sustainable Development Goal (SDG) of reducing under 5 mortality is frustrated by sparse numbers of health workers to provide essential services. Integrated Community Case Management (iCCM), endorsed by WHO and UNICEF, utilises community health workers (CHWs) to diagnose and treat MDP at a community level [3]. Evaluations across four African countries, the Democratic Republic of Congo, Malawi, Niger, and Nigeria, estimated that 6200 children were saved from 2013–16 due to the introduction of iCCM [4]. However, large scale robust evaluations have shown no effect on child mortality, often attributed to suboptimal programme implementation including lack of systems for ensuring the availability of medicines, CHW performance and community engagement [5]. If iCCM, and other CHW programmes are to reach their full potential, there is an urgent need for strategies and innovations that improve performance, supervision, and motivation of CHWs.
Behavioural theories have demonstrated how the processes that determine an individual’s behaviour, such as performance, are dependent upon interpersonal relationships and group memberships and their perceived value and significance to the individual [6]. As per this so called “social identity approach” CHWs are more likely to be motivated to perform their duties if their aspirations are perceived to be achievable through the fulfilment of their duties. Thus, interventions that seek to align collective worker goals with the aims of programmes have promise [6]. However, there is limited evidence for how to deliver such interventions at scale.
There is modest, but growing evidence that other CHW focussed interventions, such as mHealth, can improve CHW performance by increasing adherence to treatment recommendations [7,8]. Studies have also shown that mHealth tools for CHWs, such as smartphones programmed with training materials and decision support algorithms, can modestly improve quality of care through better assessment of the sick children and better referral decisions [9] and increase performance feedback and counselling of CHWs [10]. However, robust evaluations of direct impact of mHealth on health and economic outcomes are lacking, as is the evidence on the effectiveness of mHealth delivered at scale [11]. Another potential means of improving CHW motivation and performance is through community engagement. Low utilisation of CHWs by community members, resulting from a lack of buy-in, trust and alignment between CHWs and their communities, has previously been shown to undermine the potential impact of iCCM [5,12]. Therefore, community empowerment approaches, where communities actively participate in activities and decisions that affect their own health through community groups including CHWs have been recommended to improve health seeking behaviours [13].
The Innovations at Scale for Community Access and Lasting Effects (inSCALE) project set out to develop and evaluate two interventions based on the social identity approach (described above) through extensive formative research in Uganda; a mHealth intervention and a participatory community engagement approach delivered through Village Health Clubs (VHCs) [6]. Both were designed to strengthen community health services by enhancing the supervision, motivation, performance, and retention of CHWs to increase the appropriate treatment of childhood MDP. This paper presents the results from a cluster randomised controlled trial (cRCT) in Uganda, which assessed the impact of the mHealth and VHCs interventions compared to control.
Methods
Study design and participants
The inSCALE Uganda evaluation was a three-arm cRCT where the unit of randomisation was the sub-county (cluster). The study was conducted across 39 predominantly rural sub-counties in eight districts in the Midwestern region of Uganda. We excluded sub-counties which contained less than 10 villages or where there were no iCCM trained CHWs. In total, 13 sub-counties were randomly assigned to the mHealth intervention, 13 to the Village Health clubs and 13 to usual care (control). All CHWs residing in the study sub-countries were automatically included in the intervention. The study area spanned 24,450 km2 with a population of 1.8 million residing in around 4,000 villages. A detailed study protocol has been published previously [14]. iCCM training had been provided to all participating CHW’s by the Non-Governmental Organisation (NGO) Malaria Consortium and national iCCM trainers from July 2010-June 2011.
Data collection procedures
Baseline and follow up data were collected using two cross sectional household and CHW surveys across the trial sub-counties. The baseline survey was carried out in June-August 2011. The interventions were implemented from November 2012 to February 2014. In February and March 2013, a mixed methods process evaluation was conducted to understand experiences with intervention uptake and perceived impact, the mechanisms behind CHW motivation and identification with the CHW collective, and to learn about how and why the interventions worked or did not work. The follow-up household and CHW surveys were administered in April-June 2014, 17 months after intervention rollout. Both data collection rounds took place during the end of the rainy season/early dry season. Five villages were randomly selected per sub-county for the household surveys. A list of all households in each community was supplied by the local parish council and verified by the field supervisor. From this list households were selected and surveyed at random, in accordance with the sample size requirements for the evaluation [14].
Household questionnaires, based on standardised questions from the Ugandan Demographic and Health Survey (DHS) and the Multiple Indicator Cluster Survey (MICS) instruments, were administered by trained field staff to the primary caretaker of children under 5 [15,16]. They included socio-economic and demographic characteristics of households, symptoms of illness for children under 5 years in the preceding two and four weeks (at baseline and endline, respectively), care seeking behaviour, treatments received and costs of care seeking. Pictures of locally available drugs for common childhood illnesses were used to increase the accuracy of recall of treatments received for sick children. The questionnaires were extensively field tested and delivered in four local languages. To ensure consistency, 10% of household interviews were repeated by field supervisors within a week of the original and discrepancies resolved. Economic data was collected from a range of sources on the costs of implementing the interventions to estimate the cost effectiveness.
CHW motivation and social identification measurement scales were adapted for use with CHWs based on formative research and theory from the literature [6,17]. Likert scale items relating to motivation were developed and social identification items were adapted for the Ugandan CHW setting [18,19]. The validation process for both scales is published [6] and briefly described in the ‘statistical analysis’ section below.
Randomisation and masking
Restricted randomisation was conducted by the study statistician (SS) and occurred after baseline data collection using a programme written in Stata 13 (StataCorp, Texas USA) [20]. The restricted randomisation ensured that between arms at baseline, there was i) no more than a 5 percentage point difference in appropriate treatment for MDP (separately), ii) no more than a 0.5 unit difference in mean log cost of care seeking for a sick child and iii) no more than a 0.5 unit difference in average CHW motivation score [14].
Interventions
All CHWs across interventions and control sites had previously been trained in iCCM and were equipped with commodities by the Ministry of Health with support from Malaria Consortium; training and commodity provision was identical across arms. iCCM training enabled CHWs in all sites to diagnose, treat and refer sick children presenting with symptoms of MDP using artemisinin combination therapy, oral rehydration salts (ORS) plus zinc or amoxycillin. The interventions were designed based on formative research and social identity theory and have been previously described in detail and are outlined in Table 1 [6,14,19].
In brief, the mHealth intervention encompassed the provision of inSCALE branded phones and solar chargers to CHWs in the mHealth clusters. There were three intervention components: 1) Closed user groups allowing free mobile phone communication among CHWs, and between CHWs and their supervisors; 2) weekly CHW data submission using phones with instant automated performance related feedback, messages to supervisors flagging problems for targeted supervision, and summary statistics made accessible online to district statisticians; and 3) monthly motivational SMS to CHWs with performance improvement tips on good sick child management, and to CHW supervisors on good supervision practices. The overall aim was to promote quality care and to increase CHWs confidence and connectedness to the wider health system using real-time data.
Sixteen Master Trainers trained 49 ‘trainers-of-trainers’ to deliver over 50 three-day courses to 1,277 CHWs and their 37 health facility supervisors on the mHealth intervention. This cascade model used existing health system training structures and personnel. CHW supervisors were trained on targeted supportive supervision techniques and on how to solve technical problems with hardware and software. CHWs and their supervisors were provided with troubleshooting guides. MOH (Ministry of Health) district biostatisticians were trained to download, review, and respond to data submissions from the CHW phones to detect drug stock-outs or unusual data trends.
The community engagement intervention took the form of village health clubs (VHCs) as a platform for participatory and locally owned identification of health problems and solutions, with the CHW as its focal point [6]. The VHCs utilised a four-stage participatory learning and action cycle, where the CHW facilitating the clubs engaged community members to meet monthly to: 1) identify, discuss, and rank local child health challenges, 2) discuss and plan solutions, 3) take planned actions to meet identified challenges, and 4) monitor, report and communicate on their progress. Guiding principles of VHC implementation were that they be open to all in the community, be fun and focused on community health improvement through utilisation of local assets with a particular focus on the CHWs.
In total, eight master trainers trained 39 trainers to deliver 45 CHW training sessions with 884 CHWs (two per village). Training involved adult learning approaches and outlined the VHC approach and materials. Each CHW was equipped with a flipbook and a set of child illness picture cards to facilitate question and answer sessions; a facilitator’s starter kit of membership cards, stationery, certificates, and t-shirts; and evaluation forms and attendance registers. Peer to peer learning meetings with other CHWs took place at five and nine months to encourage early adoption sites to share experiences with late adopters on best practices in forming and running the VHCs.
CHWs providing iCCM based care in the control area were provided with regular refresher trainings according to the standard national CHW programme curriculum, including quarterly meetings in health facilities.
Outcomes
The primary outcome was the percentage of illness episodes (MDP) treated with the appropriate medicine according to WHO informed national guidelines in children aged 2 months to 5 years. Full definitions of suspected MDP and their treatments are provided in S2 File. Secondary outcomes were: i) the percentage of sick children taken to a CHW (first port of call, or at any point for the most recent illness); ii) the percentage of children referred to the next level of care due to stock-outs of essential items at the CHW; iii) mean CHW motivation and social identification scores; and iv) mean CHW clinical knowledge score. Additional trial outcomes were v) the percentage of children with fever who received a rapid diagnostic test for malaria; vi) the percentage of MDP episodes treated both appropriately and within a day of illness onset; vii) mean sub-domain scores for CHW motivation and clinical knowledge; and viii) appropriate treatment of a sick child or “whole child analysis”, where each child was categorised as appropriately treated if they received the correct treatment for all reported episodes of MDP occurring the last 4 weeks and not appropriately treated if they were not correctly treated for one or more episode occurring in that period.
Sample size estimation
Sample sizes were based on the primary outcome, namely appropriate treatment of cases with suspected malaria, diarrhoea, and pneumonia. Procedures for calculating sample size are described in full in the study protocol paper [14]. In brief, assuming a control group percentage of appropriate treatment of 47% episodes of MDP and a coefficient of variation of 0.17 (values estimated from baseline data) gave us 90% power to detect a 15% change in appropriate treatment as a result of either intervention, sampling 155 children in each of 39 clusters. This sample size would also enable us to detect changes of 15%-25% in appropriate treatment of reported malaria, diarrhoea, and pneumonia individually with power of at least 80% [14].
Statistical analysis
All analyses were intention-to-treat and accounted for the cluster-randomised design. This was achieved by inclusion of the variables on which the randomisation was restricted at baseline as fixed effects (appropriate treatment for MDP, health worker motivation and household log cost of care seeking), and specifying random-intercepts for sub-county cluster units for either binary or continuous outcomes.
For binary outcomes odds ratios were converted to relative risks (RR) using the marginal standardisation technique, and the 95% CIs of the RRs estimated with the delta method [21]. Sensitivity analyses of appropriate treatment (primary outcome) were conducted to account for alternative treatment definitions and statistical models (difference-in-differences and adjustment for baseline variables used in restricted randomisation). Analysis of the trial was conducted initially on a version of the results dataset which masked sub-county allocation; once analysis methods were agreed by the study team, a final unmasked impact analysis was performed.
To assess the impact of the interventions on CHW motivation and social identification, scores were generated from the validated CHW motivation and social identification measurement scales. To validate the scales, qualitative, psychometric factor analysis, and item-reduction methods were applied. These methods have been published elsewhere [6]. This analysis resulted in CHW work motivation scores estimated based on a three-factor scale (general motivation, retention in role and reward for effort) and social identification scores estimated based on a single factor, four item scale that explained the majority of variation in social identification between CHWs. The items in the final CHW work motivation (α = 0.67) and social identification (α = 0.69) tools showed good internal consistency.
The standardised CHW clinical knowledge score was created as the weighted sum of each correct procedure mentioned by the CHW in response to a scenario. Specific questions and individual weights are provided in the S1 File.
Process evaluation
In February and March 2013, a mixed methods process evaluation with 1874 self-administered quantitative CHW checklists and 64 qualitative key informant interviews with CHWs, their supervisors and district leaders were conducted. For the mHealth arm, questions included the number of CHWs who could make calls in the closed user groups, submit weekly data, received monthly motivational SMS, feedback messages after data submission, or phone calls from their supervisor. For the VHCs arm, questions included number of villages that had formed, and were still running a VHC, number and frequency of VHC meetings held and VHC attendance, and reasons for low or slow adoption of VHCs. Open ended questions were asked on attitudes about and perceptions of the interventions.
The aim was to understand intervention uptake and perceived impact, the mechanisms behind CHW motivation and social identification with the CHW collective, and how and why the interventions worked or did not work. Quantitative data were entered in Excel and analysed descriptively using frequencies. Qualitative interviews were digitally audio-recorded, transcribed verbatim into English, and manually coded and analysed using a thematic content analysis approach. Intervention fidelity is presented below; full details of the qualitative component of the process evaluation have been published elsewhere [6].
Ethics
The inSCALE trial, registered as NCT01972321, was approved by Makerere University Institutional Review Board, the Uganda National Council of Science and Technology (ref. HS 958), and London School of Hygiene & Tropical Medicine Ethics Committee in the UK (ref. 5762). Oral consent for the random allocation of districts to intervention or control groups was obtained from the sub-county leadership. Individual written informed consent was obtained from the caregiver for baseline and follow up data collection after explaining the purpose of the interview. Participants were free to decline the interview at any time.
Results
Fig 1 shows the trial profile. Baseline data showed that there was balance between arms for key demographic characteristics, including primary caretaker type, age of respondent, education level completed, marital status, job type, and religion, with some differences found in ethnicity and mother tongue spoken, of which neither were seen to be related to the outcome (Table 2). Baseline appropriate treatment rates of diarrhoea, pneumonia and malaria were not significantly different in the three arms. Roughly 47% of children with any episode of MDP received appropriate treatment (from any provider) during their illness.
At follow-up, 7679 households were surveyed, and 3413 households were found to have at least one child under 5 years. From these households, caregivers were asked about illness episodes for a total of 5573 children aged 2 months to 5 years; 1858 in the control arm, 1858 in the community engagement arm and 1856 in the mHealth arm. Out of these children, 946 (control arm), 979 (VHC arm) and 881 (mHealth arm) had had MDP symptoms in the last one month, representing 51%, 53% and 48% respectively (Table 3). Background characteristics of the children with MDP at follow-up is outlined in Table 4.
Table 5 summarises the analyses of the primary outcomes for the two interventions. Sick children in the mHealth arm were significantly more likely to receive appropriate treatment for MDP compared with those in the control arm (RR 1.10; 95% CI 1.01, 1.20; p = 0.035). Though not statistically significant, the largest effect size was seen on appropriate treatment for diarrhoea, with a risk ratio of 1.39 for providing ORS (95% CI 0.90–2.15; p = 0.085) and 1.28 for providing ORS plus zinc (95% CI 0.70, 2.36; p = 0.424). There was an 8% increase in appropriate treatment in the VHC intervention arm for appropriate treatment of MDP in children, though again this did not reach statistical significance at the 5% level (RR 1.08; 95% CI 1.00, 1.18; p = 0.056). As with the mHealth arm, the largest effect was seen in appropriate treatment for diarrhoea, with a significant risk ratio of 1.54 (95% CI 1.12, 2.12; p = 0.007). These findings are consistent with the results of a “whole child” concurrent symptoms analysis (S3 File) where children were categorised as appropriately treated only if all their symptoms were treated correctly (i.e., if they had one or more of MDP concurrently).
The inSCALE interventions did not increase community utilisation of the CHW as a care provider for MDP, or care seeking in general (Table 5). However, care-seeking from CHWs increased over time across all three arms; compared to baseline the CHW care seeking in the control arm increased by nearly 15% points at endline. The increase in CHW care seeking was less in the VHC arm than in the mHealth and control arms (S4 File).
No differences were noted in work motivation or knowledge of CHW following the introduction of inSCALE interventions, and CHW work motivation and social identification was notably high across all arms (Table 6). However, the rate of CHW attrition in both intervention arms was less than half that of the control arm over the 18 months period from initial CHW hiring and training (December 2012) to the end of follow-up (April-June 2014) after adjusting for baseline differences: 7.8% attrition rate in the control arm compared to 3.2% in the VHC arm (adjusted risk difference: -4.75%, 95% CI -8.74, -0.76, p = 0.021) and to 4.6% in the mHealth arm (adjusted risk difference: -4.42%, 95% CI -8.54, -0.29, p = 0.037).
CHWs provided the highest level of appropriate treatment across care providers in all arms of the study (Table 6) e.g., CHW’S provided on average 18% higher rates of appropriate treatment compared to private providers across arms. However, though unexpected, it is important to note that, the overall proportion of children with MDP appropriately treated by a CHW in the mHealth arm (RR 1.05; 95% CI 0.96, 1.16; p = 0.286) and VHC arm (RR 1.04; 95% CI 0.94, 1.14; p = 0.481) did not differ significantly with children treated by CHWs in the control arm. In contrast, there was a trend towards more children receiving appropriate treatment in the intervention arms for most providers apart from general shops or herbalists; though this only reached significance for children taken first to pharmacies in the mHealth arm (Table 7). Children in the VHC arm treated in private facilities were more likely to receive appropriate treatment than in private facilities in the control arm.
Based on CHW responses, intervention fidelity was moderate, with the majority of CHWs in the mHealth arm (83%) still had a functioning phone at endline survey, but only 38% had submitted reports in the last month. In the VHC arm, 94% of villages had held at least one VHC meeting and 81% of villages having organised more than three meetings in the past year (Table 8). Monitoring and process evaluation data showed that several software and hardware problems occurred in the mHealth arm during the implementation, limiting the reporting frequency. However, following system rectification half-way through the implementation 60% of CHWs were able to submit their reports on a weekly basis. In addition, most of the other intervention components, such as the closed user groups and monthly SMS feedback messages, were functional for most users throughout the implementation period. While it was anticipated that CHWs reporting on drug stock levels in the technology arms may encourage re-distribution between CHWs to reduce stock-outs, national CHWs policies prevented health facilities to act on to drug stock information.
Discussion
This is the first study that has implemented and evaluated the impact of mHealth and community engagement village health clubs (VHCs) to support delivery of iCCM. Our results suggest that the mHealth intervention increased appropriate treatment for malaria, diarrhoea, and pneumonia (MDP) compared with children in the control arm. However, contrary to our hypothesis, the mechanism of this improvement was not directly through the CHWs; rather the improvement seen in the mHealth intervention was due to increased levels of appropriate treatment at public health facilities and pharmacies. In a recent review of the use of digital health for strengthening health systems, Long et al (2018) [11] showed that solutions for health worker training, provider-to-provider communication and professional networking, and supervision of and performance feedback to health workers may lead to better health workforce development. While the body of evidence on mHealth is increasing, leading to WHO publishing its first global recommendations on digital interventions for health system strengthening [8], few studies have to date been able to document an effect on health outcomes [9,22]. Our robust evaluation, using a randomised controlled trial, is therefore one of the very few that have shown a statistically significant impact of an mHealth intervention on appropriate treatment of sick children.
The VHCs also increased appropriate treatment of MDP in children by 8%, though the improvement did not reach statistical significance at the 5% level. However, sub-group analysis showed that children in the VHC arm were more often treated appropriately for diarrhoea with ORS than those in the control group, and the effect was large (56%) and statistically significant. The only other study published on the effect of village health clubs on child health outcomes showed no effect on diarrhoea in children in Rwanda [23]. In our study, there was minor difference in diarrhoea prevalence between arms but there was a shift in providers sought when children were ill, with more children in the VHC arm being taken to public lower-level facilities compared to in the other arms.
While we cannot conclude that the two inSCALE interventions improved use of CHWs by community members, nor appropriate treatment provided by CHWs, care seeking from CHWs did increase over time across all three arms. This effect was less prominent in the VHC arm, as more children in this arm resorted to public health facilities. It is difficult to explain what caused the documented positive effects on appropriate treatment of sick children, as neither of the CHW focused interventions seem to have increased knowledge, motivation or appropriate treatment provided by CHWs directly. Challenges related to the iCCM programme (e.g. irregular drug supply, low CHW literacy and visual impairment) as well as to the specific interventions (e.g. lack of network connectivity, software and hardware maintenance, and partnerships with software partners in the mHealth arm, and lack of interest from the community, inadequate stationery supplies, and insufficient support from local leaders in the VHC arm) likely limited the impact of the interventions [24,25]. However, it is possible that these types of health systems strengthening interventions, delivered through cascade training using facility based health workers, sensitisation and support to health facility supervisors and other district stakeholders, have led to general system improvements; the exact nature and mechanism of this improvement needs further exploration in different health contexts and implementation studies. From the caregiver interviews during the process evaluation, caregivers in the VHC arm frequently mentioned that the VHC sessions have taught them that pneumonia and diarrhoea are potentially severe conditions that need urgent care, which could explain why more caregivers resort to health facilities in this arm.
In 2016 WHO released the recommendation on implementation of community mobilization through facilitated participatory learning and action (PLA) cycles with women’s groups to improve maternal and newborn health, particularly in rural settings with low access to health services [26]. In line with findings from our study, the report state that the pathways of influence on outcomes are difficult to assess, but that that women meeting to identify their needs and seek solutions plays a vital role; other activities related to the solutions identified at the meetings may also play a role. Three years later, in 2019, WHO released a guideline on recommendations on digital interventions for health system strengthening [8]. Three of the nine evidenced based digital interventions recommended were partially embedded in our mHealth arm, namely ‘notification of stocks and commodities’ using mobile devices to communicate using e.g. text messaging (SMS) and data dashboards to manage and report on supply levels, ‘digital provision of training and educational content to health workers’ using channels such as SMS text messaging, and ‘provider-to-provider telemedicine’ where less skilled health workers are linked with more specialist ones for a variety of reasons, including to get assistance with diagnoses and to conduct case-management consultations. As also concluded in our study, the guideline points out that researchers should be realistic about the extent to which digital health interventions can impact on distal health outcomes, which are often affected by a variety of factors beyond the interaction with the digital intervention, including access to medicines, their cost, family support, and biomedical factors. It is therefore suggested that health system focused digital interventions, with large complexity in the number of components, behaviours targeted, and organizational levels involved, may make designs such as randomized controlled trials for evaluating the effectiveness of these interventions difficult to apply, and other designs may therefore need to be considered [8].
CHW work motivation and social identification, which had not previously been measured in low-income settings, were high across and within intervention arms. High reported motivation is consistent with recent results in Uganda and other comparable settings [27], though may have been subject to social desirability bias despite attempts during scale development to avoid this. Elsewhere it has been suggested that CHW motivation should only be measured when adequate working conditions, such as, appropriate supervision and supply of drugs have been provided [28]. While these were controlled for in the design of the inSCALE trial, in practice they were not always in place. Potentially the interventions led to a CHW perception that adequate working conditions had been provided in intervention areas which contributed to the lower attrition rates. High reported CHW social identification, indicating shared goals and commitment to the collective, suggest that further investigation of the performance priorities of an apparently motivated CHW workforce utilising this theoretical lens has explanatory promise.
Our study limitations included the delivery of the interventions in only one area of Uganda limiting study generalisability across contexts. While contamination between study arms is possible, it was likely minimal, given that community members tend to use the CHW who they have elected for their community. In addition, the implementation, though working with the Ministry of Health, was heavily supported by Malaria Consortium, an experienced and highly motivated NGO. Evaluation of similar interventions delivered solely by government staff would be useful to consider what it would take to sustain the interventions at scale. Appropriate treatment was assessed by parental recall, as previously done in national and international surveys [15,16], rather than objective notes audit. This method was the only feasible data collection method due to the location and scale of the project. The delay in publication of these findings was due to organisational dispersal of the original study team. However, the size and robust nature of the study makes it relevant especially with the need for mobilisation and strengthening of the health workforce in Sub-Saharan Africa during the current COVID-19 pandemic.
The inSCALE interventions increased appropriate treatment of children with MDP in Uganda and reduced CHW attrition rate by half. CHW delivered iCCM resulted in encouragingly elevated levels of appropriate treatment of childhood MDP across intervention and control arms. The improvements observed in the intervention arms occurred despite programmatic challenges such as drug stockouts and irregular supervision; barriers which these interventions could not overcome. Alternative pathways, such as care seeking from public and private health facilities, was the main contribution to the improvement. We recommend that future studies aiming to evaluate drivers of CHW performance and motivation further try to unpack the mechanisms of action, as well as the care seeking pathways among caregivers who did and did not seek care from a CHW, to understand their motivations. We also see a lot of value in evaluating a combination of aspects from both the VHC and mHealth intervention, as these approaches might have complementary benefits. A combined VHC-mHealth intervention would also more closely reflect the reality of today’s CHW programmes, which increasingly adopt digital health tools for CHW task management, data reporting and eLearning, in combination with community mobilisation activities to optimise CHW management.
Supporting information
S1 File. Scoring instrument for inSCALE CHW Performance.
https://doi.org/10.1371/journal.pdig.0000217.s001
(DOCX)
S2 File. Illness and appropriate treatment definitions for fever, diarrhoea and pneumonia.
https://doi.org/10.1371/journal.pdig.0000217.s002
(DOCX)
S3 File. Appropriate treatment results based on whole child analysis.
https://doi.org/10.1371/journal.pdig.0000217.s003
(DOCX)
S4 File. Baseline-endline change in appropriate treatment and CHW care seeking across inSCALE arms (difference-in-difference analysis).
https://doi.org/10.1371/journal.pdig.0000217.s004
(DOCX)
Acknowledgments
We want to thank the research assistants who collected the data, the inSCALE team Uganda and the participants of the study for their time and insight.
References
- 1. You D, Hug L, Ejdemyr S, Idele P, Hogan D, Mathers C, et al. Global, regional, and national levels and trends in under-5 mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN Inter-agency Group for Child Mortality Estimation. Lancet. 2015;386(10010):2275–86. pmid:26361942
- 2. Roberton T, Carter ED, Chou VB, Stegmuller AR, Jackson BD, Tam Y, et al. Early estimates of the indirect effects of the COVID-19 pandemic on maternal and child mortality in low-income and middle-income countries: a modelling study. Lancet Glob Health. 2020;8(7):e901–e8. pmid:32405459
- 3. Young M, Wolfheim C, Marsh DR, Hammamy D. World Health Organization/United Nations Children’s Fund joint statement on integrated community case management: an equity-focused strategy to improve access to essential treatment services for children. Am J Trop Med Hyg. 2012;87(5 Suppl):6–10. pmid:23136272
- 4. Prosnitz D, Herrera S, Coelho H, Moonzwe Davis L, Zalisk K, Yourkavitch J. Evidence of Impact: iCCM as a strategy to save lives of children under five. J Glob Health. 2019;9(1):010801. pmid:31263547
- 5. Hazel E, Bryce J, The IIP-JHU iCCM Evaluation Working Group. On Bathwater, Babies, and Designing Programs for Impact: Evaluations of the Integrated Community Case Management Strategy in Burkina Faso, Ethiopia, and Malawi. The American journal of tropical medicine and hygiene. 2016;94(3):568–70. pmid:26936991
- 6. Strachan DL. Ugandan Community Health Worker Motivation: Using the Social Identity Approach to Explore an Accepted Constraint to Scaled up Health Strategies. Doctoral thesis (Ph.D), London: UCL (University College London); 2019. Available from: https://discovery.ucl.ac.uk/id/eprint/10073256/.
- 7. Kallander K, Tibenderana JK, Akpogheneta OJ, Strachan DL, Hill Z, ten Asbroek AH, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low- and middle-income countries: a review. J Med Internet Res. 2013;15(1):e17. pmid:23353680
- 8.
WHO. WHO guideline: recommendations on digital interventions for health system strengthening. Geneva: World Health Organization: 2019. Available from: https://www.who.int/publications/i/item/9789241550505.
- 9. Zakus D, Moussa M, Ezechiel M, Yimbesalu JP, Orkar P, Damecour C, et al. Clinical evaluation of the use of an mhealth intervention on quality of care provided by Community Health Workers in southwest Niger. J Glob Health. 2019;9(1):010812. pmid:31263555
- 10. Kaphle S, Matheke-Fischer M, Lesh N. Effect of Performance Feedback on Community Health Workers’ Motivation and Performance in Madhya Pradesh, India: A Randomized Controlled Trial. JMIR Public Health Surveill. 2016;2(2):e169. pmid:27927607
- 11. Long LA, Pariyo G, Kallander K. Digital Technologies for Health Workforce Development in Low- and Middle-Income Countries: A Scoping Review. Glob Health Sci Pract. 2018;6(Suppl 1):S41–S8. pmid:30305338
- 12. Strachan DL, Källander K, ten Asbroek AH, Kirkwood B, Meek SR, Benton L, et al. Interventions to improve motivation and retention of community health workers delivering integrated community case management (iCCM): Stakeholder perceptions and priorities. American Journal of Tropical Medicine and Hygiene. 2012;87(5):111. pmid:23136286
- 13. Rosato M, Laverack G, Grabman LH, Tripathy P, Nair N, Mwansambo C, et al. Community participation: lessons for maternal, newborn, and child health. Lancet. 2008;372(9642):962–71. pmid:18790319
- 14. Kallander K, Strachan D, Soremekun S, Hill Z, Lingam R, Tibenderana J, et al. Evaluating the effect of innovative motivation and supervision approaches on community health worker performance and retention in Uganda and Mozambique: study protocol for a randomised controlled trial. Trials. 2015;16(1):157. pmid:25873093
- 15. USAID. Demograhic Health Surveys Programme. 2017 [cited 12 September 2021]. Available from: http://www.dhsprogram.com.
- 16. UNICEF. Multiple Indicator Cluster Surveys. 2017 [cited 12 September 2021]. Available from: http://mics.unicef.org/tools.
- 17. Franco LM, Bennett S, Kanfer R, Stubblebine P. Determinants and consequences of health worker motivation in hospitals in Jordan and Georgia. Soc Sci Med. 2004;58(2):343–55 pmid:14604620
- 18. Postmes T, Haslam SA, Jans L. A single item measure of social identification: Reliability, validity, and utility. British Journal of Social Psychology. 2013;52(4):597–617. pmid:23121468
- 19. Frank T, Källander K. Developing intervention strategies to improve community health worker motivation and performance. Malaria Consortium. 2013 [cited 20 Feb 2022]. Available from: https://www.malariaconsortium.org/resources/publications/167/developing-intervention-strategies-innovations-to-improve-community-health-worker-motivation-and-performance.
- 20. Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clinical trials. 2004;1(3):297–305. pmid:16279255
- 21. Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007;60(9):874–82. pmid:17689803
- 22. Amoakoh HB, Klipstein-Grobusch K, Agyepong IA, Zuithoff NPA, Amoakoh-Coleman M, Kayode GA, et al. The effect of an mHealth clinical decision-making support system on neonatal mortality in a low resource setting: A cluster-randomized controlled trial. EClinicalMedicine. 2019;12:31–42. pmid:31388661
- 23. Sinharoy SS, Schmidt WP, Wendt R, Mfura L, Crossett E, Grepin KA, et al. Effect of community health clubs on child diarrhoea in western Rwanda: cluster-randomised controlled trial. Lancet Glob Health. 2017;5(7):e699–e709. pmid:28619228
- 24. Frank T, Marks S, Källander K. Establishing Village Health Clubs to improve community health worker motivation and performance. Malaria Consortium. 2015. [cited 20 Feb 2022]. Available from: https://www.malariaconsortium.org/resources/publications/628/establishing-village-health-clubs-to-improve-community-health-worker-motivation-and-performance.
- 25. Frank T, Marks S, Källander K. Implementing mHealth solutions to improve community health worker motivation and performance. Malaria Consortium. 2016. [cited 20 Feb 2022]. Available from: https://www.malariaconsortium.org/media-downloads/878/Implementing%20mHealth%20solutions.
- 26. World Health Organization. WHO Recommendation on Community Mobilization through Facilitated Participatory Learning and Action Cycles with Women’s Groups for Maternal and Newborn Health. 2014. [cited 20 Feb 2022]. Available from: https://apps.who.int/iris/handle/10665/127939.
- 27. Sanou AK, Jegede AS, Nsungwa-Sabiiti J, Siribie M, Ajayi IO, Turinde A, et al. Motivation of Community Health Workers in Diagnosing, Treating, and Referring Sick Young Children in a Multicountry Study. Clin Infect Dis. 2016;63(suppl 5):S270–S5. pmid:27941104
- 28. Perry HB, Zulliger R, Rogers MM. Community health workers in low, middle, and high-income countries: an overview of their history, recent evolution, and current effectiveness. Annual review of public health. 2014;35:399–421. pmid:24387091