End-user perceptions and stakeholder experiences of implementing a mobile health application in Ekurhuleni Health District, South Africa

Introduction

Global and national context of digital health

Digital health, defined by the World Health Organization (WHO) as the application of digital technologies to improve health, has emerged as a critical component of health system strengthening worldwide. Building on the earlier concept of eHealth, mobile health (mHealth) refers to the use of portable electronic devices and software applications to deliver health services and manage patient data.^1,2 The potential of these tools to improve access, quality, and efficiency of healthcare delivery has been demonstrated across multiple settings, including low- and middle-income countries where resource constraints often limit health system performance.^3,4

Globally, the adoption of mHealth accelerated in the early 2000s alongside the expansion of mobile phone ownership and internet connectivity. By 2023, more than three-quarters of the global population owned a mobile phone, underscoring the ubiquity of this platform for public health interventions.⁵ International policy momentum, such as the WHO’s Global Strategy on Digital Health 2020–2025 and the 2018 World Health Assembly Resolution on Digital Health, has reinforced the role of digital technologies in achieving universal health coverage and the Sustainable Development Goals.¹

In South Africa, discourse on digital transformation is closely tied to the Fourth Industrial Revolution Agenda first articulated by President Cyril Ramaphosa in 2018, and the adoption of the National Digital Health Strategy 2019–2024.^2,6 This strategy envisions a unified digital health ecosystem, linking people, processes, and technology to strengthen service delivery within a proposed National Health Insurance framework, contributing towards universal health coverage.² Despite these aspirations, significant implementation challenges persist, including fragmented governance, limited investment in digital infrastructure, high market-driven broadband costs, and lack of interoperability across platforms.²

A number of digital health interventions have nonetheless reached national scale in South Africa, such as MomConnect, which provides maternal health messaging, and mortality audit tools such as the Perinatal Problem Identification Programme and the Child Healthcare Problem Identification Programme.⁷ The COVID-19 pandemic further accelerated digital platforms for disease surveillance, communication, and service delivery through initiatives such as DatCov, the Electronic Vaccination Data System, and the COVID-19 Connect WhatsApp helpline.^8,9 These examples highlight the expanding role of digital health in both routine health services and emergency response.

With this increased use of digital health interventions, it is important to evaluate their implementation and analyse the facilitating factors that sustain them. This will contribute to scale-up and sustainability, while also informing better resolution of potential implementation barriers. Limited published documentation exists of district-level implementation of mHealth interventions in South Africa. Most studies focus on feasibility and user acceptability without adequately addressing issues of scale-up, system integration and sustainability.^7,10,11

The implementation of CommCare

Through a partnership between the National Department of Health and the Aurum Institute, Ekurhuleni Health District, an urban metropolis in South Africa, was selected in 2017 as a national pilot site for an mHealth application designed to support ward-based primary health care outreach teams (WBPHCOTs).¹² This digital tool aimed to replace paper-based data collection and enhance the ability of community health workers (CHWs) and outreach team leaders (OTLs) within WBPHCOTs to conduct household screening, referrals, follow-up, and tracing activities.

The Aurum Institute had served as a longstanding district support partner within Ekurhuleni for over a decade and subcontracted Dimagi, a software development company, to assist in the customisation of their open-source, cloud-based platform called CommCare for a suitable mHealth application.^13,14 Developed by Dimagi and implemented in partnership with the Aurum Institute, the CommCare pilot was initially planned for one year but extended over five years at district request, offering a unique opportunity to assess both short- and medium-term experiences of end-users and stakeholders, between 2017 and 2022 (Figure 1). During this period, the Aurum Institute also subcontracted an IT services company, CosmoNet, to maintain internet connectivity and supply approximately 1 000 mobile devices for the digital health intervention (P. Motshele, oral communication, September 27, 2022).

A cascade model¹⁵ of training was implemented for the mHealth project rollout. Using this approach, master trainers first trained OTLs, who were then responsible for training the CHWs and facility users (T. Ramabale, oral communication, November 23, 2023).

Figure 1.Implementation process of CommCare mHealth project in Ekurhuleni, 2017–2022

This study assesses the perceived usefulness and end-user satisfaction of the CommCare app. It identifies key factors for scaling and sustainability by exploring key stakeholders’ experiences, including facilitators, barriers, and lessons learnt from its five-year implementation in Ekurhuleni.

Method

Study design

This study employed a mixed-methods cross-sectional descriptive design. Semi-structured interviews were conducted with key stakeholders, and an online self-administered questionnaire was designed for end-users (CHWs and OTLs).

Study setting

The study was conducted in the Ekurhuleni Health District (Gauteng), which is organised into three sub-districts (North, East, and South). The district is a large urban area with a population of approximately 3 774 368 people, with multiple primary health care facilities and 178 WBPHCOTs comprising 1 027 CHWs and 170 OTLs.¹⁶

Study population and sampling

No sampling was undertaken for this study. Instead, all key stakeholders responsible for implementing CommCare and end-users who utilised the mHealth app were invited to participate in the study. The study population thus consisted of two informant groups comprising 1 205 participants (Table 1).

The contracted service provider, CosmoNet, was not directly involved in implementation but was subcontracted to procure and configure the handsets and manage device connectivity for the CommCare rollout. Attempts to interview CosmoNet were unsuccessful, reducing the number of key informants to seven.

Conceptual approach and measurement framework

Data collection tools were developed using an adapted conceptual framework combining domains from the non-adoption, abandonment, scale-up, spread, sustainability (NASSS) framework, the Technology Acceptance Model and Expectation–Confirmation Theory.^17–19 The NASSS framework is useful for studying the non-adoption and abandonment of digital health solutions by end-users, their ability to be embedded in a health system, and the challenges to scale-up and sustainability. The framework identifies seven domains of importance: the disease/condition, technology, value proposition, adopters, organisation, wider system environment such as the political context, and evaluation.¹⁷ These domains were explored during the semi-structured interviews with key informants.

To assess the domain of the adopters/end-users and to ascertain their perceived usefulness and satisfaction with the app, elements of the Expectation–Confirmation Theory¹⁹ and Technology Acceptance Model¹⁸ were used.

Expectation–Confirmation Theory is a marketing theory used to measure the perceived usefulness and satisfaction of end-users following use of a product. It measures whether a product met pre-conceived expectations and whether users were satisfied with it during its consumption. It also measures consumer repurchase intention.¹⁹

The Technology Acceptance Model is a conceptual framework that explains and predicts how users accept and use technology. It is based on three main pillars: system features and capabilities, user motivation, and measurable outcomes of system use. User motivation is influenced by three key factors: (1) perceived usefulness: the extent to which users believe the technology will improve performance or simplify tasks; (2) perceived ease of use: the extent to which users perceive the technology as easy to use; and (3) behavioural intention: the user’s intention to use the technology, influenced by its perceived usefulness and ease of use, and ultimately determining actual use.¹⁸

Combined, these frameworks guided the constructs, questionnaire sections and interview domains (Figure 2).

Figure 2.Adapted conceptual framework from literature^17–19

Ethics

Ethical clearance was obtained from the Human Research Ethics Committee at the University of the Witwatersrand (M230425). Gatekeeper permission was also received from the Ekurhuleni Health District to conduct this research. Written (electronic) informed consent was obtained before participation; additional verbal consent was recorded for interviews. Participants were informed of voluntary participation, confidentiality safeguards, and the right to withdraw without penalty.

Data collection

End-user survey

An online self-administered questionnaire link was distributed via WhatsApp to all OTLs and CHWs in September 2023 and remained open for two months. The questionnaire, worded in English, comprised demographic items and nine questions developed from the adapted conceptual framework. Closed-ended questions used categorical response options (Yes/No/Neutral) to assess expectations, adequacy of training, satisfaction, challenges, ICT support, and intention to recommend CommCare. Open-ended questions explored participants’ initial expectations, challenges experienced, and suggestions for improving CommCare. Quantitative responses were summarised using percentages while responses to the open-ended questions were grouped thematically. The questionnaire was administered after the five-year implementation of CommCare (2017–2022). Duration of app use by individual participants was not separately captured.

Key stakeholder interviews

Stakeholders were approached by email in September 2023 and interviews were conducted during September–October 2023, primarily via Microsoft Teams. Interview guides were derived from the adapted conceptual framework with a focus on the domains of technology, organisation, the wider system, and best practices.

Data management and analysis

Data from the online questionnaire were de-identified and aliases were used for the interviews and transcripts. Survey responses were then analysed to summarise perceived usefulness, satisfaction, and reported challenges among end-users. Semi-structured interviews were conducted and recorded by the principal investigator. The interviews were transcribed, and then were reviewed multiple times to ensure immersion and understanding.

A hybrid deductive–inductive thematic analysis using the Framework Method was conducted.^20,21 Pre-specified domains derived from the conceptual frameworks informed the initial codebook. Interview transcripts were coded in Atlas.ti with line-by-line inductive coding to capture unanticipated concepts. Themes were then refined to ensure coherence, depth, and alignment with the study objectives.

Rigour and trustworthiness

Tool development was anchored in an established multi-framework approach; data source triangulation (stakeholder interviews and end-user survey) supported convergence on major themes. A coding framework derived from the adapted model enhanced analytic consistency, with iterative refinement to incorporate inductive findings.

Results

Perceived usefulness and end-user satisfaction amongst OTLs and CHWs using CommCare

Biographical information

A total of 464 out of the 1 197 end-users responded (39%). Participants were mostly female (94%); 39% had completed Matric and a further 39% had completed some secondary schooling. Over half (55%) were aged 40 or older (Table 2).

Initial expectations of CommCare among end-users

OTLs and CHWs reported clear expectations that CommCare would streamline workflows, strengthen follow-up and tracing processes, support CHW supervision and monitoring by OTLs and improve data management through safer storage of patient information and easier integration into the District Health Information System (DHIS). End-users further viewed the application as an opportunity to improve their digital and technical skills. These expectations suggest that participants perceived CommCare not merely as a data-capture tool, but as a mechanism to improve efficiency, supervision, accountability, and continuity of care within the WBPHCOT programme. The main themes extracted from the survey results are presented in Figure 3, and supporting quotes are recorded in Appendix 1a.

Figure 3.Themes and sub-themes derived from responses of CHWs and OTLs around their expectations of CommCare

Reported perceived usefulness and end-user satisfaction

Perceived usefulness was high: 83% of end-users reported that CommCare met their expectations, 85% expressed overall satisfaction, and 95% indicated they would recommend adoption in other districts.

Challenges experienced by OTLs and CHWs while using CommCare

Only 35% of end-users reported challenges using CommCare. The main challenges related to poor internet connectivity, limited screening tools, data management, hardware and software malfunctions, and inappropriate screening and referral prompts. Poor connectivity affected data reporting, while limited screening options reflected the restricted range of tools available. Inappropriate prompts included referrals that were not considered relevant to the client being assessed. Hardware and software challenges included application freezing and device malfunctions, while data management challenges related to difficulty in deleting erroneous or obsolete records. Some end-users stated safety concerns when using the device in certain areas. These findings suggest that despite high overall end-user satisfaction, infrastructural, usability, and contextual barriers continue to affect the day-to-day utilisation of the application. Representative themes are depicted in Figure 4 and supporting quotes are shown in Appendix 1b.

Figure 4.Themes derived from responses of CHWs and OTLs around their challenges with CommCare

Suggestions from OTLs and CHWs to improve CommCare

Approximately 47% of respondents provided suggestions for improving CommCare. These primarily focused on improving system functionality, usability, and inclusivity. For example, expanding CommCare’s screening tool set: “I believe I could change screening tool and add mental health, asthma, cancer, stroke, PAP smear and epilepsy”, (CHW 22); incorporating vernacular language options; and strengthening data management functions such as the ability to deactivate or remove records for households no longer under their care (e.g., after death or relocation). OTLs further requested more comprehensive narrative reports generated from the data and built-in functionality to support remote monitoring of the CHWs they supervise: “The [app] must allow us to see weekly work for the CHW or daily work”, (OTL 18). These findings suggest that end-users viewed CommCare as a valuable platform that requires continuous refinement to better align with the operational realities and diverse needs of frontline users.

Stakeholder perspectives: facilitators and barriers to district-level implementation

Stakeholder interviews (WBPHCOT sub-district coordinators, district ICT officials, and partner representatives) highlighted cross-cutting technological, organisational, and system-level factors (see Table 3 for an overview of themes).

Analysis of interview data revealed several implementation challenges across multiple domains, as reported by stakeholders and discussed below.

Technology and support

Participants reported that a difficult user interface and frequent technical failures, such as application freezing and unrepaired malfunctions, impeded regular use. Limited ICT support was noted for devices procured outside standard government tender: “They’ve [district ICT support] got so much work already … you may find we only have one IT technician in the entire sub-district so when they encounter problems with mHealth it’s not possible for them to help”, (Sub-district Coordinator 2). To address these issues, participants identified the appointment of mHealth coordinators to provide technical support to users at facilities and the use of developer-created troubleshooting videos as supportive measures.

System integration

Interview data indicated challenges in collaboration between facility-based and outreach teams: “…so you find that now there’s conflict between a facility manager and the team leader to say ‘we are busy’ and you are busy on the phone … and the phone is actually a working tool for the OTL”, (Sub-district Coordinator 1). These were characterised by unclear role delineation and resource boundaries across government levels, which at times resulted in teams lacking dedicated workspaces. The orientation of facility managers was reported by participants to have improved both buy-in and access to facilities.

Training and capacity

Logistical challenges in securing venues for large training cohorts were reported. Disparities in digital literacy among participants were also noted. Some stakeholders additionally suggested reassigning CHWs with limited digital skills to alternative activities better aligned with their skillsets: “We even suggested taking some CHWs who are struggling with technology to be reassigned as mentor mothers or queue marshals”, (Sub-district Coordinator 3). The cascade training model and the pairing of CHWs with differing skill levels were described as strategies used to mitigate these issues.

Oversight and monitoring

Remote monitoring was constrained by poor connectivity and rapid data depletion from devices. An imperfect initial mapping of CHWs and their OTLs was also reported. Furthermore, the absence of timestamps and the incomplete capture of activities reduced the visibility of fieldwork and the ability to check when CHWs were working.

Asset management

Participants reported widespread loss of gadgets, which prompted the implementation of stricter asset control policies. These policies included holding CHWs liable for negligent loss. Clearer reporting pathways for lost assets were reinforced through joint ICT and risk-management roadshows: “Since we’ve done that [roadshows], we’re seeing a big decrease in terms of reporting of lost gadgets”, (ICT Helpdesk Technician).

Best practices and lessons learnt for scale-up and sustainability

Stakeholders identified practical lessons for future digital health initiatives.

Capacity building

All interviewees identified the cascade model of training as a best practice for training the CHWs and OTLs on the use of CommCare: “The cascade model, I think made things easier because once the team leaders were trained, so we were able to train community health workers in different places at the same time”, (District Support Partner 1).

Multi-stakeholder engagement and governance

Early and continuous engagement among the district, sub-districts, facility managers, ICT, district support partners and community members was viewed as essential:

Involving community leaders when engaging stakeholders will be important for increasing the protection of CHWs when they are out in the community. If the community leader is aware of their presence, then they are protected and it decreases the chances of them losing their gadgets to crime while at work.

(Sub-district Coordinator 1)

Dedicated technical capacity and responsive support

Ongoing technical support, with district-level technicians trained on the app and devices, a ticketing workflow and agreed turnaround times, were seen as necessary for continuity: “We must not rely on district partners. They must train people with the WBOT programme to provide technical support”, (Sub-district Coordinator 2).

Infrastructure readiness

Some participants stated that government would need to internalise its mHealth programme by investing in and maintaining its own IT infrastructure and hosting software in order to achieve sustainability and successful scale-up of digital health interventions: “I think, overall, my view would be that internalising as much as possible … building all this capacity and creating space and building these teams to manage the programme within the NDOH”, (Sub-district Coordinator 1).

User interface and accessibility

Participants recommended adding vernacular language options to enhance end-user comprehension and a more intuitive interface so facility managers could access information with minimal training. They also proposed controlled cross-access to cases among paired CHWs or those within the same facility to support continuity of care when the assigned household CHW is unavailable.

Data quality and integration

Interviewees called for richer, standardised narrative reports with direct DHIS integration to reduce user-dependent, unreliable verification. They further recommended time-stamping all activities and explicitly recording tasks not performed to strengthen remote monitoring and identify service-delivery gaps.

Implementation of digital health interventions and governance alignment

Interviewees emphasised that scaling digital health required treating implementation as a stand-alone, resourced project. This would entail the establishment of government-level technical teams and allocation of human resources for training and support. Roll-out should be phased over longer periods with comprehensive stakeholder sensitisation: “… [it is important to look] at a digital rollout not as a small component of a large programme, but a programme in itself”, (District Support Partner 1).

Discussion

Implementation of the CommCare mobile health application in the Ekurhuleni Health District provides key insights into how digital tools can strengthen South Africa’s journey towards a learning health system and the realisation of the National Digital Health Strategy (NDHS). Findings from this study indicate that end-users perceived the digital intervention as streamlining workflows through improved data management, enhanced decision support and more efficient service delivery. However, long-term sustainability requires systemic readiness, digital infrastructure investment, capacity building, and governance across all levels of the health system.

End-user acceptance and system barriers to digital transformation

The high perceived usefulness and satisfaction reported by CHWs and OTLs demonstrate a foundational readiness for digital transformation at the frontline of care, a core principle of the NDHS. The fact that CommCare met expectations for streamlining workflows, reducing paperwork, and supporting clinical decision-making aligns with global evidence and confirms the value proposition of mHealth for community-based care.^22,23 This strong user buy-in is a vital asset for advancing the NDHS, as a willing and capable workforce is the first requirement for a learning health system.

These findings are consistent with experiences from other settings. A South African study similarly found that CHWs considered a mobile data collection and decision-support application useful, easy to use and supportive of their workflow.²⁴ In Guatemala, CommCare was associated with improved clinical management, while positive perceptions of mHealth applications for the improvement of data management were also reported in Ethiopia.^22,23,25 Although end-user buy-in at the frontline is necessary, it is just one condition for systemic change. The issues of poor connectivity, data depletion, software glitches, and lack of multilingual functionality are not merely operational inconveniences but persistent barriers that threaten sustainability at scale. The addition of more screening components was viewed as a means to improve usability, similarly, the incorporation of vernacular language options was highlighted by both end-users and stakeholders as a means of improving inclusivity particularly among CHWs with varying levels of digital literacy and educational backgrounds.

The absence of interoperability between CommCare and the DHIS further fragments data flows, increasing manual reporting burdens and compromising data use for planning. When CHWs cannot submit data in real-time or are forced to maintain parallel paper records, the NDHS’s vision of timely, integrated data for decision-making is compromised.^2,23,24 These barriers represent a fundamental gap in the underlying digital infrastructure required for a reliable national health information system. Therefore, end-user satisfaction alone should not be misinterpreted as a sign of system readiness; instead, it underscores an urgent need to invest in foundational technological pillars such as ICT infrastructure, interoperability frameworks and zero-rated digital health applications that will allow for real-time, in-depth reporting and remote monitoring of CHW activities.

Owing to the high levels of user satisfaction and the intention to continue using the platform, the CommCare application is currently being scaled up to all other health districts in the province where this study took place. This reinforces its potential as a national strategic tool to enhance workforce digital literacy and strengthen decision support within the WBPHCOT programme.

Organisational barriers to digital transformation

The barriers identified by stakeholders extend beyond technology to core organisational and environmental challenges that hinder digital health integration. The lack of timely technical support stemming from an under-resourced district ICT team points to a critical capacity-building gap. The initial model, which relied on external mHealth coordinators, was effective but unsustainable, exposing systemic vulnerability when donor support ended in 2022. This finding resonates with studies in Peru and North West province (South Africa), which stress the necessity of embedding technical support within the government structure.^26,27 For the NDHS to succeed, it must be accompanied by a parallel strategy to develop a permanent, skilled digital health workforce within the public sector, including dedicated helpdesk staff and mHealth coordinator roles funded by the state.

Furthermore, challenges in collaboration between WBPHCOTs and facilities, coupled with widespread device losses, highlight asset management gaps.

The successful mitigation of these issues through the orientation of facility managers and the enforcement of a formal asset management policy offers a powerful lesson. It demonstrates that overcoming barriers requires clear accountability frameworks and proactive stakeholder management. Engaging stakeholders such as facility managers and community leaders is imperative for building a resilient system. Active involvement of community leaders and facility managers not only improved access to training venues and local buy-in but also strengthened trust in CHWs, mirroring evidence from Senegal and other contexts.^28,29 These non-technical factors are, therefore, foundational to the scale-up and sustainability of any digital health intervention and must be formally integrated into NDHS implementation plans.

Best practices for sustainable digital health implementation and scale-up

The best practices and lessons learnt from stakeholders provide a blueprint for advancing the NDHS in a deliberate and sustainable manner. The district-supported cascade training model is a prime example of effective capacity building that can be replicated nationally. It efficiently scaled digital literacy across a large, diverse workforce and empowered OTLs as local champions, creating a multiplier effect essential for large-scale rollouts.

Beyond training, stakeholders consistently called for the ‘internalisation’ of digital health programmes. This encompasses two strategic shifts essential for the NDHS. First, it means building internal technical capacity to end reliance on external partners. Second, it necessitates investing in local IT infrastructure, such as local server hosting, to ensure data security, interoperability, and long-term cost control. The current lack of direct interoperability between CommCare and DHIS is a major barrier to a seamless health information system; solving this requires internal investment. Similar findings from the implementation of a primary health care mobile app in Ethiopia showed that implementation at scale requires sustained technical support and strengthening of the broader health system.²³

Finally, the recommendation to treat digital health implementation as a programme in itself is perhaps the most critical. It argues for replacing ad-hoc, project-based thinking with a strategic, long-term, and well-resourced approach that functions as an integrated system. This means ring-fencing funding, establishing dedicated government technical teams, and phasing rollouts to allow for proper embedding. The importance of deliberate planning, government ownership and integration within the existing health system is further demonstrated by the successful national scale-up of MomConnect in South Africa.¹⁰ This shift in mindset from implementing a tool to building a system is a crucial next step for translating the NDHS from a strategic framework into operational reality.

Limitations

Three main limitations should be considered when interpreting the results of this study. First, the 39% survey response rate may underrepresent end-users with lower digital literacy, potentially inflating satisfaction scores. The online questionnaire was administered in English, which may additionally have limited participation among users with varying language and literacy competencies, highlighting the importance of multilingual functionality and inclusivity in future digital health interventions. However, the large sample size (n = 464) and thematic concordance with stakeholder interviews support the validity of the findings.

Second, the retrospective design is susceptible to recall bias. This was mitigated by triangulating anonymous survey responses with key informant accounts to cross-verify events.

Finally, the generalisability of findings from one metropolitan district may be limited by contextual variations in governance and resource allocation. Nonetheless, the core implementation challenges and facilitators identified offer transferable insights for similar public health settings in South Africa.

Recommendations

Advancing the NDHS will depend on concurrent and sustained investment in three areas: (1) institutional investment in ICT infrastructure; (2) strategic capacity building to create a permanent, internal digital health workforce; and (3) proactive governance and stakeholder engagement to foster accountability and ownership at all levels. By focusing on these pillars, South Africa can harness the clear potential of digital tools like CommCare to build a learning health system capable of supporting universal health coverage.

Conclusion

The CommCare implementation experience in Ekurhuleni illustrates that the successful scale-up of digital health in South Africa is multi-faceted. Strong end-user adoption provides a solid foundation, but is threatened by systemic barriers in infrastructure, technical capacity, and governance. The path forward, as charted by the stakeholders in this study, requires a deliberate transition from pilot-dependent projects to a strategically managed district-level ecosystem.

To fortify the implementation of digital health interventions, key actions are proposed. These include enhancing the application’s technological utility by simplifying its user interface and incorporating local languages. Concurrently, organisational capacity must be strengthened by upskilling district-based ICT technicians in advanced software management and conducting thorough readiness assessments across all districts prior to national scale-up. These steps are critical for building a sustainable, equitable, and data-driven digital health ecosystem.

Acknowledgements

I would like to express my deepest gratitude to my dedicated supervisors for their invaluable guidance.

My sincere thanks go to the WBPHCOT Sub-district Coordinators, Petunia, Thembi, and Thembisile, for their significant contributions and assistance.

Abbreviations