Introduction
Globally, it is estimated that 22% of maternal and 23% of neonatal deaths could be avoided by increasing midwife interventions such as skills training.1 In South Africa (SA), training healthcare workers (HCWs) to manage obstetric emergencies has resulted in a 29.3% reduction in maternal mortality.2 The decline in maternal and neonatal mortality observed between 2000 and 2020 was negatively impacted by COVID-19 in SA and globally.3,4 It is unlikely that the Sustainable Development Goal of 70 maternal deaths per 100 000 live births by 2030 will be achieved5; in SA it was 127 per 100 000 in 2020.3 These maternal and neonatal mortality rates elevate the importance of quality improvement (QI) interventions for maternal and neonatal healthcare (MNHC).
The QI intervention of interest in this study is the Breakthrough Series (BTS) Model for Improvement with its associated Plan-Do-Study-Act (PDSA) cycle.6 The BTS, initially developed for QI in the manufacturing industry,7 has become well-established in healthcare since the 1980s.7 In SA, it is only since the 2000s that QI interventions have been used, mainly to improve TB and HIV services.8 The underlying principles of QI models when applied to healthcare are: (1) HCWs establish themselves as a QI team; (2) the team develops and iteratively assesses small-scale reconfigurations of service delivery processes to solve problems (‘change ideas’); and (3) these change ideas are implemented largely without additional resources and within the HCWs’ power to change.9
QI programmes have had mixed effectiveness in SA; while the PDSA cycle significantly improved HIV viral load monitoring in women, it had no effect on HIV re-testing during pregnancy.10 This aligns with conclusions across three effectiveness reviews that QI outcomes are varied and often modest.11–13 Five factors explain the mixed outcomes. First, the competence of the QI advisor is important. The steps of QI models are complicated and the advisor is key in helping teams to navigate the methodological complexities.14 Advisors need accredited training to technically support QI teams.15 Second, personality attributes of QI team leaders: in addition to general leadership skills16 they must have an aptitude to collect and analyse the change idea assessment data.17 They must create a learning climate and psychological safety for members to own their mistakes.18 Third is team composition and member attributes. The team must include influential members to motivate other HCWs,19 and members must have the leaders’ ‘data aptitude’.20 Fourth is the need for facility management support regarding time and resources for QI activities.21 Finally, is the fidelity with which the QI model is implemented. In two reviews22,23 it was respectively reported that only 20% (of 73 studies) and 4% (of 72 studies) kept to the prescribed PDSA cycle. This led these review authors to conclude that low fidelity negatively impacts the model’s effectiveness.
The South African National Department of Health (NDoH) developed and led the implementation of Mphatlalatsane, a QI programme to strengthen MNHC services in the country to improve patient health outcomes and experience of care. The programme ran from 2018 to 2022 with the objectives to:
-
reduce institutional maternal, neonatal, and still birth mortality rates;
-
improve access to contraception and safe termination of pregnancy; and
-
ensure better experience of patient care.
This paper presents an evaluation of how the programme was implemented in the participating facilities and the factors that shaped the performance of the QI teams before, during, and after the COVID-19 pandemic. The study unpacks the interplay between the advisors, leaders, members, and facility that created a microsystem that explained performance variations. The term ‘microsystem’ is used because this study suggests that the four elements are interlinked and shape each other in such a way that the ‘whole becomes more than the sum of the elements’. Viewed through a learning health systems lens, this paper offers insights into how QI teams combined internal service experience and routine data with external QI methods and advisor support in a learning process, and how that learning was translated into their daily MNHC practice. This lens is consistent with the BTS Model for Improvement, where teams ask what they are trying to accomplish, how they will know whether a change is an improvement, and what changes they can make, before using PDSA cycles to test and adapt change ideas in context.6 The paper uses the concept of a microsystem to examine how HCWs harvested local knowledge and embedded effective change ideas into routine care.
Method
Study design
Mphatlalatsane was integrated into existing provincial, district, and facility MNHC structures. This paper presents the findings from a longitudinal, qualitative evaluation of Mphatlalatsane using an implementation research approach. A qualitative methodology was applied to get an in-depth understanding of the implementation processes and factors that shaped team performance.24 In four previous papers, the authors reported in detail on how the advisor, leader and member attributes, as well as team composition and management support respectively, shaped performance.25–28 This paper investigates how these factors combined to impact team performance in implementing the QI methodology.
Ethical considerations
This study was approved by the Ethics Committees of the University of the Western Cape (BM19/10/16) and South African Medical Research Council (EC019-11/2019). The research committees of the respective provincial health departments approved the study. During the recruitment visit, each participant received an information document that detailed the study aims, what participation would entail, and measures to ensure participant wellbeing. A signed consent letter was collected from each participant before the first interview was conducted with them.
Setting
Mphatlalatsane (meaning the ‘bright morning star’ in Setswana) was a multi-partnered, five-year (2018–2022) programme, implemented in 21 purposively selected facilities across four health districts in the Eastern Cape, Limpopo, and Mpumalanga (Map 1). The facilities were selected based on average or below-average performance on an NDoH maternal and perinatal performance matrix, and represented urban, semi-urban, and rural settings. The seven facilities in each province (two clinics, two community healthcare centres (CHCs), two district hospitals, and one tertiary hospital), reflect the patient referral pathways in the districts.29
Health care facilities
For this study, 15 purposively sampled facilities were chosen for evaluation. Facilities were selected to represent rural and urban settings, all four facility types, and high- and low-scoring facilities based on a QI readiness assessment matrix which the Institute for Healthcare Improvement (IHI) and NDoH had conducted. The matrix assessed how prepared the facilities were for the QI project implementation and was administered a few months before QI training commenced. One facility declined participation in the study because they felt that COVID-19 was too disruptive to their QI activities for meaningful participation. Fifteen teams from the 14 consenting facilities participated as one facility had two teams. At the time of the NHI / NDoH assessments, one provincial department had not yet consented to being part of Mphatlalatsane. Consequently, 11 evaluation facilities from the two consenting provinces were studied and a further four were included from the province that joined late. The study timeline is presented in Figure 1.
QI model
The IHI served as the QI training partner for the first nine months of implementation. Each facility nominated between three to six MNH staff who attended IHI’s BTS/PDSA induction training. Upon returning to the facilities, the trainees established their QI team, comprising between four and 12 HCWs working in existing facility service teams. The team members maintained their existing facility duties and did not receive remuneration for participating in the QI team. The facility management selected the team leader from the trainees. Only District 1 teams received two further scheduled trainings before the COVID-19 pandemic lockdown. Districts 3 and 4 joined the programme late (February 2020), given a delay in consent from their provincial department to be part of Mphatlalatsane, so these districts only experienced three weeks of implementation before lockdown began (Figure 2).
The change idea detailed in Box 1 and Figure 3 is typical of the small-scale, low-cost interventions that QI teams developed to improve the quality of their services, and was implemented in a clinic. Appendix 1.a provides the details of one successful change idea from, respectively, a regional hospital, district hospital, and CHC.
The Clinton Health Access Initiative (CHAI), as the Mphatlalatsane implementation secretariat partner responsible for day-to-day activities, appointed and managed the advisors. Each advisor supported the seven teams per province. Due to advisor resignation, District 1 was without an advisor for one month (August 2021). The District 2 advisor resigned in August 2022 so their teams were supported by the District 1 advisor for the last five months of implementation. Districts 3 and 4 were without advisor support for four months (April-July 2021). Between March to July 2020, Mphatlalatsane implementation was suspended because of the COVID pandemic lockdown.
Data collection
Data were collected over 34 months (February 2020 to November 2022) from five sources exploring different foci within each (Table 1).
QI leaders and members (Table 2) were interviewed at their respective facilities at three time points: May 2021, September 2021, and September 2022. Leaders shared their documentation during the interviews. Most of the advisor debriefings were joint interviews with all of them present, but some were individually conducted. Debriefings were conducted and recorded on Microsoft Teams software. The evaluation team was invited to attend the Mphatlalatsane management meetings, and observations and reflections of the meetings were recorded in a fieldwork journal. All the interviews were conducted in English.
Data analysis
All data were thematically analysed by three people.30 The first few debriefings and interviews were professionally transcribed, and the lead author developed initial codes. These were then presented to the other two researchers who checked and amended the code list. Regular analysis meetings ensured that emerging codes from the remainder of the interviews were added. After the coding was completed, codes were grouped into sub-themes and themes. Accuracy checks of the transcripts were performed and coding used ATLAS.ti (version 8.1) software.
Assessing team performance
Teams were qualitatively assessed on their implementation of the QI methodology over the evaluation period. Based on fieldwork observations and preliminary data analysis of the advisor debriefings, leader and member interviews, and fieldwork journal, the analysis team defined a well performing team as one that had: (1) a functional leader; (2) revitalised their QI work by resuming the implementation of change ideas or developing new ones following reimplementation of the programme after COVID lockdown; (3) a leader who was a QI enthusiast and reported positively about the team’s performance; and (4) were assessed as ‘QI matured’, i.e., were able to function without advisor support. As it turned out, all the well performing teams met all the criteria towards the end of the programme implementation.
Examples of assessment evidence is provided in Appendix 1.b. The less well performing teams did not meet any two or more of the above criteria. This low threshold is evidence of how severe the COVID-19 disruptions were, in particular during the lockdown periods.
To the authors’ knowledge, Mphatlalatsane is the second MNHC QI programme reported in peer-reviewed literature in SA that was implemented outside the context of HIV/AIDS.
Findings and Discussion
Participant demographics
The study included 17 leaders (Table 3) across 14 teams, as there were three replacements. All leaders were senior maternity staff and female: one was a doctor, and the rest were midwives. In the hospitals and three CHCs, the leaders were operational managers (OPMs) in the maternity ward. In the remaining two CHCs and all three clinics the leaders were the OPMs. Of the 47 members (Table 3), 45 were female professional nurses, one a male professional nurse, and one a male data clerk. Four advisors were female and one was male and between 28 and 53 years old. Four of them had accredited training and between 17 months and 15 years’ advisor experience. The fifth advisor had no training and little experience.
Team performance assessment
Six of the 14 teams (Table 4) were well performing teams: four in District 1 and two in District 2. There were no well performing teams in Districts 3 and 4.
The teams’ performance reflected how well each facility microsystem converted QI knowledge into routine work, as a leader of a well performing team explained about institutionalised their change idea: “I’ll show you the protocols that were put in place since then [Mphatlalatsane]. So, we’re not recording or observing the arrival on what time anymore, we just have it as a routine and when a patient arrives, the first thing they do are the vitals”, (leader, well performing team).
In the well performing teams, the BTS/PDSA method became a practical way to learn from daily service problems. Information came from observed service gaps, routine records, run-chart data, programme documentation, and advisor feedback. Deliberation occurred when leaders, members, and advisors discussed why a gap persisted, which change ideas were feasible, and how the change could be fitted into existing workflows. Action occurred when teams tested small-scale changes, assessed whether they improved care, adapted them where needed and retained those that worked, and converted tacit institutional knowledge into tested changes in practice.
Mphatlalatsane confirms international evidence
This evaluation regarding implementation processes and factors that shape team performance in part, confirmed international QI literature (see Appendix 1.c). The importance of a technically proficient advisor15 was suggested by the fact that all teams in Districts 3 and 4 were less well performing and the advisor for those districts was not QI qualified. Across all districts, leaders of well performing teams displayed an aptitude and skill set capable of using the PDSA cycle successfully,17 and created a safe learning climate in their teams.18 These attributes were absent in leaders of less well performing teams. In the less well performing teams most members did not buy into the methodology. Well performing teams had a core group of approximately three HCWs who accepted the methodology and motivated colleagues to participate.19
With its specific steps and prescribed PDSA-cycle stationery, QI was initially thought to add to HCWs’ workload.6 It was therefore important that the PDSA process became embedded into everyday routine and promoted as a ‘back to the basics of care’ methodology.31 All teams adapted the QI methodology23 by replacing prescribed stationery with self-initiated stationery, and having longer assessment cycles than the prescribed rapid assessments. In contrast to the literature,22 these adaptations did not appear to be less effective in solving the targeted service delivery gaps: change ideas done with self-initiated stationery (the CHC change idea, Appendix 1.a), yielded as good results as those ones done with the prescribed stationery (the hospitals’ change ideas, Appendix 1.a). These adaptations reflect how HCWs navigated the intervention complexities.
The COVID-19 pandemic was reported as the main driver in teams’ adaptions of the methods which corroborates the reported literature on the emotional trauma32 and service disruptions the pandemic caused.33
The interplay between advisor, leader, members, and facility context
One of the key intentions of the evaluation was to understand why the six well performing teams returned to their QI work after the lockdown periods and fully embraced the methodology, while the other eight less well performing teams did not, despite all 14 teams reporting the same resilience in maintaining routine MNHC services during the pandemic. Also observed across all the previous analyses25–28 was a dynamic interaction between advisor, leader, members, and facility, that created a microsystem. Within this microsystem, the interaction between these elements shaped each other. Untangling these interactions provides insights explaining team performance variations regarding the adoption of the QI methodology.
The advisor is more than a technician
The importance of the advisor is under-reported in the literature as it mostly refers to them only as technicians.15 In Mphatlalatsane, advisors played a huge role in getting HCWs to participate in the programme, and required much patience with many of the less well performing teams, continuously encouraging them to continue with QI: “… [We did nothing following the QI training] until Advisor B came to support us, saying: ‘Hi people, you must start’”, (leader, less well performing team).
The advisors were the drivers for teams to revitalise their QI activities after the lockdown period. The importance of their interpersonal skills was exemplified during COVID-19, when they became counsellors as much as being technical advisors. Advisors had to find a balance between helping teams deal with COVID-19 and implementing their PDSA cycles: “So, the [QI support] call can go two ways. You can get them to be calm so that you review it [QI activities], or sometimes they will be on the call as a debriefing, you won’t necessarily get anything out of the QI work but what they’ll be telling you is sort of a debrief …” (advisor).
A proficient advisor and enthusiastic leader are not enough
All District 1 teams had the same competent advisor for the first 23 months of implementation, yet two of the six teams were assessed as less well performing. In the one less well performing team, Facility A, there was an enthusiastic QI leader but team members lacked QI aptitude. Upon the leader’s return from QI training, she described her team’s disconnect with the methodology as follows: “It’s like you are coming from another planet and then you are dropped into another planet …so it looks like everything that you have learnt, it’s like you exhaust them …”, (leader, less well performing team).
In the other less well performing District 1 team, Facility B, the members had a similar apathy towards QI, but the team also suffered leader instability. With the leader on sick leave for a long period, there was no one who sustained the PDSA implementation.
The prevailing team culture and leader stability are not enough
Though the importance of leader stability in relation to team performance is well reported in other contexts,34 no QI-specific literature was found on this. The observations from this study suggest that leadership stability plays a part in team performance, as most less well performing teams had leadership changes. Often there was no formal handover, and at times it was difficult to know who the leader was. This was contrasted with no leader changes in any of the well performing teams. Well performing teams also had a second-in-charge leader, and their importance is reflected here: “She [leader] must be in charge of the project. But she must not be alone. Like what I’m saying, tomorrow I’m off [leader], she’s [second in-charge leader] there . . . she must check what we have been doing,” (leader, well performing team).
However, an enthusiastic and stable leader assigned to a team with a prevailing negative work culture, i.e., members did not work well together in the existing service teams from which they were recruited, did not lead to a well performing team: “QI is a different way of looking at your problem … how to go about identifying and solving your problem … It made sense to me”, (leader, less well performing team).
Similarly, a stable but less enthusiastic leader did not contribute to QI team success: “We are clinicians, we’re not researchers. We don’t know how to do research [referring to QI methodology]”, (leader, less well performing team).
Whilst the literature reviewed yielded only a single reference19 on the impact of the prevailing culture on team performance, this study’s evaluation showed it to be important. In all six well performing teams, the members were recruited from existing service teams who had a positive prevailing culture of working together: “I see the level of commitment from the [maternity ward] managers. They are committed, because if there’s an adverse event, each and everybody is taking responsibility”, (member, well performing team).
In less well performing teams, members reported a negative prevailing culture, illustrated by one who reported that when asking for help from her existing service team, staff would decline, saying: “No, I’m not trained”, (member, less well performing team).
Reciprocal relationships
Within the said microsystem , team performance was a function of the reciprocal nature of interactions between the team members. In some instances, it was evident how this played out: the advisor established rapport with the leader, who in turn created a safe learning culture for members, and they reciprocated with appreciation on being consulted in developing change ideas: “So, solutions come from the members, and they say: ‘Okay, we can do this, we can do this.’ Then we choose a few that we see, have more ways [advantages] …”, (leader, well performing team).
Begin well stays well
It was evident that having less than a month of implementation before the COVID-19 lockdown (Districts 3 and 4 teams), versus the six months for the other two districts, impacted performance negatively. However, well performing teams remained well performing, while none of the less well performing teams transitioned to well performing. This demonstrates the importance of the initial implementation months to ensure good team performance over time. A strength of QI is that change ideas often result in drastic improvements,18 as was the case with some Mphatlalatsane change ideas; for example, improving antenatal TB screening rate from 53% to 100%. This suggests that starting the QI team with a change idea that will drastically improve service delivery may promote early buy-in.35
Strengths and limitations
The main strength is the longitudinal data collection that allowed establishing rapport with participants and observing trends over time.36 The rapport set a context where the participants not only shared positive experiences, but were also forthcoming about things that did not work well in their teams. Conducting the leader and member interviews at the facility deepened the researchers’ understanding of the many challenges they faced, e.g., how an over-crowded ward was more pressing than conducting a QI meeting. Using multiple data sources allowed data triangulation, for example a leader’s apathy towards QI recorded in an interview was often reflected in poor QI documentation.
This research is limited by the absence of quantifying the impact of team performance on maternal and neonatal mortality. Though the broader Mphatlalatsane evaluation showed QI did not reduce these mortalities at district level, measuring its impact at facility level would have balanced the qualitative performance assessment. There may have been member recruitment bias as it was left to leaders to recruit member participants, and leaders may have favoured members with positive views over those who held negative views.
Recommendations
The Mphatlalatsane initiative should be sustained and the lessons learnt utilised. The programme aligns with the NDoH’s envisaged ‘quality learning centres’.37 Though these centres will not exclusively focus on QI for MNHC services, the Mphatlalatsane well performing teams have much to share with colleagues embarking on QI. For example, how facility managers can integrate QI into routine care to address HCWs experiencing QI as an additional burden to their workload. This may include having team meetings as part of routine facility meetings, and using routine facility data to monitor change idea effectiveness rather than setting up new systems. The study’s teams were all set in resource-constrained facilities and are therefore able to share how, for example, they made-do with handwritten planning and reporting when there was no working printer.
Recommendations to set-up and manage a QI team to optimise team performance28 are depicted in Figure 4. Step 1 deals with identifying team leaders and appointing the advisor. Step 2 focuses on leader training, including which elements of the methodology can be adapted, and team member recruitment. Step 3 describes setting up the team and managing buy-in at the facility. Step 4 suggests starting implementation with an easy-to-implement change idea that is likely to see dramatic improvement in how care is delivered.
To replicate the success of the Mphatlalatsane well performing teams, the following three recommendations are essential. First, appoint advisors with sound technical and interpersonal skills, then tailor their support so that teams become independent of them (Step 1). Second, recruit HCWs with a QI aptitude as leaders and members (Steps 2 and 3). Third, introduce QI as a methodology that strengthens back-to-basics care to address concerns that QI will be an additional burden to staff (Step 3). As posited in this paper, future implementers should use QI to establish an improvement ethos amongst HCWs and not as a quick fix for selected service delivery gaps. Integrating QI into routine facility practices will be a first step towards a sustained improvement of healthcare services at facility level.
Future QI programme implementers should record successful change ideas in detail and institutionalise these ideas before external advisor support is withdrawn. Harvesting ideas requires more than recording that a target was reached. The ideas should be converted into institutional artefacts, including standard work routines, meeting agenda items, orientation material for new staff, peer-learning examples, and district-level repositories of tested change ideas. Provincial and district managers can strengthen this process by using well performing facilities as learning sites for other facilities with similar resource constraints. This would make the learning generated by QI programmes such as Mphatlalatsane less dependent on the original project team and more likely to survive staff rotation, leadership changes, and end-of-project funding. Additionally, local adaptations of QI methodologies should be retained as they may encourage sustained implementation because they are self-initiated and what HCWs can manage in their work contexts.
Qualitative evaluation should be supplemented with quantifiable measures of programme components. This will allow exploring correlations between team performance and configurations of variables, such as leaders’ skills, team size, and patient volume.
Conclusion
The HCWs in the well performing Mphatlalatsane teams showed the resilience to successfully implement and sustain QI, despite COVID-19 and intervention complexities. The findings of this paper confirm a set of established key factors that shape how teams adopt QI methods to improve MNHC services. These factors lie within the power of those who set up QI programmes, for example, appointing an advisor whose technical competencies are matched with sound interpersonal skills, and recruiting leaders and members with QI aptitude. However, nurturing a QI team to maturity and sustained use of the PDSA methodology, requires acknowledging that the advisor, leader, members, and facility context creates a microsystem that becomes more than the sum of its parts. QI success lies in being attuned to the interplay in this microsystem, and not just in the absence or presence of individual elements.
Acknowledgments
We thank the five QI advisors who generously shared their perceptions, experiences, and time, during the evaluation. We are grateful to the QI team leaders and members for allowing us to gain insight of what QI is about, and sharing their successes and challenges with us. We are indebted to the respective provincial and district management staff for supporting the evaluation and facilitating facility access. The NDoH was the custodian of the Mphatlalatsane initiative, and CHAI (South Africa) acted as the Mphatlalatsane implementation secretariat.