Effectiveness of Leadership, Management, and Governance Competencies for Performance Improvements at Primary Health Care Entities in Ethiopia: A Before and After Study

Leadership, management, and governance (LMG) interventions play a significant role in improving management systems, enhancing work climate, and creating responsive health systems. The Ethiopian Federal Ministry of Health with the support of the USAID Transform: Primary Health Care Activity has been cascading basic LMG training and interventions to primary health care staff. The results of this research will help policymakers, program managers, and implementers to make informed decisions in the area of performance improvement. The study used an interventional, a before and after non-experimental survey design to assess the effectiveness of LMG trainings and interventions at primary health care entities. It was conducted from August 28, 2017, to September 30, 2018, in Amhara, Oromia, Tigray, and Southern Nations, Nationalities, and Peoples’ regions. Data collection took place through interviewer and self-administered questionnaires across 136 health facilities, in which there were 293 health workers who attended the basic LMG trainings and interventions. In addition, training records were reviewed to capture the changes in knowledge and skills through formal course tests before and after attending the basic LMG training and interventions to ascertain baseline and end-line scores on performance improvement project achievements. Furthermore, 333 non-trained health workers were given the opportunity to rate their organization’s leadership, management and goverHow to cite this paper: Argaw, M. D., Desta, B. F., Muktar, S. A., Tewfik, N., Tefera, B. B., Abera, W. S., Bele, T. A., Buseir, S. S., Rogers, D., & Eifler, K. (2021). Effectiveness of Leadership, Management, and Governance Competencies for Performance Improvements at Primary Health Care Entities in Ethiopia: A Before and After Study. Journal of Human Resource and Sustainability Studies, 9, 250-275. https://doi.org/10.4236/jhrss.2021.92016 Received: March 3, 2021 Accepted: May 4, 2021 Published: May 7, 2021 Copyright © 2021 by author(s) and Scientific Research Publishing Inc. This work is licensed under the Creative Commons Attribution International License (CC BY 4.0). http://creativecommons.org/licenses/by/4.0/ Open Access


Background
According to the World Health Organization (WHO, 2007), a health system is defined as "all organizations, people, and actions whose primary intent is to promote, restore, or maintain health". The WHO health system framework identifies six essential building blocks, or key functions: service delivery; health workforce; information; medical products, vaccines, and technologies; financing; and leadership and governance. To strengthen a health system, in addition to managing the interactions among the six building blocks, each building block needs to be strengthened (WHO, 2007). However, a lack of leadership, governance, and managerial capacity at all levels of health systems was identified as a major constraint to the scale up proven and affordable services in low and middle-income countries (WHO, 2005;Mutale et al., 2017).
Ethiopia's Federal Ministry of Health (FMOH, 2015) defines four pillars of excellence in health, namely: service delivery; quality improvement and assurance; leadership and governance; and, health system capacity to achieve its vision of "healthy, productive and prosperous Ethiopians". Leadership, management, and governance (LMG) interventions thus play a significant role in improving management systems, enhancing the work climate, and creating respon- Associates Inc., Malaria Consortium, EnCompass, and Ethiopian Midwives Association). During its first year, the project provided technical support in 300 woredas (districts) and has since expanded to 360 woredas in four agrarian regions in Ethiopia, (Amhara, Oromia, Southern Nation Nationalities, and Peoples' [SNNP], and Tigray). The project aims to contribute to the accomplishments of the Government of Ethiopia's Health Sector Transformation Plan and to prevent maternal and child deaths (USAID Transform: Primary Health Care, 2017a).

Statement of the Problems
Ethiopia has adopted and implemented several LMG trainings to strengthen its health system and build services that are accessible to more people by developing inspired leaders, sound management systems, and transparent practices within and among individuals, networks, organizations, and governmental bodies.
Some of these training modalities are: leadership for strategic information, a year-long training for policymakers and strategic decision-makers; leadership, management, and governance (Rolle et al., 2011), a set of three different training packages for policymakers and district, health center and hospital managers, implemented through four workshops completed over 6 -9 months; the primary health care unit management development program (FMOH, 2017), consisting of four workshops over 6 -12 months per district; and transformational leadership program (Global Health Leadership Institute, 2016), facilitated by the International Institute of Primary Health Care in Ethiopia for primary health care managers and program directors over the course of five days (International Institute of Primary Health Care in Ethiopia, 2017).
The findings reported in this study are a part of a comprehensive evaluation conducted to determine the effectiveness and efficiency of LMG trainings. The need to redesign training content and approach will be considered to publish in peer-reviewed journals subsequently. The aim of this study was to determine the effectiveness of leadership, management and governance competencies on performance improvement at primary health care entities using the modified Kirkpatrick & Kirkpatrick (2006) conceptual map of training evaluation (Armstrong, 2009) through implementation over the course of a one-year period.

Objective of the Study
The general objective of this study was to evaluate the effectiveness and efficiency of approaches of the basic LMG trainings for health managers in USAID Transform: Primary Health Care project-targeted woredas in the four agrarian The specific objectives of this study were to: • Investigate the effectiveness of the basic LMG trainings implemented in primary health care facilities in Ethiopia.
• Compare the outputs of the two types of basic LMG training modalities (block and segmented) implemented in primary health care facilities.

Hypothesis
The following two hypotheses were constructed based on the study objectives: HO1: There is no statistically significant difference between the mean knowledge score before and after basic LMG training.
HO2: There is no statistically significant difference between the block and segmented basic LMG training approaches.

Study Design
This study employed a prospective, an interventional a before and after survey design (Creswell & Plano Clark, 2011;Bowling, 2014;Bullock, 2017) and was conducted between August 28, 2017, and September 30, 2018, in Amhara, Oromia, SNNP, and Tigray regions of Ethiopia. Kirkpatrick & Kirkpatrick's (2006) model is well known for the analysis and evaluation of training and educational programs. In addition, it considers the different styles of training, both formal and informal, and is useful to determine aptitudes based on its four levels of criteria (Kurt, 2016). Therefore-for this study-a modified Kirkpatrick & Kirkpatrick's (2006) model of training evaluation (Alvarez et al., 2004;Armstrong, 2009) was used as a guiding framework for the data collection, analysis, and interpretation ( Figure 1). Figure 1. Conceptual map of LMG training evaluation (Armstrong, 2009;Kurt, 2016).

M. D. Argaw et al. Journal of Human Resource and Sustainability Studies
A conceptual map was adopted to evaluate leadership, management and governance trainings and interventions in Ethiopia. The framework depicts the relations between self-efficacy, cognitive learnings and training performances with individual and organizational factors in contributing for better health outcomes.

Study Setting
USAID Transform: Primary Health Care Activity is being implemented in four agrarian regions of Ethiopia, (Amhara, Oromia, SNNP, and Tigray) (Figure 2).
According to the health and health-related indicators of Ethiopia 2016, the 77 million inhabitants of these regions represent 85.9% of the national population (FMOH, 2016 Care, 2017b). One hundred and thirty-six health facility teams were recruited and enrolled in this study. The teams were exposed to an "integrated leading and managing for result" model. Each trainee was given the opportunity to self-assess their LMG knowledge and skills. An equal number of non-trained staff had opportunity to rate their organization's leadership, management and governance culture before and after the interventions. Map of Ethiopia depicting the USAID Transform: Primary Health Care Activity supported districts. The green colored districts are high performers, yellow colored districts are medium performers and red colored districts are low performers in terms of maternal and child health indicators.

Adoption of Leadership, Management and Governance Training
and Interventions Based on the lessons from national experiences, the USAID Transform: Primary Health Care Activity developed a blocked and segmented training package with subsequent six to nine-month leadership projects to reach staff at primary health care facilities. The project developed and implemented LMG trainings and interventions using nationally endorsed course materials that allowed trainees to earn 78 continuing education units upon completion (FMOH, 2017). The five-course modules include: 1) introduction to health systems; 2) introduction to leadership, management and governance; 3) improving performance through enhanced leadership, management, and governance; 4) resource management; and, 5) health service delivery management. Each module consists of units, sessions, and sub-sessions designed to achieve desired competencies. These practices are: four leading practices, (scanning; focusing; aligning and mobilizing; inspiring); four managing practices, (planning; organizing; implementing; monitoring and evaluating); and, five governing practices, (cultivating accountability; engaging stakeholders; setting shared goals; stewarding resources; continuous governance enhancement).

Basic LMG Trainings and Coaching Interventions
USAID: Transform: Primary Health Care Activity facilitated the two training modality option blocks i.e. those administered all at once, and segmented blocks i.e. those that were broken into a certain number of days over a period of time, for each of the five-course modules. From June 8 th to September 30 th , 2017, ten-block and fourteen segmented basic LMG trainings were facilitated and baseline measurements were taken. The LMG trainings were conducted in 24 districts with a total of 555 trainees. The details of participants in each category and region are presented in Table 1. Based on the national course syllabus (S1 File 1), trainees are informed about the priority of health area, and concept of leadership, management and governance for result are covered. During the classroom trainings, the LMG core teams developed projects using a tool named the Challenge Model (MSH, 2005).
This tool helped the core LMG teams face their organizational challenges through developing performance improvement projects by reviewing their organizational mission and strategic priorities; develop a shared vision for the future that contributes to the accomplishment of the organizational mission or strategic priorities; define one measurable result that can be achieved in a short amount of time; assess the current situation to take an accurate baseline measurement; identify the obstacles and root causes using a fishbone analysis, the why technique or workflow analysis which helps the core LMG teams reach the stated de-

Target Population
The study targeted two groups of the population, namely: LMG trained health workers, and non-trained health workers employed in the four intervention tar- LMG trained and non-trained health workers with perceived changes in management systems, work climate, and responsiveness of the health system to new challenges were measured before and after the intervention.

Eligibility Criteria
All study participants work in USAID Transform: PHC Activity's LMG intervention targeted primary health care entities. All basic LMG trainees and non-trained staff in the public health sector who volunteered to participate and those that had completed the pre-post training interviews were targeted. Health workers in facilities not targeted for the LMG intervention, health workers who did not volunteer to participate and health workers who had not completed the pre-post interviews were excluded from the study.

Sampling Size and Sampling
All twenty-four LMG training and interventions targeted districts were purposively selected and two different sample sizes were calculated.

Sample Size Determination for LMG Trainees
The first sample size (n) for number of LMG trainees was calculated using Slovin's formula (Tejda & Punzalan, 2012), Since, the population and their distribution is too large to sample: where: n = designates sample size for the study; N = designates total number of employees; e = designates maximum variability or margin of standard error at 95% confidence interval; and 1 = designates the probability of the event occurring.
Therefore: the estimated sample size for the trained group 233; and considering 25% for non-response, refusal, and dropout, it makes the sample size-293 (Pagano & Gauvereau, 2000). The sample size was allocated based on the number of trainees in each intervention district and a simple random sampling technique was used to enroll participants.

Sample Size Determination for Non-LMG Trainees
The second sample size (ni) for non-trained staff was calculated using a single population proportion formula (Daniel, 2009).
Where, ni is sample size, p is the proportion of LMG non-trained staff who have the desired knowledge on thirteen practices, and d is the margin of error.
In addition, the following assumption was used: since p is 0.50% (p = 0.50, q = 0.50), allowing 5% for expected margin of error (d) with 95% confidence level (Z α/2 = 1.96), the required sample size ni is 384. The sample size was distributed based on population proportion to the size of LMG trainees and a simple random sampling technique was used to enroll non-trained staff in targeted health facilities.

Data Collection
The data were collected from basic LMG trainees and non-trained staff working in intervention primary health care entities using interviewer-administered and self-administered structured questionnaires, respectively. The questionnaires were prepared after a thorough review of relevant literature and the national leadership, management, and governance in-service training materials for hospitals and health center managers (Wiseman, 2011;FMOH, 2017;Mutale et al., 2017). The questionnaires have two major parts. The first part is dedicated for LMG knowledge and skill survey tools. The second part is dedicated to capture the health systems strengthening survey tools. The details of each tools are presented below.
Sixteen data collectors, each with clinical, health management, social science, or public health training were recruited. Data collectors and supervisors were trained on ethical principles, data collection tools, and interviewing techniques.
Before the actual data collection started, all tools were piloted in USAID Transform: Primary Health Care Activity-supported woredas and were amended accordingly. The questionnaires for basic LMG trainees had four sections: 1) instruction and consent to participate; 2) sociodemographic characteristics of participants; and, 3) organizational culture rating tool; 4) document review using data abstraction for scores of each trainee captured during classroom before and after course tests, and performance improvement project measurements at baseline and end-line.

Health Systems Strengthening Survey
Eighteen questions with a 10-point Likert scale (Likert, 1932) were used to measure responses to statements presented on: strengthened management systems (9 items); enhanced work climate (5 items); and responsiveness or capacity of the health system to overcome new challenges (4 items). The sample questionnaire for each category is presented in Box 1. To measure the changes in LMG practices of trainees, a tool was used to assess their perceived practices before and after the basic LMG training was offered to them. The respondents were LMG M. D. Argaw et al.
Sample tool Category

WC1
In this office, employees understand the organizational structure and reporting lines of their unit/department and how their job functions relate to overall departmental objectives and goals.
Strengthened management systems.

WC10
My contributions at work are acknowledged and appreciated. Enhanced work climate.

WC13
In this office, supervisors delegate challenging assignments to assistants, which helps them to develop their skills and expertise.
Responsiveness or capacity of health system to overcome new challenges.
trainees and non-trained health workers operating within the intervention area's primary health care entities. These were adapted from the United Kingdom's clinical leadership competency framework as cited in Mutale et al. (2017). The questionnaire was developed in English and translated into local languages (i.e., Amharic, Afan Oromo, and Tigrigna), then translated back into English.

Validation of organizational LMG culture measurements
The questionnaires for non-trained staff had three sections: 1) instruction and consent to participate; 2) sociodemographic characteristics of participants; and, 3) organizational culture rating tool. The results of these data were used by researchers to validate the knowledge and skills of LMG trainees.

Data Analysis
The data were analyzed using SPSS IBM version 20 (SPSS Inc., 2011). The tools' internal reliability was assessed using Cronbach's alpha values (Gershon et al., 2004). According to Bland and Altman (1997), if the Cronbach's alpha value score is more than 0.7, the scale can be considered reliable. A statistical test using multi-collinearity analysis through determining the variance inflation factor (VIF) was run to check the tools' divergent validity (Bowling, 2014). According to Menard (1995), if the VIF reported value exceeds 10, it implies the associated regression coefficients are poorly estimated because of multi-collinearity. In this pilot study, the collinearity test VIF results were:

Ethical Considerations
Ethical clearance was obtained from four regional institutional review boards.  Table 2 presents the statistical summary of the socio-demographic data collected from trainees. One-hundred and three trainees, (35.2%) were recruited from the SNNP region; 62.5% of study participants, (183) are between 26 and 35 years of age, and the mean age with standard deviation (SD) was 29.9 ± 6.9 years. The majority, (222/75.8%) of the study participants were male. Seventy, (58.0%) were nurses who had worked in the profession for an average of 7.6 years. Oromia, 37 (27.4%) from Amhara, and 43 (31.9%) from SNNP regional states were included in the study. One-hundred-and eleven (82.2%) were health centers, 12 (8.9%) were primary hospitals, 11 were district health offices and 2 were zonal health departments.

Knowledge Change before and after LMG Training
The LMG trainees' mean pretest knowledge score with SD was 44.97% ± 15.50%.

Performance Change at Baseline and End-Line Survey
Of the developed 136 performance improvement projects, 53, (38.9%) were on delivery services. Twenty-five, (18.5%) focused on family planning, and 25, (18.5%) on health system strengthening ( Figure 3). All 136 core LMG teams were assessed; the mean performance improvement project baseline score with    LMG packages had a higher score, with a statistically significant difference at Chi-square = 8.32, p = 0.01. Certification showed that more block LMG trainees completed the course than segmented trainees, with a statistically significant difference at Chi-square = 7.16, p = 0.02.   Table 4. Work climate, management system strength, and ability to respond to new challenges before and after assessment of LMG trained and non-trained staff, September 2018.

Health System Improvement
Serial no. Characteristics Trainees n = 293 before mean ± SD Trainees n = 293 after mean ± SD Staff n = 333 before mean ± SD Staff n = 333 after mean ± SD WC1 In this office, employees understand the organizational structure and reporting lines of their unit/department, and how their job functions relate to overall departmental objectives and goals.

WC2
For most meetings in this office, agendas are circulated to all before the meeting. 7.00 ± SD 2.64 7.63 ± SD 2.52 5.47 ± SD 2.99 7.22 ± SD 6.07

WC3
For most meetings in this office, minutes are circulated to all soon after the meeting, indicating follow-up items.

WC4
The leadership here keeps staff well informed about what is going on within the organization. 5.90 ± SD 2.30 8.08 ± SD 1.99 6.00 ± SD 2.58 6.89 ± SD 2.75

WC5
In this office, cooperation and teamwork between staff in different units is encouraged.

WC6
In this office, we are encouraged to use data to guide decision-making, priority-setting, and planning.

Validation of Leading, Managing and Governing Organizational Culture
Three hundred and thirty-three non-trained staff were asked to measure their work climate, management systems, and health system responsiveness to new challenges before and after LMG trainings and interventions in their respective facility. The mean before and after training scores on strengthening management systems with SD were 6.34 ± 1.87 and 7 p < 0.001. The overall before and after training health system improvement scores were 6.24 ± 1.76 and 7.51 ± 1.74, respectively. The score showed a statistically significant improvement with t = −17.09, df = 332, p < 0.001 (Table 4).

Discussion
This a before and after study was conducted to evaluate USAID Transform: Primary Health Care Activity-supported basic leadership, management, and governance training and interventions which were cascaded to primary health care workers and managers in the four regional states of Ethiopia. The LMG training included didactic, theoretical-based classroom training implemented either as a block or sequential training combined with subsequent implementation of performance improvement projects for six to nine months at the trainees' place of employment (FMOH, 2017 (2015) who reported that after leadership development interventions, the on-job performance of participants and their supervisors were improved, and levels of self-efficacy significantly increased over time.
The evaluation indicated that trainees contributed to the improvement of primary health care service coverage over the 12 months of LMG interventions.
This result was also attributed to the implementation of multi-faceted LMG interventions, which ranged from classroom trainings to leadership-project development and implementation. The performance improvements observed include improved management systems and work climate in the study areas. The findings also concur with La Rue et al. (2012), who assessed a leadership development program in Kenya by comparing baseline, end-line, and post-intervention data as well as with Rowe et al.'s (2018) systematic review, which reported professional health care provider practices improved using multiple strategies rather than by a single strategy. Similarly, Arinez et al. (2002) confer that good leadership and management practices improve the work climate, staff satisfaction, motivation, and performances.

Limitations of the Study
The major limitation of this study is related to the employed before and after non-experimental study design. Unlike a randomized study design, it is difficult to conclude the causal association between the LMG trainings and interventions with the resulting performance improvements. Despite this limitation, the investigators tried to ensure the relationship through collecting data from a longitudinal follow-up report, onsite coaching and developed agreed action plans. The study only targeted LMG training intervention sites, hence, before generalizing, the results should be interpreted based on the context. In addition, there were a significant number of non-responses among non-trained targeted staff.

Summary
Leadership, management, and governance trainings and interventions evaluation in different part of the world revealed its positive impact on performance and health care outcome improvement in a given health system. In this before and after LMG training evaluation, the knowledge and skills of trainees were significantly improved by 36.0%. In addition, as a result of the leadership projects, each primary health care entity improved health service coverage by an average of 16.5%. Among the LMG trainees, 70.0% (205/293) were successfully complete the national training requirements for certifications. While comparing the results, a higher and statistically significant scores were documented for blocked than segmented LMG training approaches. Furthermore, the evaluation showed M. D. Argaw et al. Journal of Human Resource and Sustainability Studies a significant improvement in strengthening management system (P < 0.001, enhancing work climate (P < 0.001) and responsiveness of the health system for new challenges (P < 0.001). Therefore, the LMG training and intervention supported by the USAID Transform: Primary health Care Activity was found to be effective.

Conclusion and Recommendations
The performance of the primary health care entities was significantly improved as a result of enhanced knowledge and skills through LMG trainings and interventions. Therefore, the basic LMG training and interventions were effective in enhancing the knowledge and skill of health workers. It is recommended that providing LMG trainings for more health workers and managers working at primary health care entities will accelerate the implementation of prioritized health sector interventions and is helpful in achieving the Sustainable Development Goals as a global target. An evaluation of the efficiency of the basic LMG training package is also recommended.

Funding
This study was fully funded by USAID under cooperative agreement AID-663- A-17-00002 as part of the USAID Transform: Primary Health Care Activity in Ethiopia. The funding body had no role in the design, data collection, analysis, interpretation and writing stages of the study undertaking.