A Cross Sectional, Mixed-Method Cohort Study Evaluating the Implementation of Leading Better Value Care Initiatives ()
1. Introduction
Around the world, every healthcare system is struggling with rising costs and inequality (Porter & Robert, 2011). This is despite various initiatives designed to reduce errors, standardise practice guidelines and involve consumers in decision making (Porter & Robert, 2011; Porter et al., 2018). Healthcare is adapting to suit the changing needs and expectations of communities, patients and carers (Gentry & Badrinth, 2017).
Due to the complexity of health care delivery, many international health systems are struggling with unwarranted clinical variation, increases in service demand beyond population growth, an aging population, complexity of and chronic disease rising costs and uneven quality despite the hard work of well-intentioned, well-trained clinicians (Porter, 2011; Gentry & Badrinth, 2017). This highlights the need to shift the focus from volume and service outputs as emphasised by Activity Based Funding (ABF) to the patient outcomes achieved. The concept of Value-based Health Care (VBHC) has the ability to be a more effective approach for creating a sustainable health system than traditional approaches, while improving the effectiveness and appropriateness of care to patients (Gentry & Badrinth, 2017; Koff & Lyons, 2020). The VBHC approach has been applied in other health care systems overseas (Gentry & Badrinth, 2017; Koff & Lyons, 2020; Nilsson et al., 2017). NSW Health has focused on the transformation to VBHC in which every stakeholder in the health care system has a role to play. There is a dearth of literature to understand viewpoints of participants involved in implementation of these initiatives. Thus, it is important to explore stakeholder views to understand the barriers and challenges and to identify strategies to overcome these. In turn, this can facilitate improvement of implementation and acceptability of initiatives at each stage of the process (Koff & Lyons, 2020; Agency for Clinical Innovation, 2018a; Agency for Clinical Innovation, 2013).
In 2017, the NSW LBVC program is being implemented by the Local Health Districts (LHD) with support from state governed health agencies. Specific to the health domain for focus, the agencies play a pivotal role in developing evidence based models and providing support for development and implementation of initiatives in LHDs (Koff & Lyons, 2020; Agency for Clinical Innovation, 2018b; Agency for Clinical Innovation, 2013). In this context, health outcomes are defined as outcomes that matter to patients with the key goal of improving value for patients. By doing so, uniting the interests of all health care system stakeholders is vital (e.g. patients and their families, NSW residents, clinicians, LHDs, health service agencies/networks, support organisations and the Ministry of Health (MoH) (Koff & Lyons, 2020; Agency for Clinical Innovation, 2018a; Agency for Clinical Innovation, 2013; Agency for Clinical Innovation, 2020).
Acknowledging the challenges for the NSW Health System with its considerable size and to achieve scalability of initiatives to deliver impact, the NSW MoH identified eight ‘Tranche One’ clinical initiatives in 2017/18 for implementation as a state-wide priority (Koff & Lyons, 2020). The implementation of this first tranche of initiatives was supported by two state health agencies known as Agency for Clinical Innovation (ACI) and the Clinical Excellence Commission (CEC) (Koff & Lyons, 2020; Agency for Clinical Innovation, 2018b; Agency for Clinical Innovation, 2013; Agency for Clinical Innovation, 2020). These clinical initiatives included:
1) Osteoarthritis Chronic Care Program (OACCP)
2) Osteoporotic Refracture Prevention (ORP)
3) Diabetes High Risk Foot Services (HRFS)
4) Inpatient Management of Diabetes Mellitus (IMDM)
5) Management of Chronic Heart Failure (CHF)
6) Management of Chronic Obstructive Pulmonary Disease (COPD)
7) Renal Supportive Care (End Stage Kidney Disease) (RSC)
8) Adverse Events: Falls in Hospitals (Falls)
This study will examine the successes, challenges and barriers in the first 14 months of implementation of the LBVC initiatives at a local level. This will lead to better development of a localised implementation model.
2. Methods
This is a cross-sectional, mixed-method cohort study for the programmatic evaluation of LBVC initiatives implementation in a local LHD. Both qualitative and quantitative data were collected concurrently in 2018 (14 months post implementation of the LBVC) for the better understanding of the implementation. According to Creswell (2003) and Onwuegbuzie & Johnson (2006), stemming from a pragmatic theoretical perspective, the mixed methods approach taken in this study uses a basic concurrent style where both qualitative and quantitative data are collected around the same time point and analysed separately after which data is interpreted to inform findings (Creswell, 2003; Onwuegbuzie & Johnson, 2006).
A survey was developed by the study investigators using information from a literature review and also by adapting implementation questions from other studies. In order to gain a wider understanding of participant views on issues associated with implementing the LBVC program, a 10 point Likert scale was used, where “1” was considered as very unlikely or poor and “10” was considered as extremely likely or excellent. These data were analysed descriptively using Microsoft Excel and IBM Statistical Package for the Social Sciences (SPSS), Version 20.
Sample
The survey was conducted with the purposive sample of the stakeholders who participated in the development and implementation of the LBVC initiatives within South Western Sydney Local Health District (SWSLHD). These stakeholders include members of the Steering Committee and Working Groups, Executive Sponsors and Clinical Leads for each initiative and project team. Survey Monkey was utilized as the online platform for collection of information from the stakeholders.
Semi-structured interviews were used to explore participant views about the issues identified in the research aims. The semi-structure interview themes and questions explored the participants’ expectation and experience in implementing the initiatives. The participants who completed the survey were invited to participate in a semi-structured interview. An invitation package was then emailed to the eligible participants. The package consisted of participant information sheet, an offer to participate and a consent form. Participation in this study was voluntary and all participants were consented prior to the interviews. The interviews were conducted either face-to-face or over the phone. All interviews were conducted by a trained researcher who had no direct association with the LBVC implementation. A program logic model was used to identify the themes for semi-structured interviews. The interview guide was developed using current literature on value-based care and information from the implementation plan, project flow-charts, NSW LBVC documents and operational meeting notes. Some of these documents were also used during the data analysis to cross-reference against the project expectations and implementation. To maintain the unanimity of the interview questions, predesigned semi-structured questions were used. However, the interviewees were given the option to express or raise any concerns or issues with regards to the program.
All interviews were digitally recorded, transcribed verbatim and then de-identified. All participants were given an option to receive a copy of their transcribed interviews for their review. Participant recruitment and data collection stopped when the emergence and occurrence of key themes reached saturation. The qualitative data was progressively analysed with the aid of NVivo Qualitative Data Analysis Software V11. Transcripts were reviewed by researchers for accuracy prior to coding. Any feedback provided by the participants, was incorporated in the transcripts before coding.
Ethical approval for this study was granted by the South Western Sydney Local Health District Human Research Ethics Committee (Application Number: HE18/212: 2018/ETH00299).
3. Results
A total of 22 stakeholders (52% response rate) completed the survey. Over 82% (n = 18) were involved in the implementation of the LBVC initiatives for 14 months. Fifty-five percent of the respondents were clinicians as executive sponsors, clinical leads or part of a clinical team (Table 1).
While 54% of the respondents felt that the LHD is delivering better care for
![]()
Table 1. Demographic data of the participant.
patients via the implementation of LBVC, most of the participants (60%) were unsure whether it was better use of the financial resources invested in the initiatives (Table 2). Fifty-one percent of the participants reported that there was very good sharing of information and ideas within the LHD where clinicians were provided with data to support better decision making (77%). The stakeholders were overall moderately to very satisfied (60%) with how the program was implemented within the LHD.
When asking about the factors on resources and expertise supports from the local LHD, MoH and the relevant support health agencies, the respondents rated higher in the aspects: “Expertise, advice in support for the implementation of patient reported outcome and experience measures” and “Health Economist resources” (Table 3). However, when adding percentages for ratings “1” to “4”, i.e.: collapsing these columns, for each aspect, participants rated highest for negative perception of likelihood of adequacy for:
1) Economic resources (41%)
2) Staff resources (36%)
3) Expertise, advice in support for the implementation of patient reported outcomes and experience measures (36%)
4) Time to work with the initiative (36%)
Semi-structured interviews
A total of 10 interviews were conducted with Executive Sponsor (n = 3), Clinical Leads (n = 5) and project team (n = 2). Analysis of the transcripts identified four core themes linking different aspects of the implementation of the LBVC initiatives: 1) Engagement; 2) Understanding of implementation process; 3) Challenges; and, 4) Future strategies for implementation. In the following sections, short excerpts from these interviews are included.
![]()
Table 2. Response on the implementation of the program.
![]()
Table 3. Aspects on resources and expertise.
Engagement
Good communication is perceived to be a central factor in likely success of how LBVC initiatives are implemented and promote the engagement of the multidisciplinary team, patients and their carers in the management of their medical condition, and therefore contribute to the deployment of the LBVC initiatives. There was general satisfaction and reflections on the level of communication regarding the engagement and implementation of LBVC amongst those who were involved in the implementation of the program.
“I think communication was bit better than other programs I involved before. I know about the leading better value care program much more than my work colleague because I was involved with it”. (Clinical Lead)
Some participants stated that clinician engagement was likely to be hard to maintain because LBVC responsibilities are an add-on to the existing professional responsibilities increasing their workload. Extra tasks include auditing, patient reported measures and data collection/reporting, additional administration duties, increased communication and interaction with patients.
“It might be early to evaluate as an overall. Per initiative may be easier, as right now the engagement/burden is intensive at program level. The extra workload is great and unsustainable.” (Executive Sponsor)
Understanding of implementation process
The participants viewed LBVC as a “Positive” initiative and a number of keywords were noted: fantastic, great, good, dynamic and effective program.
“Yes, I think, it’s a fantastic initiative, and I think it has kept people out of hospital by managing them in the Non-admitted level” (Clinical Team)
The eight initiatives were identified for implementation as part of Tranche One of the LBVC program because of the existing guidelines and the recognition of the prior work and evidence by the specialty clinical network. One of the stakeholders determined the examination of how LBVC initiative is delivered as:
“A great opportunity… given there are strong evidences around this clinical area and a lot of changes around clinical practice and guidelines”. (Clinical Lead)
However, some participants believed that it may be too early for early benefit.
“In theory but have I seen the changes “no” “not so much” but I understand like the theory of where its heading and I could see some positive but I am not saying it’s changing much”. (Clinical Lead)
Challenges
Although there seemed to be adequate support from the LHD Project Team in the implementation of the initiatives, general comments were around the lack of local resources for the change of practices.
“There is not just enough from the ground clinical staff to make the changes that have been recommended, it’s just not feasible.” (Clinical Team) “Investment small compared to size of problem. Good initiative, but under investment will make outcomes/ burden relief difficult to realise” (Executive Sponsor)
There were a number of comments on the relevance of the audit tool developed by the supporting health agencies.
“The audit tool from ACI does not measure anything sensitively or provide any meaningful recommendations for any clinical changes in our LHD”. (Executive Sponsor) “I was very much run by what the ACI audit process looking at obviously the evidence by care and how a measurement of how we were doing as an area into local health district into managing patient based on evidence based care and then reviewing that thing where we were weak and trying to implement change to manage those things better. However, it’s time consuming but meaningless exercise”. (Clinical Lead) “What happens when someone discharged out of hospital, whereas most of the questioning on the audit were based on how-what was the management of the patient within the hospital system and I don’t think we looked at that areas where isn’t improvement well enough”. (Project Team)
Future strategies for implementation
There were numerous implementation and lessons learnt from this study. Some participants suggested further education not just for the implementation team but wider audiences including primary care and patients.
“I just don’t think that the people on the ground have got any idea of what leading better value care is about to be honest. So I think it’s communication/education as well across not just those that are involved in the process and not just the manager level, but I think the explanation that work with these patients needs to be better educated and have a better understanding.” (Clinical Team)
4. Discussion
This formative evaluation appraises the implementation of LBVC in a Local Health District. By using a mixed-methods approach, it allows the collective analysis of different data and improves the consistency of the observations.
Half of the respondents (80% clinical staff) in this study feel that the program is having a positive impact and there is good support provided from the LHD Project Team. This may be the result of the adopted governance structure and positive stakeholder engagement in the implementation, both in the uptake of the initiatives and also uptake of the formative evaluation and interviews. This local governance structure was acknowledged at the Leading Better Value Care Roadshow and Listening Tour: “Mixed approach: Clinical engagement in the study LHD. This study LHD provides one example of a governance structure that has enabled strong engagement. The change and project management approach allows clinical streams to lead the work, but still have a central point of contact driving the program across the district. Illustrating the good clinical engagement across this district, every audit for LBVC underwent an ethics approval process so that clinicians could publish results. The LHD also sent an LBVC evaluation survey to all staff at the end of the first year of implementation which elicited good clinician response. The district has aligned the program with their overarching vision of ‘transforming your experience’ which applies to clinicians, staff, patients and carers”.
The clinical leadership in the implementation of the LBVC also promotes sharing of information and ideas across the LHD. Mostly, clinicians acknowledged that they were provided with data to support better decision making. Discussions regarding clinicians’ concerns and their plans for care and treatment requires not only administrators but also clinical staff to put in place interdependent steps needed to improve value. The weakness of the audit tool has been a barrier and attributed to delay in the uptake of the initiatives. Audits were only carried out for a couple of cohorts (not all) and conducted in response to the Bureau of Health Information (BHI) report as no other clinical data are available. The key purpose of the audit is to partially fill those gaps and drive improvement as the “solutions” are too long coming.
Overall, the participants viewed LBVC as a “Positive” initiative. These results echo findings from other international studies which review and evaluate initiatives underpinned by value-based health principles (Gentry & Badrinth, 2017; Nilsson et al., 2017; Porter et al., 2018; Collden & Hellstrom, 2018). Gentry and Badrinath (2017) for example, reviewed value-based health care initiatives in England emphasising agreement on value in different contexts. Central to census is the opening up of conversations between health system stakeholders to apply value-based health care locally which takes time and resources. The authors canvass examples of procedure overuse and unwarranted variation where through consultative and evidence based approaches bring the notion of value and best use of resources into focus for change success (Gentry & Badrinth, 2017). Participants in a Swedish study ultimately see value-based health care as a worthwhile concept in terms of ‘value for patients’ and ‘measuring health outcomes (Nilsson et al., 2017). More time is necessary before actual outcomes and benefits can be revealed.
LBVC provides opportunities to create a more sustainable health system. An indicator of sustainability is the difference between the Business as Usual (BaU) predicted level of resource utilisation, that is, what might have happened if nothing had changed and the post-intervention actual (estimated) level of utilisation. This freed up capacity presents LHDs with the opportunity to repurpose and re-invest efforts towards other patient care priorities. In July 2019, NSW MoH analysis shows, on an annualised basis, that the study LHD was expected to achieve a better outcome against BaU than other LHDs for High Risk Foot Services, Osteoporosis Refracture Prevention and Osteoarthritis Chronic Care Program initiatives. This outcome was indicative of the successful application of the LBVC funding to reduce the reliance on admitted patient modalities and the better use of low cost non-admitted services which can improve health outcomes and reduce the need for more costly inpatient services. Whilst current trend analysis leads to this favourable result, ongoing close monitoring of the program is required. Patient reported measures and patient outcome measures must be conducted to support a true patient centred care approach.
While participants identify a number of potential benefits arising from LBVC, they also report various challenges and barriers in the implementation of the Tranche One initiatives including lack of resources and tight timeframes. Nilsson and colleagues (2017) qualitatively interviewed a diverse representation of project team members implementing value-based health care finding issues of time, increased workload and capacity (Nilsson et al., 2017). Enthusiasm exists for value-based health care as does similarly for LBVC, however ongoing staff engagement in initiatives is affected by other priorities and whether there is better use of financial resources by investing in the LBVC initiatives This emphasises the importance for staff to see the value for patients in the form of outcome measurements (Nilsson et al., 2017). The participants’ comments in this study reinforce the scale of the change, the resource implications and the potential limited buy-in from the broader range of clinical staff involved in these areas. The results of this study echoing from published literatures, are key features that might influence the acceptability/implementation of these kinds of wide-scale practice change initiatives.
There are reported gaps in staff knowledge that may need to be addressed through further education. Further promotion and education of value-based health care principles and what these mean locally for patients and health care professionals is important to invest in to improve chances of acceptance of initiatives in the overall journey towards embedding LBVC (Gentry & Badrinth, 2017). It is worthwhile for those implementing initiatives to be aware of the different meanings and attitudes given to value-based health care by the different stakeholders and to find commonalities aligning with value-based health care strategy through increased communication (Andersson et al., 2015). Favourable outcomes using the value-based health care approach depend upon all health system stakeholders aligning to the notion of value (Horne & Manion, 2019).
Participants feel that the roll-out of the initiative was a bit “rushed” and that it would benefit from more considered and structured implementation. Even though “Access to tools, guides and implementation resources” from the LHD Project Team and supporting agencies are rated relatively high in the survey, the tight timeframe set by the MoH has created significant challenges for the implementation. Published literature provided limited detail on how other programs are implemented and no-one has gone into the specifics of it and how it transforms health care delivery or what is the best design for the program. Drawing on translation theory to inform consolidated framework of implementation research, Collden and Hellstrom (2018) note importance of local interpretations to the larger concept of value-based health care for stakeholders charged with its implementation. Local smaller iterations of the broader concept can in turn can act as accompaniments to improve the conditions for change to gain the desired outcomes (Collden & Hellstrom, 2018). The notion of translation as being complementary suggests that perhaps a more localised approach to implementation could occur to help counter stakeholder sentiments that timeframes were ‘tight’ or the audit tool lacked sensitivity through more localised planning and audit tool design. These viewpoints align with clustered mid-range survey responses for likelihood ratings of adequacy of supports and resources. This could indicate a need to bring supports and resources to the forefront of implementation more robustly with increased opportunities for peer mentoring, collaboration and capability development activities more locally. Furthering these opportunities will also assist to improve communication and understandings of the value-based health care concept.
Limitations of this study include the small sample size and the study only covers one LHD so this may limit the generalizability or applicability of the study outcomes. However, theexperiences of the participants nonetheless provide insights into implementation and contribute to knowledge for implementing future initiatives. The study design is dependent on volunteer participation for two arms of the study: surveys and interviews. Even though the uptake of the participation (56% for survey and only 10 participants for the semi-structure interview), the information received is believed to reach the data saturation. The investigator team does not believe that the sample size and purposive sample characteristic have any significant impact on the results for this study.
One of the background aims of this study was to identify and develop research questions based on the study experience. This study provides a process evaluation (Smith & Ory, 2014) for a state-wide initiative being implemented at a local level. Measuring patient experience is beyond the scope of this study as focus is on implementation of the program initiatives. Patient voice is currently being incorporated in the form of the Patient Reported Measures (PRMs) initiative covering experiences and outcomes of the LBVC initiatives. PRMs is still in its early stages of state-wide and local level implementation and offers a potentially effective platform to capture patient experiences and outcomes in the LBVC context. Broader scope for evaluation is also important to identify the best ways of ensuring sustainability, identify outcomes related to the effect on patient care and provide lessons for other regional health service interventions.
The results of this study will be ideally placed to inform future policy amendments in the area of practice change, as well as serving as a guide on implementing new models of care in the future, its sustainability and how it aligns with the NSW MoH strategic direction.
5. Conclusion
This study has provided experiences of stakeholders participating in the implementation of the LBVC program and about how it was being implemented across the Local Health District.
This study has also served to increase understanding of the practical issues for the implementation. While suggestive of being useful for the year one implementation, further analysis and research are required to demonstrate that the LBVC initiatives have improved patient outcomes and experience. Future projects should be targeted to appropriate patient cohorts and include patient outcomes and experience data.
Funding
This research received no specific funding.