Exploring the Impact of a Structured Model of Journal Club in Allied Health — A Qualitative Study

Background: This paper presents the findings of a qualitative study associated with an experimental trial which examined the impact over time, of a structured model of journal club (JC) on the knowledge, skills and evidence uptake of allied health practitioners (AHPs). The primary aim of this qualitative study was to explore the experiences of AHPs regarding the use of iCAHE (International Centre for Allied Health Evidence) JC as a medium for evidence uptake. The secondary aim was to explain the quantitative findings in the iCAHE JC trial based on the perspectives of the JC members. Methods: Semi-structured individual interviews with purposively selected participants from the JCs were undertaken. The participants were asked about their experiences with the iCAHE JC, and to comment on the results of the iCAHE JC trial. Analysis of data was undertaken using the principles of the framework approach. Findings: A total of 12 AHPs participated in the interview. Their experiences of the JC were classified into seven themes: knowledge and confidence gain, convenience, empowerment, evidence utilisation, impediments, peculiarities, and refinements to iCAHE model. The participants found the current structure of JC suitable and useful in generating creative thinking about how practices can be influenced by research evidence. They identified ways of how they have used research evidence to inform their clinical decisions and described circumstances when research findings are not applicable to practice. The participants thought that the variability in EBP outcomes across disciplines found in the iCAHE JC trial was not surprising given that allied health disciplines operate using different models of care, and vary in terms of culture, attitude, professional orientation and the volume of evidence base available in their specific disciplines. There were a few minor suggestions to modify the current iCAHE model of JC. Conclusion: When lack of EBP knowledge and skills, and limited access to evidence sources are reported as barriers, they can be effectively addressed by running a structured JC such as the iCAHE JC. In instances when barriers other than lack of knowledge are present, more than one approach may be required. Future research should determine the impact of other approaches that may be integrated with the iCAHE JC to promote evidence uptake and sustain practice behaviour change.


Introduction
This paper presents the findings of a qualitative study associated with an experimental trial which examined the impact over time, of a structured model of journal club on the knowledge, skills and evidence uptake of allied health practitioners.

The Importance of Using Multiple, Integrated Approaches in EBP Evaluation
Outcomes research in general has traditionally used quantitative methods to measure the impact of interventions on health care outcomes (Curry et al., 2009).In evidence-based practice (EBP) studies specifically, there used to be a strong emphasis on evidence associated with quantitative, experimental approaches, with little regard for the value of other types of research such as qualitative studies (Miles et al., 1997(Miles et al., , 1998;;Mitchell, 1999).More recently there has been increasing recognition that quantitative methods have limitations in capturing more complex aspects of healthcare delivery, such as organizational context and perceptions of patients and practitioners (Petticrew & Roberts, 2003).An approach that draws on diverse sources of evi-dence can provide a more comprehensive understanding of these complex issues.
Qualitative research has a role to play in addressing questions which cannot be investigated by experimental studies (Goldsmith et al., 2007).Experimental studies can be used to explore effects following deliberate interventions, but unlike qualitative research, it cannot describe the complexity, breadth and range of occurrences or phenomena (Curry et al., 2009).Qualitative methods can contribute in several ways to the evaluation of health-related interventions including EBP interventions.They are useful in gaining insight into the potential reasons or mechanisms associated with an outcome and generate hypotheses about such mechanisms (Lewin et al., 2009;Curry et al., 2009).They can also explore individual perspectives and responses to the intervention and examine variations in effectiveness within a sample (Lewin et al., 2009;Curry et al., 2009).
The use of both quantitative and qualitative approaches in the evaluation of strategies to facilitate EBP uptake has become increasingly common because they provide a broader understanding of research issues than ither method alone (Robins et e Table 1.Summary of the iCAHE journal club trial.

Design
A pre-post study design without a control group

Intervention
The intervention consisted of six monthly JC sessions using the iCAHE model.

Data Collection
Measurements were collected prior to the implementation of the iCAHE JC with follow-up six months later.The Adapted Fresno Test (AFT) assessed knowledge and skills in the major domains of EBP, such as formulating clinical questions, searching for and critically appraising research evidence., 2008).The two approaches complement one another, such that the qualitative method will "explore and obtain depth of understanding as to the reasons for success or failure to implement EBP or to identify strategies for facilitating implementation while quantitative methods are used to test and confirm hypotheses based on an existing conceptual model and obtain breadth of understanding of predictors of successful implementation" (Palinkas et al., 2011).
The iCAHE (International Centre for Allied Health Evidence) Journal Club Trial Lizarondo et al. (2012) examined the impact of a structured journal club (JC), known as iCAHE JC, on the knowledge, skills and behaviour of allied health practitioners relevant to EBP using two established instruments, the Adapted Fresno Test (McCluskey & Bishop, 2009) and Clinical Effectiveness and Evidence-based Practice Questionnaire (Upton & Upton, 2006).A summary description of this trial is provided in Table 1; a complete description of the implementation of this intervention is reported elsewhere (Lizarondo et al., 2012).
The primary aim of the research reported in this paper was to explore the experiences of allied health practitioners regarding the use of iCAHE JC as a medium for evidence uptake.The secondary aim was to explain the quantitative findings in the iCAHE JC trial based on the perspectives of the JC members.

Ethics
This study was approved by the University of South Australia Human Research Ethics Committee and the Human Research Ethics Committee (Tasmania) Network.

Participants
The participants for the semi-structured interviews were recruited by email invitation through the facilitator of the JC who was in monthly contact with the primary author.The facilitators were asked to assist with identifying key informants for the interview from the pool of AH providers who participated in iCAHE JC.The authors aimed to involve practitioners from L. M. LIZARONDO ET AL. every iCAHE JC, from different allied health disciplines.

Data Collection
Semi-structured individual interviews were undertaken with participants, by the primary author in a private and quiet room.Individual interviews, in particular, are useful when detailed information to explore issues is required (Liamputtong & Ezzy, 2005;Sandelowski, 2002).They provide more in-depth information than is available in other data collection method such as focus group interviews.The participants were asked about their experiences with the iCAHE JC and differences it might have made, if any, to their clinical practice.The results of the iCAHE JC trial were then presented and the participants were asked for their reactions and the reasons they attributed to the apparent success of the JC to some allied health practitioners and reasons for the absence of positive change in the other practitioners.The following broad questions were asked: 1) What are your views/perspectives of the iCAHE JC that was organized in your department?How well did the staff embrace the JC project?
2) What did you like most/least about the iCAHE JC?
3) How can we improve the JC process?4) What is the impact of the JC to your practice?5) Do you use the evidence obtained from JC meetings to inform your practice?Why/Why not? 6) What are your thoughts regarding the results of the iCAHE JC trial?7) Are the results something you would have expected?Why/Why not?

Data Analysis
All interviews were audio-taped, and transcribed by an independent transcription service.Analysis of data was undertaken using the principles of the framework approach (Pope et al., 2000), which includes the following stages: 1) Familiarization-immersion in the raw data by reading the transcripts more than once; 2) Determining a thematic framework-identifying themes by which the data can be examined and referenced; 3) Indexing-applying all the themes to all the data in textual form by labeling the transcripts with codes; 4) Charting-rearranging the data according to the appropriate part of the thematic framework to which they relate, and forming charts; 5) Mapping and interpretation-using the charts to inform the key objectives of the research.
Two authors jointly coded the data and identified themes (LL, SK) and another author (KGS) reviewed the analysis to ensure accuracy of the interpretations.

Investigator Perspective
The primary author (LL) has been the project officer of all the iCAHE JCs for several years.The other authors (KGS, SK) were responsible for the conceptualisation and development of the iCAHE model of JC.

Findings
A total of 12 allied health practitioners participated in the interview, and comprised two physiotherapists, two dieticians/ nutritionists, three speech pathologists, two social workers and three occupational therapists.More than half of the participants were facilitators of an iCAHE JC.The majority had completed bachelor's degree and most held senior positions.
The experiences of the JC members were classified into seven themes: knowledge and confidence gain, convenience, empowerment, evidence utilisation, impediments, peculiarities, and refinements to iCAHE model.

Knowledge and Confidence Gain
There was consensus among participants concerning the impact of JC on their level of knowledge and skills relevant to EBP.They agreed the JC sessions improved their ability to formulate a clinical question and helped them critically examine the quality of research evidence.The participants felt more confident in appraising research articles which allowed them to obtain relevant information that can be integrated with clinical decisions.
"I think people got a bit more savvy with their knowledge… they were sharper at picking things which are not right or picking gaps.People start talking about power calculations and things like that which we never talked a lot before." Another participant said: "People have become more confident and now they are able to identify study design and they also know how to look a little bit more at the details of the study.I really think people did improve." The allied health practitioners described JC discussions to be very informative and facilitated broad learning which made them think more deeply about what EBP really meant.Many participants felt that JC discussions fostered a critical and reflective way of thinking that went beyond the EBP skills (i.e.formulating a question, critical appraisal) that were being purposefully taught.
"So I think the benefit-the increased understanding of levels of evidence happened and we did discuss that but the real power comes through the discussion that came after that.It's the reflection of what should be done with patient care and how best to achieve that."

Convenience
Concerns about the lack of time to search for research articles and limited information technology skills have been identified as major barriers to EBP (Bennett 2003, Jette et al., 2003).According to JC members, the searching for relevant articles undertaken by iCAHE researchers served to address these barriers, which made the JC process a lot more convenient for busy practitioners.The participants reported that JC members were motivated to participate because the iCAHE JC model takes the load off them in terms of searching for literature.
"Having a pool of articles to pick from was fantastic!Without having to do that research side of it ourselves was very helpful.Also, having someone else gone through the searching whom you know is an expert in the field is just handy." The participants valued the structured format of the critical appraisal and felt that the appraisal summaries provided to the facilitators was helpful.The facilitators expressed that leading the discussion was challenging, especially during the first few JC sessions, but having the appraisal summaries made them feel more certain and confident of their skills.
"I think it was really helpful having a structured format, and the same format that we all use to critique a journal, it just means we could be more consistent in our discussions, that we weren't missing sort of key parts."One of the facilitators explained: "I think probably one of the things that were a bit challenging is being able to look at the data and interpret that, and know whether that was statistically significant and if there were enough participants and things like that.So I found it useful that we got some information about that in the summaries that we receive from you before the JC meeting."

Empowerment
The participants described how they felt empowered to take responsibility for improving their practice through the integration of research evidence with their clinical decisions, although they may have been initially sceptical to the principles of EBP.They felt a sense of ownership of the journal club despite obtaining strong support from iCAHE.The participants thought the regular meetings where they discuss clinical questions/cases relevant to their practice and the sharing of clinical experience was very informative.
"What I like most is the joint ownership of doing it.I think it raises the bar in terms of collective responsibility for looking at evidence and presenting it.I think team meetings such as this are really empowering… it raised the participation rate of all the members in presenting and thinking through so I think the incorporation of everyone's involvement was one of the best things about it." One of the participants added: "It's always good to have discussion with co-clinicians and then as a group pool our expertise and made a conclusion ourselves really."

Evidence Utilization
When asked about the usefulness of research findings discussed during JC meetings, the participants reported they were valuable, and often they use them to validate their clinical decisions and actions.Many of the JC members felt there wasn't always a need to change clinical practice because their current practice is often supported by evidence.
"Okay so there is good evidence out there, but what we are doing is what's according to the evidence.It was more of reinforcing the things we do, although we were mindful that if there is something that needs to be changed, we will have to." The practitioners additionally reported they use research evidence to educate patients who felt more confident that the treatments they are receiving are no less than the best possible care.
"The most common place I use evidence is in patient education, so to be able to say-look the best evidence out there says that this is what we should be doing.So if I know that what I've found out there shows that this is effective, and it's been shown that recently, then I guess that changes clients' perspectives and perception of what's important." Some participants suggested that research findings from JC sessions facilitated discussion of current practice which in turn shaped future strategies or approaches to treatment.One of the practitioners commented: "What we read from the article helps us to frame or start thinking about a service that we can potentially provide clients; it might help us shape our intervention.The research evidence provides us opportunities to decide where we're going." The participants believed in the value of continuing education and felt that it was part of their professional responsibility.They recognised that allied health practice is continually evolving and therefore it is important that their professional knowledge is constantly updated.The JC members also understood the need to be constantly updated of the changes and developments in their practice which they can learn by reading or discussing current literature.
"I think the most important thing is when there is good evidence out there; we should know what it is."

Impediments
The evidence from research was seen as fundamental to clinical decision-making, but practitioners felt there were instances when they were not feasible.For example, one of the JCs discussed in their meeting the most effective intervention for a particular condition and found that the highest and best available evidence required very intensive form of therapy.One of the participants said: "I think a lot of it came down to resources that are available to us, so a lot of the therapeutic interventions that had the highest level of evidence were very intensive, and I suppose within our service, it is difficult because we work in a community so we're servicing people who live hours away, and to sort of give them twice-weekly therapeutic interventions is a bit of a challenge." Another participant commented: "For example, an hour of therapy thrice a week is sure to be effective but it's not applicable if there's only point five staffing in that post." Lack of skills to implement the recommendations from research was also described as a major barrier to EBP uptake.

"I can read about and discuss research and go yes that's great, but my actual skills and ability to use that technique from that study without having some extra training or working with somebody who has got experience in using it successfully, I don't know that I am necessarily able to implement that on my own."
There were participants who felt disappointed because of the limited research available to address the clinical queries they have on certain clients.
"I suppose one of the problems is sometimes there's very little information on the things that we're asking, and that's really frustrating." In some instances, however, while there were high levels of evidence about the JCs' topics of interest, the findings are not relevant to their practice.
"Some of the RCTs, even though they sort of address our queries, are not of intrinsic value to everyday practice because they were far too removed.We were hoping to find something a bit more relevant to us.It just wasn't something we could act on."

Peculiarities
Towards the end of the interview, the participants were presented with the results of the quantitative study into JC effectiveness.The quantitative findings showed variability in EBP learning and behaviour outcomes across disciplines following participation in the iCAHE JC.The participants were shown that only the physiotherapists demonstrated improvements in all outcomes (i.e.knowledge, attitude and behaviour); speech pathologists and occupational therapists demonstrated an increase in both objective and perceived knowledge but not for attitude and EBP uptake; social workers and dieticians/nutritionists showed positive changes in their objective and perceived knowledge, and EBP uptake but not for attitude.
The participants commented the results were consistent with their expectations.They were not surprised to find that other disciplines scored better in some outcomes whereas others did not improve.There were participants who reported differences in the model of care by which their discipline operates.For example, one participant said: "The model of care in allied health would differ; it can be a medical model for the physios whereas with OTs we go by an occupational model.For physios for example, they give exercises pre and post whereas with OTs it's comprehensive-you consider their occupation, family situation and things like that."There were some who described differences in terms of professional attitude and culture.As one participant commented: "I guess physios are quite known for being very scheduled, and when things are mandatory, they really are mandatory whereas some other disciplines are more flexible in their approach.As social workers, we tend to focus more on process and systems and outcomes whereas others can be a bit more clinical and a bit more focused." The participants thought allied health disciplines could be grouped into at least two distinct communities-clinical or biomedical scientists and social scientists.They speculated that physiotherapists, speech pathologists, dieticians/nutritionists lean towards clinical or biomedical science while occupational therapists and social workers consider themselves under the field of social science.The participants perceived that these two groups tend to cluster around quite specialised bodies of knowledge and preferred research methodologies.
"Right from the beginning there's a difference.The first thing probably is I think we, in social work, see ourselves more as a social science than a biomedical science, so already we're kind of different to other disciplines like physios and speeches and so on.Second, we are more interested in context and individual experiences-you know, in what way did this happen, or which type of event was it-more about the richness of data which is more leaning towards qualitative research." There were some participants who recognised that the evidence base is often different for every discipline.They suggested that some disciplines have a lot more research to inform their practice than other disciplines.One participant commented: "I think physio has been a sort of longer standing allied health profession in some aspects, and a real branch directly from the medical field.So I think there is a lot more research, this is just from my understanding, a lot more research that advises physiotherapy practice."

Refinements to iCAHE Model
Overall, the participants were positive about the iCAHE JC model and felt that it was a worthwhile experience.However, they believed that there were still some opportunities for improvements which could increase its effectiveness.The most telling comments came from the facilitators who felt that being responsible for leading the JC discussions every meeting was challenging and difficult at times.They thought that the role of the facilitator should rotate every meeting so that every member will have a sense of responsibility and ownership of the club.
"I was probably the main person responsible for it, and that was a little bit hard to sustain, just because of time.It would have been better if they all did it, so the facilitating and the discussion, everything like that, so it wasn't just all on one person to arrange." There were participants who noted that the training provided to facilitators at the start of the iCAHE JC focused on intervention types of clinical questions which highlighted the superiority of randomised controlled trials and other experimental designs over observational studies and case studies.They also felt that quantitative studies were favoured more than qualitative studies as sources of evidence.The participants suggested that EBP training should not only focus on the hierarchy of evidence that includes quantitative studies but should include both quantitative and qualitative studies as important in informing clinical decisions.
"I think there was angst about the privileging of the hierarchy-you know that medical kind of quantitative hierarchy.We actually got some literatures that were more specific with social work and evidence-based practice but the facilitators talked about us not fitting that model and that we should find higher forms of evidence."

Discussion
The current study explored how the participants in the iCAHE trial experienced the intervention (i.e.iCAHE JC), and the reasons for the findings in the trial.The themes that emerged from this study confirmed that a JC is an effective strategy in improving EBP knowledge, skills and confidence of allied health practitioners.The participants found the current structure of JC suitable and useful in generating creative thinking about how practices can be influenced by research evidence.They identified ways of how they have used research evidence to inform their clinical decisions and described circumstances when research findings are not applicable to practice.The participants thought that the variability in EBP outcomes across disciplines found in the iCAHE JC trial was not surprising given that allied health disciplines operate using different models of care, and vary in terms of culture, attitude, professional orientation and the volume of evidence base available in their specific disciplines.There were a few minor suggestions to modify the current iCAHE model of JC.

iCAHE as an Educational Tool
Journal clubs have long been considered in the medical profession as a tool for improving reading habits (Linzer et al., 1987;Linzer et al., 1988;Khan et al., 1999), critical appraisal skills and research knowledge of practitioners (Seelig, 1991;Burstein et al., 1996;Spillane & Crow, 1998;Khan et al., 1999;Macrae et al., 2004;Mukherjee et al., 2006).Similar outcomes were reported in the nursing profession (Thompson, 2006;Steenbeek et al., 2009).In allied health, while there were a number of articles which described the potential benefits of a JC, there was no study which explored the learning experiences of JC members.The findings from the current study make a significant contribution to the existing evidence about the effectiveness of JCs in educating practitioners about the processes involved in EBP.Overall, the JC participants were satisfied with the model and found the JC discussions very useful in understanding EBP concepts such as critical appraisal, which are congruent with the results of the iCAHE JC trial.The convenience experienced by the practitioners because of the support from iCAHE researchers tends to fit well with the clinical demands of allied health practice.Such experience increased their motivation to engage in the learning activity (iCAHE JC), which could explain the positive impact of the JC on learning outcomes.Moreover, participants felt empowered following participation in the JC, despite the strong support offered by iCAHE."Individual empowerment means people feeling and actually having a sense of control over their lives" (Woodall et al., 2010).To empower individuals, it means building their confidence or enhancing their personal skills in order for them to make choices (Woodall et al., 2010).The participants of the JC assumed a degree of control by having input on the learning targets every JC session.Establishing an empowering learning environment appeared to have positively affected the practitioners' motivation to learn which potentially led to improved knowledge and skills.

iCAHE as a Medium for Facilitating Change in Practice
Becoming an evidence-based practitioner requires skills and knowledge in terms of formulating a question, searching for relevant evidence, critically appraising and using research evidence.However, while knowledge is necessary, it is not sufficient in itself for practice behaviour change.In allied health, lack of knowledge and skills to undertake the EBP processes and the lack of access to research evidence are commonly reported barriers to EBP (Bennett, 2001;Jette et al., 2003;Heiwe et al., 2011).The findings of the experimental study and this qualitative study showed that a JC approach, particularly a structured model such as iCAHE JC, can address these barriers.This study also found that the iCAHE JC was instrumental in providing the evidence base which allied health practitioners used to educate their clients, discuss and reflect about future strategies, and validate their current practice.However, there were other barriers to evidence uptake in allied health which cannot be addressed by participation in a JC alone.There were practitioners who expressed that lack of requisite skills to implement the recommendations based on research was a significant barrier.There were others who described that their setting did not have the resources or staff time to deliver the recommended interventions.Other challenges include factors such as limited evidence base or recommendations not being relevant to the local patient population or practice.The participants also highlighted differences across allied health disciplines, in terms of models of care, attitude, culture, evidence base and professional orientation (i.e.biomedical versus social science).These observations demonstrate how the characteristics of the practitioners, aspects of the practice setting, and organisational context can influence the use of research-based recommendations.Experts in the field of evidence implementation suggest that a comprehensive approach at various levels is needed to address the individual professional, teams, organisations and wider systems (Grol & Grimshaw, 2003;Estabrooks et al., 2007).The EBP inter-vention/s should therefore be linked to the barriers identified at each level.
While the evidence highlights the effectiveness of iCAHE JC in addressing one particular barrier (i.e.knowledge barrier), it appears that a JC approach is an important medium that nurtures a different kind of thinking (i.e.reflective and critical reasoning); a level of thinking which is crucial in changing one's behaviour.This was highlighted by the participants who felt that the learning experience went beyond just acquiring the knowledge about EBP processes.As Buswell (1998) reported, there are two compelling reasons why a JC can serve to bridge the gap between evidence and practice.First, it can develop analytical, reflective and evaluative skills and second, it enables individuals to disseminate the results of research.Therefore, a JC, while it cannot address all the other barriers, plays a significant role in creating a higher level of thinking required in individual practitioners in order to change their practice behaviour.

Implications for Practice
To develop a successful intervention that will facilitate evidence uptake, there needs to be a careful understanding and consideration of the barriers faced in healthcare.The choice of EBP intervention should then be linked to the identified barriers, and guided by the local circumstances or context.When lack of EBP knowledge and skills, and limited access to evidence sources are reported as barriers, they can be effectively addressed by running a structured JC such as the iCAHE JC.In instances when barriers other than lack of knowledge are present, more than one approach may be required.Managers and policy makers who implement interventions should understand that different approaches will be effective for different practitioners and different settings.

Implications for Research
Further research should determine the impact of other approaches that may be integrated with the iCAHE JC to promote evidence uptake and practice behaviour change that is ecumenical and sustainable.Exploring which types of approaches work for whom and in what circumstances and context, requires considerable careful future investigation.It may also be worthwhile to identify a specific area of practice (e.g.role of alternative therapies in cognitive function) and carefully analyse the current practice before strategies are designed.

Limitations
As in any other research, there are limitations in this research which need to be considered when interpreting the results.First, the facilitator of the interview is the project officer of iCAHE JC which may have limited the participants from being critical of the iCAHE model.Second, because the participants volunteered in the study, they are likely to represent only the practitioners who may have been more motivated to change their practice than the average allied health practitioner.

Conclusion
The findings suggest that allied health practitioners found the iCAHE JC an effective tool in improving their knowledge and confidence about EBP, and that the current structure is convenient to busy practitioners.The participants considered the research evidence obtained from JC meetings useful in validating their current practice, educating clients and determining future strategies.However, there were other factors which influenced their uptake of research evidence, including lack of skills and resources to implement evidence-based recommendations, limited evidence base in some areas of practice and research findings not being applicable to local practice.These findings, along with observations that allied health disciplines vary in their attitude, culture, models of care, evidence base and professional orientation, underscore the need to design and implement targeted EBP interventions to facilitate evidence uptake and behaviour change.Future research should explore the impact of these approaches when integrated with a JC approach as an educational tool.
The Evidence-based Practice Questionnaire measured EBP uptake, attitude to and perceived knowledge (self-reported knowledge) about EBP.