Received 28 January 2016; accepted 27 March 2016; published 30 March 2016

ABSTRACT

The use of Progress Monitoring (PM) measures has been shown to improve outcomes in therapy for clients who do not follow the normal trajectory of improvement. In addition to improved outcomes, there are several other documented benefits of PM that may motivate clinicians to use PM. Research has examined the broader field of selecting mental health care quality assessment tools and a review of the literature has pointed to the importance of considering motivation for assessment when selecting a measure. However, how motivation influences the selection or maintained usage of PM measures has not been studied. This study examined initial motivation as well as measure selection and continuing use of PM. Consensual Qualitative Research methodology was applied to characterize how clinicians (n = 25) started, selected, and maintained use of PM measures and how initial motivation related to measure selection and continued use. Regardless of initial motivation, convenience and effectiveness emerged as important when selecting and continuing to use a measure. Results are compared to current frameworks for selecting mental health- care quality indicators. Our results suggest that PM measures need to strike a balance, emphasizing convenience as well as efficacy in order to improve clinical uptake and adherence.

Keywords:

Progress Monitoring (PM), Outcome Monitoring, Evidence-Based Practice, Consensual Qualitative Research (CQR), Measure Selection

1. Introduction

Progress Monitoring (PM) measures are tools that have been developed to track client progress over the course of psychotherapy to identify clients who are not progressing. The use of these measures has been shown to improve outcomes in therapy (Anker, Dunacan, & Sparks, 2009; Reese, Norsworthy, & Rowlands, 2009; Reese, Toland, Slone, & Norsworthy, 2010; Schuman, Slone, Reese, & Duncan, 2015; Slone, Reese, Mathews-Duvall, & Kodet, 2015) , particularly for clients who do not follow the normal trajectory of improvement (Lambert, Harmon, Slade, Whipple, & Hawkins, 2005; Lambert & Shimokawa, 2011; Lambert et al., 2001) . The tools can increase efficiency, allowing not-on-track clients to receive more sessions and on-track clients to receive fewer (Harmon et al., 2007; Lambert et al., 2001; Reese, Norworthy, & Rowlands, 2009; Schuman, Slone, Reese, & Duncan, 2015; Slone, Reese, Mathews-Duvall, & Kodet, 2015) . Efficient allocation of resources has both clinical and financial advantages. In addition to improved outcomes and efficiency, clinicians have indicated that they find outcome measures useful for altering treatment, practicing ethically, determining strengths and weaknesses, meeting managed care or employer requirements, carrying our research, and practice marketing (Hatfield & Ogles, 2004) . Despite the well-documented benefits of PM, the majority of clinicians have not incorporated these tools into their practice (Hatfield & Ogles, 2004, 2007; Ionita & Fitzpatrick, 2014) .

Practical concerns have been identified as one reason that clinicians do not use outcome measures (Hatfield & Ogles, 2007) ; one practical challenge may be the burden of selecting of a measure. A variety of PM measures currently exist and vary with respect to what they assess, target populations, administration procedures, scoring/ feedback/interpretation procedures, cost, training required and privacy (Overington & Ionita, 2012) . Considering all these variations, selecting an assessment tool can be a daunting task.

1.1. Selecting Measures for Mental Health-Care Quality Assessment

Research has not yet examined how clinicians come to select a PM measure. To better understand this process, we examined the broader field of mental health-care quality assessment. In this field, an overwhelming number of assessment measures are available (Froyd, Lambert, & Froyd, 1996; Hermann et al., 2000) . The number and heterogeneity of quality measures inhibits comparisons and limits the development of broader understandings. Proposals have been made to adopt a small set of “core measures” to reduce the burden of selection, increase comparability of services, and focus resources on the most promising measures (Hermann & Palmer, 2002) . Several frameworks have been created to aid organizations, researchers and clinicians in developing such a core set (Burlingame et al., 2005; Burlingame, Lambert, Reisinger, Neff, & Mosier, 1995; Erbes et al., 2004; Green & Gracely, 1987; Hermann et al., 2006; Hermann et al., 2004; Hermann & Palmer, 2002; Lambert, Ogles, & Masters, 1992; Slade, 2002; Waraich et al., 2010) .

1.2. Existing Frameworks

We conducted a review of the existing frameworks for selecting mental health-care quality indicators and established a rank order of the most commonly identified desirable measure features. The frameworks most commonly cited factors were cost, psychometrics, suitability to the target population, and comparability as selection criteria (see Table 1). The second most commonly cited suggestions referred to the measure’s evidence-base, comprehensibility, interpretability, and domain selection or coverage. Other commonly cited features included feasibility, appropriate case-mix adjustment, and sensitivity to change. From a researcher perspective, these are key factors to consider in the mental health-care quality assessment (Burlingame et al., 2005; Burlingame et al., 1995; Erbes et al., 2004; Green & Gracely, 1987; Hermann et al., 2006; Hermann et al., 2004; Hermann & Palmer, 2002; Lambert et al., 1992; Slade, 2002; Waraich et al., 2010) .

However, these may not be the features of greatest interest to practitioners. Because of a substantial practice- research gap in psychotherapy (Stewart & Chambless, 2007) , it is equally important to explore how clinicians actually come to select a measure. Clinicians’ motivation for assessment likely influences their selection and continued usage of measures. Although several of the cited frameworks emphasize the importance of considering individual stakeholder needs (Hermann et al., 2006; Hermann et al., 2004; Hermann & Palmer, 2002; Waraich et al., 2010) , how motivation for assessment influences selection criteria or maintained usage has not yet been studied. Insight into this process may assist in the development and improvement of the measures, and ultimately increase the uptake and adherence of PM.

This study had three objectives: 1) to understand what motivated a group of clinicians to start, select, and maintain use of PM measures, 2) to compare existing frameworks for selecting measures to clinicians’ actual

^a Hermann & Palmer, 2002 ; ^b Hermann et al., 2004 ; ^c Green & Gracely, 1987 ; ^d Lambert, Ogles, & Masters, 1992 ; ^e Hermann et al., 2006 ; ^f Waraich et al., 2010 ; ^g Burlingame, Lambert, Reisinger, Neff, & Mosier, 1995 ; ^h Slade, 2002 ; ⁱ Erbes et al., 2004 ; ^j Burlingame et al., 2005 ; ¹Evidence-based; ²Psychometrics; ³Valid; ⁴Reliable; ⁵Feasible; ⁶Affordable; ⁷Comprehensible; ⁸Interpretable; ⁹Comparable; ¹⁰Available Norms; ¹¹Available Benchmarks; ¹²Standardized; ¹³Stakeholder Needs; ¹⁴Operationalized Components; ¹⁵Case Mix Adjustment; ¹⁶Under User’s Control; ¹⁷Domain Selection/ Coverage; ¹⁸Address Knowledge Gap; ¹⁹Suitability for Target Group; ²⁰Usefulness Across Settings; ²¹Compatible with Various Theoretical Orientations; ²²Procedural Simplicity; ²³Means of Treatment Effects; ²⁴Brevity; ²⁵Sensitivity to Change.

selection and maintenance of use behavior, and 3) to understand how initial motivation relates to selecting a measure, and to continuing to use a PM measure in practice.

2. Method

2.1. Participants

Twenty-five clinicians (i.e., counselors, mental health workers, psychologists, social workers, and marriage and family therapists) who were currently using PM measures participated in the study. The majority of participants were male (76%), over the age of 51 (64%), and working in private practice (56%). Additionally, the majority of participants were using the Partners for Change Outcome Management System (PCOMS) (Miller, Duncan, Sorrell, & Brown, 2005) (76%). When asked to rate how much of their practice was influenced by various theoretical orientations, 56% of clinicians rated Eclectic/Integrative as the most influential (see Table 2).

2.2. Procedures

Participants were recruited in three ways. Some participants were invited to participate after completing a national survey on the usage of PM measures. Others were recruited via notices on psychotherapy related social

networking webpages (e.g., Facebook, LinkedIn, etc.). Lastly, notices were distributed via PM measures’ host companies or creators to known PM measure users. Notices included a brief description of the study and a statement indicating that all participants would be entered into a draw to win a $100 Amazon gift certificate to thank them for their participation. The notices also included a link to a secure website, where participants could provide informed consent and indicate a telephone number where they could be reached for the interview.

Semi-structured interviews were conducted as part of a larger program of research investigating PM use among clinicians. The interview was piloted with six graduate students, two clinical professors, and two clinicians. Participants were provided with a brief description of PM measures and a few examples; they were then asked to indicate which PM measures they used and how often they used them. The interview then inquired about participants’ experience using PM measures. Lastly, demographic information was collected. After the interviews were completed, each participant was asked to provide feedback on the interview process. Participants all reported that they felt that the interview was able to capture their experience with these measures and no changes were suggested. All interviews were conducted via telephone and lasted between 18 and 53 minutes.

2.3. Data Analysis

Consensual Qualitative Research (CQR) was selected for its ability to characterize similarities across participants, while also providing an in-depth examination of the participant experience. Its use of consensus among multiple researchers honors multiple perspectives, thereby enhancing trustworthiness (Hill, 2012) . CQR follows a process of developing domains, where general topic areas discussed by the participants are identified; coding domains, where chunks of interview data are partitioned into the domains; constructing core ideas, where summaries of the data that capture the essence of the participant’s statement relating to the domain are created; cross analysis, where data is compared across participants by placing all participant’s core ideas into representative categories; and determining representativeness of themes, where frequency information for each category is calculated. Auditing occurs at several points: after developing the domains, coding the domains, constructing core ideas, and performing the cross analysis (Hill, 2012) .

Six female researchers (i.e., two PhD students, one MA students, two undergraduate students, and one professor) were involved in the present study. One of the doctoral students and the professor served as auditors throughout the process, while the remaining team members served as CQR coders. The other doctoral student conducted all the interviews. All researchers who were unfamiliar with CQR were trained in this research method. Prior to data collection, the research team discussed their beliefs regarding PM measures to minimize the influence of biases. The doctoral students and professor believed that PM measures are useful in clinical practice; however, only one doctoral student and the professor has experience using them in practice. The undergraduates had no prior beliefs about outcomes monitoring. These biases were acknowledged and discussed in consensus and auditor meetings to reduce their influence on the results.

Four undergraduate student researchers transcribed the interviews and checked each transcript for accuracy. After reading several transcripts, three coders obtained consensus on a domain list, which was then audited and adjusted to incorporate the auditors’ feedback. Transcripts were then sectioned into domains and subsequently abstracted into core ideas. The auditors reviewed the sectioning of data into domains; one auditor reviewed the abstraction of core ideas. Coders attended weekly consensus meetings to create and refine categories and domains across all participants. One auditor reviewed the cross analysis. For this study the domains of Initial Motivation, Selecting Measures, and Maintaining Use were analyzed.

To characterize what motivates clinicians to start, as well as the process of selecting and maintaining PM measures, frequencies of each category within the three domains (i.e., Initial Motivation, Selecting Measures, and Maintaining Use) were calculated. Based on the recommendations of Hill (2012) , categories were classified as general (all or all but one participant), typical (more than half of participants―general), and variant (3 or more participants―typical). Rare categories (≤ 3) were not explored further. Categories with fewer than six participants were excluded because our research questions required the examination of overlap among categories so that very small categories did not contain sufficient meaningful data.

To address the final question of how initial motivation relates to the selection and maintenance of PM, Selecting Measures and Maintaining Use categories were analyzed in reference to the Initial Motivation category. For example, nineteen of the twenty-five participants endorsed Fit as one of their important Initial Motivations. Among those 19 participants, typical and variant categories for Selecting Measures and Maintaining Use categories were then identified to understand what happens when Fit is an important reason for deciding to implement PM. The frequencies to denote general, typical, variant, and rare were calculated in reference to the n of the motivation category being explored (e.g., in reference to the 19 participants who nominated Fit as an initial motivation).

3. Results

Categories are presented within each domain explored (see Table 3). Connections between domains―specifi- cally, Initial Motivation factors as they relate to Selecting Measures and Maintaining Use―are presented in Figure 1 and Figure 2. Initial Motivation categories included the following: fit with therapist (n = 19), effectiveness and therapist development (n = 13), enhancing the role of the client (n = 11), gaining objectivity or additional data (n = 8), and accountability (n = 7). Within each Initial Motivation category, the selection criteria

and maintenance factors that were reported with general and typical frequency are described; only relevant or especially interesting categories with variant frequencies are presented.

3.1. Fit with Therapist Approach, Theory, and Values

Participants typically indicated that they were initially motivated to use a PM measure because it fit with their theory, approach, or values (n = 19). A scientific orientation, solution-focused approach, and the common factors perspective were commonly endorsed orientations that participants felt aligned well with the use of a PM measure. One participant described their initial motivation as coherence between PM and their solution-focused and client centered orientation.

I was doing a Master’s in solution-focused brief therapy and one of my colleagues in the course introduced the measure to the class [...] I felt that [it] was very coherent with my style of work. Trying to really understand what was going to be helpful to the client, the client’s point of view, so that’s how it all started.

3.1.1. Selecting Measures

Among those who cited Fit as an initial motivation, convenience factors were typically considered in selection (n = 11). Brevity, practicality, affordability, interpretability and comprehensibility of the measures were all named; brevity was discussed by all but two of these participants. Psychometrics (n = 7) and usefulness (n = 7) were also considered when selecting a measure, although with less frequency (variant). When considering psychometrics, validity, reliability, and correlations with other reputable measures were often discussed. Regarding usefulness; engaging the client, increasing efficiency, and focusing the treatment were commonly the focus of attention. One participant discussed their consideration of a measure’s brevity and usefulness in the context of their orientation.

The OQ-45 is 45 items long and even in the best implementations, it is given once or twice, maybe three times in a course of treatment. That doesn’t provide for the … type of feedback that I think is helpful in guiding patients or patients guiding the treatment in terms of what progress is being made. So a 45-item implement survey is certainly not helpful for us…

3.1.2. Maintaining Use

Among those who discussed Fit as an initial motivation, engaging clients was typically discussed with regard to maintaining use (n = 13). Client engagement encompassed ideas such as discussion in session, collaboration between the therapist and client, feedback from client, client reflection, and client-directed therapy (Duncan, Solovey, & Rusk, 1992) . This participant discussed the importance of client empowerment in implementing these measures long term.

…what we are really doing is creating an environment in which consumers or clients become the subject of their treatment instead of the object of their treatment. So this helps us [practice the techniques that research has shown to contribute] a substantial amount more to the various treatment outcomes, than does the model that we were using. And by in large our data shows it. We’re seeing patients on a more regular basis, we're having less no shows, we’re having termination sessions. Before we did this, I don't think we had more than 2% of our clients come in for a termination session […], but now, we actually have people plan discharges and it actually contributes to planning the course of their treatment […]I think that is a very healthy and positive thing.

Improved effectiveness (n = 10), guiding treatment (n = 10), and convenience (n = 10) were also typically discussed as factorsthat contributed to maintaining use. One participant was motivated by an awareness that clients tend to be more kind than honest and that having clients fill out a measure revealed more, shaping their work and making it more effective.

Well, it has saved my butt several times!… [I] have gotten some really good feedback about stuff that I had no idea was going on … so it has really helped me to shape my professional approach and the things that I pay attention to and it is just kind of the way I do therapy with clients.

Participants also typically discussed continuing to use measures because they helped to guide treatment (i.e., guide session, guide treatment, and/or guide treatment length). The tools allowed clients to guide the session; “and the biggest thing that really goes into changing a culture like that is allowing that process of empowering the voice and choice of clients in their own treatment process.” This factor was especially relevant to participants endorsing a client-directed model (Duncan, Solovey, & Rusk, 1992) .

Finally, participants also typically discussed continuing to use measures because they were convenient (i.e., fast, easy, inexpensive, and unobtrusive).

3.2. Effectiveness and Therapist Development

The second reason that typically motivated therapists to use PM measures was the benefits related to Effectiveness and therapist development (n = 13). These benefits included assessing effectiveness, improving effectiveness, assessing and improving outcomes, developing skills, developing trainee skills and developing confidence in skills. One participant highlighted his desire to remain aware of his limitations and compensate by using PM measures.

[I] really became convinced that it was very important for me to stay aware of those outcomes, to stay aware of my own limited ability to assess those outcomes in a comprehensive and valid manner, and to compensate for my own clinical deficiencies in judgment by using psychometrics. And that was the beginning of a very long journey.

3.2.1. Selecting Measures

Among those participants who were initially motivated by benefits related to Effectiveness and therapist development, convenience (n = 6) and psychometrics (n = 6) were typically involved in their selection process. Convenience factors that were named included brevity, practicality, affordability, and interpretability. Motivated by efficiency, one participant stated, “But that was my first concern, to be efficient in the work that I did. That was always an issue of mine”. This same participant also considered brevity, efficiency, and a low energy investment when selecting a measure.

When considering psychometrics, participants discussed validity, reliability and correlations with other reputable measures. One participant who discussed calculating effect sizes as an initial motivation considered psychometrics when selecting a measure.

3.2.2. Maintaining Use

Participants motivated by Effectiveness and therapist development typically (n = 8) indicated that improving effectiveness also motivated continued use. Thus, initial motivation remained the reason to continue use. Participants also typically discussed continuing to use measures because they helped identify issues in therapy (n = 8). Several types of issues were named including therapist mistakes, client problems, alliance ruptures, and lack of progress. One participant discussed how PM measures were used to identify issues that needed to be resolved, informing the participants of changes that needed to be made or work that needed to be done (i.e., therapist development).

…sometimes there have been cases that, if it weren’t for the measures, I would’ve thought were doing poorly, but they aren’t. And [there are] also cases where um I thought everything was going fine and [it] wasn’t […] It alerted me [to the fact that] I needed to change my game… to what cases do I bring to supervision and which ones do I just leave alone.

3.3. Enhancing the Role of the Client

Less frequently (variant), participants discussed being initially motivated to use a PM measure to enhance the role of the client in therapy (n = 11). Participants explained that they were motivated to use a tool that would foster collaboration in treatment, facilitate client-centered therapy, and help gain the client perspective.

I think [what] helps people to change [is] giving them a voice in their own care, respecting their input and not just respecting the input but adjusting services based on that input… [the measures] are just laid right on the table and discussed in very transparent ways and that’s how they were designed to be used. And I know that that’s what the co-developers intended for those measures.

3.3.1. Selecting Measures

Among those citing Enhancing the role of the client as an initial motivation, usefulness was the most commonly considered factor when selecting measures (n = 7). Participants typically highlighted the usefulness relative to program evaluation, session-by-session feedback, prevention of drop-out, client engagement, and gaining client perspective. Utility for client engagement and gaining client perspective were also factors that were initial motivators in this group. “The other thing that fascinated me in reviewing [the PCOMS] was that it appeared to have an ability to engage folks more, […] In fact, we changed our whole intake process, we even renamed it to the initial engagement session because engagement was still one of the most critical factors in the type of work that we do.”

Convenience was also typically considered when selecting measures (n = 6). Specifically, participants discussed considering brevity, affordability, practicality, and scoring procedures. One participant specifically discussed considering a measure because it struck a balance between satisfying their initial motivation to enhance the role of the client and convenience.

We could first allow our clients to continue to use whatever treatment they deemed appropriate and necessary […], and secondly not take time away from the session in any great degree, something that could be filled out quickly-this seemed to be something that was worthwhile.

3.3.2. Maintaining Use

Among those citing Enhancing the role of the client as an initial motivation, improving effectiveness, enhancing the role of the client, and engaging the therapist were commonly discussed as reasons for continuing use. Relative to improved effectiveness, participants typically discussed continuing to use measures because they helped improve both therapist and treatment effectiveness (n = 8). One participant specifically mentioned that they were motivated to continue by the increase in percentage of reliable change outcomes within their organization.

Participants also typically returned to their initial motivation and continued to use a PM measure because it enhanced the role of the client (n = 8). Specifically, the measures were reported to have facilitated discussion, collaboration, client feedback, client reflection, and client-directed therapy (Duncan, Solovey, & Rusk, 1992) . One participant discussed how the measures improved discussion, “[you learn] overtime how to read [them], how they answer those questions. And it provides a rich source to go back and talk to them about it.” How client-feedback improves treatment was also noted, “You get honest responses back from people that allow you to make adjustments. It really goes a long way to increasing the alliance with your clients.”

Participants who were initially motivated by Enhancing the role of the client also typically discussed continuing to use a PM measure because it helped to engage the therapist (n = 7). One participant who used a measure with trainees discussed the benefits of feedback to both the client and therapist.

I think [PM] becomes interesting, certainly for us, in terms of our own our own feedback but it’s also interesting for the clients too […] Are [things] going better, worse, about the same? And that’s the same thing that I do with supervising, is encourage them to graph the results, share them with clients, keep these results as part of their own professional accountability.

3.4. Gaining Objectivity or Additional Data

Less frequently, participants discussed being initially motivated to use PM measures to gain objectivity and/or additional data (n = 8).

I’ve always been well aware that the nature of the work that we do can be very subjective … sometimes you think someone is getting better and they’re not. And sometimes you think they’re not, and they are. I’ve been doing this for long enough that I’ve been humbled enough not to trust my opinion either way.

Some participants were more specific with what they were looking for:

There were a few criteria that we wanted [the PM measures] to have―one of them being, for example, being able to say whether someone falls within a clinical distribution or not, whether they are in the normal range, and the other one having some kind of metric for whether somebody has made a meaningful change, whether clinically significant or statistically significant change.

Maintaining Use

With typical frequency, participants who were initially motivated to gain objectivity or additional data continued using measures because they helped guide treatment (n = 5). One participant discussed how they use the additional data to know when to end treatment.

These measures are much more outcome focused … if the client comes and they’ve gotten what they need from therapy and it’s only taken them 4 sessions then that’s enough. You shouldn’t interpret that as resistance or the person is avoiding big issues in their lives. […] I think you reach a point where the response rate is so flat that adding more sessions doesn’t incrementally give you that much more. I’m much more interested at the point where they start to find out a bit. At that point for me, that is when therapy should start [to] begin winding down.

3.5. Accountability

Participants (n = 7) indicated that accountability was another initial motivation to use PM measures. Accountability was discussed by therapists both externally, in relation to third-parties, and internally, as a desire to be accountable to themselves and their clients.

We tend to blame it on the clients instead of looking at our own process and looking at what we have done or not done. And so I've always wanted to have some measure of accountability, to find out whether or not what we are doing is helpful for whom and what we can do to make our services more efficient and more useful.

Maintaining Use

Among participants who cited accountability as an initial motivation, providing information to third parties (n = 4), increasing effectiveness (n = 4), guiding therapy (n = 4), and convenience (n = 4) were discussed as reasons for maintaining use. Third parties mentioned included a hospital, physicians, the ministry and future clients (i.e., advertising). This factor was not mentioned by all participants initially motivated by accountability, as some participants were only seeking internal accountability.

Typically, participants also discussed increasing both therapist and treatment effectiveness. For this participant, the improved efficacy of treatment is attributed to accountability, as their response to negative feedback is changing techniques.

[If] there’s no progress after three sessions, I talk about it with clients … “I’m concerned that you’ve seen me three times and there’s no appreciable change, or you’re going in the wrong direction, and I wonder what we could be doing differently.” So we have that conversation and then again at six or seven sessions. And I’ve transferred clients or…tried to do something different, dramatically different.

Typically, participants also discussed continuing to use a measure because they helped to guide treatment in terms of directing sessions, the course of treatment, and the length of therapy.

Lastly, participants also typically discussed continuing to use a measure because it was convenient. Among those who were initially motivated by accountability, maintenance of use was facilitated by easy, inexpensive, and unobtrusive measures.

4. Discussion

4.1. Initial Motivation and Selecting Measures

Regardless of initial motivation, participants most often named psychometrics, usefulness, and convenience as reasons for selecting a measure. Psychometrics is also the most commonly nominated factor of influence in current research frameworks for mental health quality measurement (Table 1). Hence, researchers and clinicians agree about the importance of reliable and valid measures. Similarly, at least one of the convenience factors - brevity, practicality, affordability, simplicity, and interpretability ? was discussed in most frameworks (Table 1) and was typically a reason for selection in the clinician-participants. It seems that clinicians align well with researchers in wanting psychometrically strong and convenient measures.

Usefulness is a different story. This feature was only cited by two of the ten frameworks, both of which discussed usefulness in the context of utility across settings (Erbes et al., 2004; Green & Gracely, 1987) . Conversely, the clinician-participants framed usefulness in terms of improving client engagement, program evaluation, session-by-session feedback, prevention of drop-out, and creating focus in therapy. We can understand this as an example of the practice-research gap that is so frequently discussed in psychology (Kazdin, 2008) and other applied disciplines. The reviewed frameworks were developed by groups of researchers, revised from past frameworks, and derived from reviews of the literature. Relative to utility, the frameworks take an administrative or organizational perspective, referencing usefulness across settings. The clinicians understand utility from a clinical perspective; what is useful is what makes my work with clients better. In an effort to close the practice-research gap, frameworks need to give greater voice to clinician perspectives; in particular, utility should be framed in terms of what benefits clients.

Convenience was the most commonly cited selection factor in terms across the examined Initial Motivation categories. Convenience was also cited as a typical reason for maintaining use (Table 3). This finding may be a reflection of our sample, the majority of whom were PCOMS users. PCOMS users may value convenience, as a notable feature of the PCOMS is its brevity. In fact, the PCOMS was developed to address practical concerns regarding the Outcome Questionnaire, namely the length of the measure (Duncan, 2012) . That being noted, practical reasons have been identified as a primary barrier to using outcome measures in general (Hatfield & Ogles, 2007) , so it is not surprising that they may also figure prominently in selecting and maintaining use of a any measure, not just the PCOMS. A number of measures already have online interfaces that are easy for clients to access and use; reports can be delivered to clinicians quickly and conveniently. If we are serious about increasing the use of PM in practice, developers and distributors of measures need to prioritize communications about convenience to clinicians. Convenience needs to figure prominently in training and clinicians need opportunities to understand how to integrate rapid-use systems into their clinics.

4.2. Initial Motivation and Maintaining Use

Understanding factors that helped to maintain use of measures provides insight into the framework factors that should be emphasized to facilitate continued use. Clinicians commonly discussed the importance of improving effectiveness as a maintenance factor. Several frameworks suggest considering measures that demonstrate the means by which treatments produce effects or ones that have an acceptable sensitivity to change (Table 1). These factors would certainly contribute to increased efficacy. Emphasizing these factors or more explicitly stating efficacy as a selection factor may help to increase PM adherence.

The remaining factors that influenced continued use for these participants were not specifically mentioned in the frameworks; these include engaging clients, enhancing the role of the client, helping to identify issues in therapy, helping to guide treatment, providing information to a third party, and helping to engage therapist. The lack of acknowledgement of these factors may be because the frameworks were developed in reference to all mental health quality indicators, whereas clinicians were referring to PM measures specifically. A framework for selecting PM measures specifically should consider these important clinical parameters.

A common point of discussion within several of the maintaining use factors was the endorsement of client-directed therapy (i.e., engaging clients, enhancing the role of the client and helping to guide treatment). Client-directed therapy is a term coined by a PCOMS developer (Duncan, Solovey, & Rusk, 1992) ; it refers to therapy that uses ongoing client perceptions about the fit and progress of therapy to direct options and provide the ultimate litmus test for success. Although client-directed therapy does not preclude use of PM measures other than the PCOMS, participants who discussed their endorsement of this practice were PCOMS users. Hence, the facilitation of client-directed therapy in maintaining use may be specific to PCOMS users.

Effectiveness and therapist development, enhancing the role of the client, and accountability were all factors that initially motivated participants to consider PM and also made them want continue to using it. Effectively, the participants found that the measures delivered on what they perceived to be their promise in these areas. This is consistent with a study examining practice change in primary care settings that discussed a spiral of motivation and change: motivation was necessary for change, and practice feedback and an appreciation for the feasibility of change were necessary for motivation (Ruhe et al., 2005) . The fulfilled expectations experienced by our sample of participants acted as positive feedback and interpreted feasibility; these factors motivated continued use. As the field moves to a broader dissemination of measures, it will be important to investigate clinician perceptions of how the measures deliver on their promises. In particular, we need to study how measures contribute to therapist development, to client engagement, and to practitioner accountability to clients and to third-parties.

Improved effectiveness was the factor that most commonly impacted maintenance of use; it was noted with typical frequency under all but one Initial Motivation category (Gaining objectivity and/or additional data). This finding is indicative of the high priority these clinicians placed on efficacy. Kitson et al.’s (1998) model of successful implementation of evidence-based practice suggests that successful implementation is a function of evidence, context, and facilitation. Evidence, more specifically, comprises evidence in the form of research findings, clinical experience, and patient preferences. For our participants, experiencing efficacy in clinical practice was evidence in the form of clinical experience. According to Kitson et al.’s (1998) model, these types of experiences facilitate maintained use of evidence-based practice (i.e., usage of PM measures). Regardless of initial motivation, clinical experience of PM efficacy tended to facilitate continued use. Considering the importance of this factor and convenience, PM measures need to balance efficacy and convenience.

5. Limitations

One important limitation of this study is the large proportions of the participants who were male, over the age of 51, and using the PCOMS.In particular, because the PCOMS is notably brief, the significance of convenience may have been overstated.However, it is important to consider that the PCOMS is the most commonly used PM measure among Canadian psychologists (Ionita & Fitzpatrick, 2014) ; therefore, this sample may be somewhat representative of Canadian PM users. Regardless, the skewed sample is recognized as a potential limitation of this study; older, male, PCOMS users may differ from other PM users in their motivation, selection, and maintenance behaviors. Additionally, both the Selecting Measures and Maintaining Use domains were analyzed with respect to the Initial Motivation domain. The analysis was based on the assumption that initial motivation impacts selection and maintenance behavior. It is also possible that other, unexamined variables may be responsible for the patterns demonstrated in the data. Finally, although the researchers attempted to privilege the voice of the participants and to use multiple perspectives in the CQR analysis, researcher bias regarding the importance of PM may have influenced the findings.

6. Summary and Implications

This study compared researchers’ frameworks for selecting measures to clinicians’ actual experience of selecting and continuing to use PM measures. The results suggest that for frameworks to be more applied to PM use, they need to integrate clinician perspectives; in particular they should consider paying more attention to clinical utility factors. To improve PM adherence, frameworks should include guiding treatment, identifying issues, enhancing the role of the client, engaging the therapist, and providing information to third parties. Frameworks created in consultation with clinicians can decrease the burden of PM measure selection and increase the proportion of clinicians using and continuing to use PM measures. Of importance in these findings are the roles of convenience and effectiveness in maintaining use regardless of initial motivation. The voices of participants clearly point to the importance of striking the right balance; it is not either effectiveness or convenience, but both.

Acknowledgements

The work was supported by the Canadian Institute of Health Research under the Frederick Banting and Charles Best Canada Graduate Scholarship.

Cite this paper

Megan Knoll,Gabriela Ionita,Jann Tomaro,Vivian Chen,Marilyn Fitzpatrick, (2016) Progress Monitoring Measures: The Interaction of Clinician Initial Motivation with Selection and Maintenance Issues. Psychology,07,444-458. doi: 10.4236/psych.2016.73046

References