Association between Implementation and Effect in the Randomized Controlled Trial “Move for Well-Being in School"

Background: Research in school-based physical activity interventions has primarily focused on effectiveness. The main aim of this paper is to assess the association between the degree of implementation of a multicomponent physical activity intervention and effectiveness on students’ physical self-worth (PSW). Methods: The Move for Well-being in School study used a cluster randomized controlled study design including 24 schools with a total of 3123 students 4 th to 6 th grade. 1269 students and 148 educators were, for a whole school-year, part of the activities at 12 intervention schools, The intervention program targeted settings for school-based physical activity in recess, in-class and Physical Education (PE). The association between change in PSW and implementation were analysed using multilevel linear regressions models es-timated in R. Results: Three components were to be implemented at all intervention schools. For PE, half of the performed lessons were from the program, which was in line with the protocol aim. Classes with a higher implementation rate of the PE-component had a significant higher PSW at baseline. An average of 5.8 brain breaks was implemented per class per week, and no significant differences were established between implementation and change in PSW. Recess implementation was the linking implementation and effects in multicomponent school-based activity interventions. This area of research needs further development to increase the understanding of how physical activity-related interventions are implemented and how this affects final outcomes. Trial Registration: Date of registration: 24 April 2015 retrospectively registered at Current Controlled Trials with study ID ISRCTN12496336.


Background
Physical activity (PA) has the potential to strengthen a wide range of positive dimensions related to both physical and psychological well-being among children and young people (Babic et al., 2014;Biddle, 2015;Landry & Driscoll, 2012). In addition to the long list of health benefits of PA, positive experiences with PA affect perceived physical competence -an important aspect of overall mental health, not least for children and youth (Fox, 2000;Lubans et al., 2016). Physical capability has been assessed using the theoretical constructs of physical self-perceptions e.g. the Children's Physical Self-Perception Profile (C-PSPP). In this paper, "physical self-worth" serves as the primary outcome measure.
However, when evidence-informed programs are broadly implemented into school settings, the effectiveness is often disappointing (Durlak & DuPre, 2008;Van den Berg et al., 2017). To understand the discrepancy between evidence-based PA programmes and their effectiveness in real-world settings, authors often refer to unobserved implementation issues that lead to the observed program failure and lack of fidelity (Schaap, Bessems, Otten, Kremers, & Van Nassau, 2018). Several studies have addressed the challenges with implementing PA programmes in schools, identifying numerous barriers such as lack of time among, for instance, educators, lack of supportive school climate for PA and lack of available resources (Durlak & DuPre, 2008;Naylor et al., 2015).
Authors have found more than 60 variables that might influence implementation in real world settings, and employed more than 20 measures to capture several different components of implementation (Durlak, 2015). The golden standard of measuring implementation is yet to be fully agreed upon (Moore et al., 2015). At the same time, it is generally agreed upon that implementation research could be significantly improved by a more systematic approach to the use  (Eccles, Grimshaw, Walker, Johnston, & Pitts, 2005).
Reviews covering implementation issues in relation to school-based physical activity interventions, point out that current research to a large degree has focused on measuring effectiveness. Whereas factors that are related to the implementation of programmes or the effects of the implementation on primary and/or secondary effect measures, have been researched only sporadically (Domitrovich & Greenberg, 2000;McGoey, Root, Bruner, & Law, 2015. While there is a growing recognition of the need to conduct comprehensive process evaluations, and a growing number of studies focusing on exactly process aspects, the bulk of studies focus on programme effectiveness (Gray, Contento, & Koch, 2015;Naylor et al., 2015;Schaap et al., 2018;Van Sluijs et al., 2007).
The studies that address the correlation between implementation and primary outcome are, with few exceptions, efficacy studies (Naylor et al., 2015;Schaap et al., 2018). The general lack of such studies to successfully translate into real-world settings could, as described by Domitrovich and colleagues, be related to the fidelity and quality by which the intervention is implemented-which again is related to the possible effectiveness of the intervention (Domitrovich et al., 2008).
Reviews on school-based PA interventions suggest a rather consistent correlation between implementation and the effectiveness on primary intervention outcome (Durlak, 2015;Naylor et al., 2015;Quested, Ntoumanis, Thøgersen-Ntoumani, Hagger, & Hancox, 2017). In short, school-based PA interventions should not just be informed by the best available evidence regarding efficacy and/or effectiveness, they also need to be well-implemented to truly make a significant impact (Durlak, 2015).
In order to construct evidence applicable to the school setting it is important to evaluate the feasibility of the intervention as well as fidelity. Measuring implementation fidelity in relation to complex interventions is challenging (Craig et al., 2008;Moore et al., 2015). The intricate work to link implementation and effectiveness, especially in complex settings with multicomponent interventions, may explain the lack of research within the area (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004;Naylor et al., 2015).
Therefore, the aim of this paper is to assess the association between the degree of implementation of a multicomponent PA intervention and effectiveness on students' physical self-worth in the RCT intervention Move for Well-being in School. Furthermore, we want to contribute to the limited research regarding implementation scores and discuss how these can be associated with intervention effects.

Study Design, Setting and Participants
This study looks at the association between implementation-measured through Move for Well-being in School (MWS) study used a cluster randomized controlled study design applying the RE-AIM framework as an overall framework to guide implementation and evaluation (Smedegaard, Christiansen, Lund-Cramer, Bredahl, & Skovgaard, 2016). The framework elements were furthermore guided by the work of Nayler et al. and Durlak and Dupree (Durlak & DuPre, 2008;Naylor et al., 2015). In 2015, 11 municipalities in the Central Denmark Region and Region of Southern Denmark were selected based on maximizing geographic spread, the difference in size and allocated budget for public schools and invited to participate in the study (Smedegaard et al., 2016). Seven municipalities consented and either contacted schools or allowed the research team to invite schools. Out of 126 possible schools in the 7 municipalities, 24 schools agreed to participate and were randomized taking into account school size, district socio-economic status and an even distribution of schools from each municipality in the intervention group and the control group. In the current study, the analyses are based on the 12 intervention schools that participated in the 1-school year interventions period. The enrolment procedure and the study design has been described in detail elsewhere (Smedegaard et al., 2016) as well as previously papers has addressed results from the MWS project Christiansen, Clausen, Smedegaard, & Skovgaard, 2019;Holt, Smedegaard, Pawlowski, Skovgaard, & Christiansen, 2018;Smedegaard, Brondeel, Christiansen, & Skovgaard, 2017

Intervention
To ensure high quality and ensure a socially inclusive PA environment, the approach to the physical activity intervention in the MWS PA program was grounded in Self-Determination theory and designed to target three innate psychological needs: competence, autonomy and relatedness in order to improve motivation for physical activity for all students (Ryan & Deci, 2000).
The intervention program consisted of initiatives targeting three settings for school-based physical activity: 1) Physical Education (PE) Classes; 2) In-class activities; 3) Recess activities. During PE classes, a minimum of 6 out of 8 specially designed PE courses each lasting 4 × 90 min were to be conducted throughout the school year. Central features across all PE courses included teamwork and individual competence development rather than competition. In-class activities consisted of minimum two brain breaks per day lasting at least 5 minutes, i.e.
short physical activities briefly interrupting the regular courses. The in-class ac- to be used to initiate a variety of activities supported by teachers and teacher assistants during recess minimum three times 30 min/week. Furthermore, the schools were encouraged to introduce activity-enhancing initiatives such as opening up in-door areas for activities during rainy days, make rules for limiting use of electronic devices during recess etc. Lastly, three theme days, spread out during the school year, were to be conducted. The aim was to engage students in the development of activities and to initiate joint discussions on physical activity and general well-being in school (Smedegaard et al., 2016).
The deliverers of the intervention, teachers and teacher assistants teaching 4 th to 6 th grade, were equipped with a Tailored Activity Program, including educational materials, planning guides and course plans for incorporating PA throughout the school day. The Activity program was supported by a Competency Development Program consisting of four full-day workshops focusing on the underlying theoretical approach and opportunities to try out core activities in practice.
Participation and teacher preparedness to deliver the intervention as a result of the Competence Development Program have been address elsewhere (Smedegaard et al., 2017). At each school, a local appointed Coordination Group guided and motivated the implementation of the intervention program receiving frequent support from the research team via information e-mails and local supervision visits throughout the intervention period (Smedegaard et al., 2016). The outcome variable, physical self-worth was measured using the Children's Physical Self-Perception Profile (Fox & Corbin, 1989;Whitehead, 1995). PSW was calculated as the mean of six items with four categories (1 indicating the lowest level and 4 the highest).

Student Data
To be included in the analyses students had to provide data on PSW for at either baseline or follow-up. This resulted in a final dataset with 2331 observations (baseline or follow up) from 1269 students in 57 classes at 12 schools. Educators were asked to complete an online questionnaire two months (T1), five months (T2) and nine months (T3) into the program period (Smedegaard et al., 2017). At T1, 141 educators answered of whom 42 were PE teachers, at T2, 135 educators answered of whom 48 were PE teachers and at T3, 139 educators answered of whom 39 were PE teachers.

Educator Data
The educator survey was used to evaluate the implementation of the intervention components at class level: in-class activities, PE-courses and recess activities. The aggregation and matching of data from educator level to student level is presented in Table 1.
The brain breaks were reported for each class in the educator questionnaire.
In case there was more than one educator that reported on the same class, the sum was calculated. The brain break variable include missing values for the students in one class, because no one reported brain breaks here (n = 24 students).
The implementation of intervention specific PE-courses was assessed using one question concerning the proportion of taught lessons on a 5-point Likert-scale from none to all. The answers were grade specific and if several PE-teachers taught within the same grade the average was calculated. The PE variable includes missing values for eight classes (n = 171 students).
The implementation level of recess activities was assessed by the average of two items capturing number of days (0 -5 days) with planning in individual classes before recess and with initiatives during recess for more classes in the schoolyard. Regarding the planning question, we identified in which classes the educators had taught from their response on brain breaks. If an educator were attached to more than one class, number of educator planning days was divided to the number of classes. Finally, if more educators were identified planning for the same class they were summed. Regarding initiatives during recess, it was assessed

Results
Descriptive baseline characteristics are presented in Table 2. Students provided information on gender, primary language spoken at home, and family social class in three categories (high, medium, low) (Smedegaard et al., 2016). There was an equal distribution between boys and girls with a small overrepresentation of fourth graders compared to 5 th and 6 th graders. With respect to social class, the study sample is comparable with the Danish population as a whole (Smedegaard et al., 2017). Table 3 presents the implementations scores for Brain Breaks, Physical Education and Recess, respectively. According to the implementation protocol, 10 brain breaks per week per class were the target. The results show an average of 5.9 brain breaks being delivered. There was, however, large variation from one to 19 brain breaks. For PE, an implementation score at 3.4 equals approximately half of the total lessons, which matches the implementation protocol. Finally, a score on 0.8 day a week where educators initiate or help plan recess activities is  The results show that there are no significant association between PSW and brain break implementation at neither baseline nor follow-up. Figure 1 indicates that classes receiving the largest amount of BB´s are made up of students with an insignificant lower PSW at baseline. This difference increased insignificantly at follow-up.
For PE there were a significant association at baseline between PSW and implementation. At follow-up, this difference between classes were slightly diminished ( Figure 2). The classes receiving most MWS lessons are those with the highest average PSW at both baseline and follow-up.
Finally, results show no significant association between PSW and recess

Discussion
The aim of this study was to assess the association between implementation of the Move for Well-being in School intervention and effect on students' physical self-worth. Implementation was assessed for the three primary intervention components at class level. The three components were implemented at all schools to a variating degree. PE and brain breaks were implemented closest to the protocol, while the aimed at implementation level for recess was low. Such a pattern is not uncommon. In some studies, the range has been reported as high as 87% when comparing the lowest and highest implementation levels within the same studies, and 20% to 40% differences between providers e.g. educators or sites e.g. schools are not uncommon (Durlak & DuPre, 2008).
The regression analyses showed a significant association between PSW and implementation for PE at baseline and for recess at follow-up. The significant association between the degree of PE implementation and PSW at baseline could, in our opinion, have several causes. Firstly, implementation might have been easier in classes with more physical confident students. Some of the PE lesson plans, provided to the intervention schools, put higher demands on both teachers and students, which could have been a challenge in classes with overall lower physical confident students. Secondly, the difference in PSW at baseline may also be due to previously different PE practises, where teachers, more prone to implement, also had more focus on promoting PSW for all students in the classes before the intervention. which have promoted PSW. The effort may have engaged students with lower motivation and self-worth to participate in different physical activities during recess. The recess component was the most difficult for schools to implement. At the same time it was very popular among students . While brain breaks and PE are mandatory and educator controlled, recess is largely students' free time. In Denmark, educators are mainly passive observers during recess, and active engagement is rarely seen in 4 th to 6 th grade. This freedom is enjoyed by many students, which appreciate recess and the opportunities for

Implementation Scores in Multicomponent Interventions
We chose to assess the implementation for each of the primary components adjusted for the other two components. The first reason for doing so was that we found a low correlation between implementation in the three areas. Secondly, if all three components were compiled in one total implementation score we were not able to assess if the components had different associations with PSW.
Another choice in this study was to keep the number of variables low in the implementation score. We limited ourselves to questions regarding dose delivered assessed via educator questionnaires. Other approaches in the implementation literature have included data from parents, students, school management   (2008). They found a weak correlation between the implementation scores and students well-being (Askell-Williams et al., 2013). The LEAP study classified schools as having either 'high' or 'low' implementation levels related to the participation of girls in vigorous PA activity at school (Saunders et al., 2006). The differentiation was based on a mixed-methods approach and was significantly associated with the girls' participation in vigorous PA activity at school (Saunders et al., 2006).
Summing up, the literature on implementation scores to be associated with intervention effects offers different strengths and weaknesses. A score including many variables have the strength to make it possible to consider many aspects, but it can afterwards be a challenge to identify particular important implementation factors. A score using mixed methods can to a higher degree assess quality of implementation but faces the risk of different assessment criteria and interrater differences. Finally, the approach in the current study offers a specific assessment of dose delivered in the respective components, but do not take quality or other aspects into account.

Validity of the Implementation Score
In the current study, a number of methodological issues related to the validity of the implementation score were experienced. The main challenge in creating the class based score, was the matching of the educators actions to the individual classes. Implementation of brain break was assessed at individual class level and PE variables at the grade level. The recess assessment was not reported class or grade specific and we therefore used the brain breaks variable to match the questions regarding planning recess activities. This is an assumption, which can be discussed. The second recess question regarding initiatives during recess, we averaged all educators' activities per class. This could potential underestimate the true number of days with educator involvement during recess, because these activities can involve several classes and the protocol did not state the number of active teachers during recess. If three educators initiated activities during recess at three different days at one school with six classes, the average per class would be 0.5, while the true number of days would be three. So even though we used a simple implementation score based on primarily dose delivered, it was not straight forward and several assumptions can be discussed. Especially, regarding the alignment of dose delivered and dose received at class level.
In our approach, we chose separate implementation scores for each component in the same regression model. This allowed us to assess the effect of each component on PSW and at the same time adjusting for the implementation of the other two components. A potential problem with this approach is that we do not take potential synergetic effects between components into consideration e.g.
if a higher rate of implementation in more components is more effective than the single components alone. This aspect of multiple components design in school interventions and the issues around assessing proposed synergistically ef-  (Lloyd et al., 2017).
Implementation of school based PA interventions is complex in nature even though the intervention itself may seem simple. Documenting the implementation process and evaluating the association with primary outcomes is crucial to improve school-based interventions. In this study, we used a quantitative approach, computing a numeric score based on educator surveys. However, we recognize that this is a rather simple way of evaluating a complex intervention.
In a previously published paper, we qualitatively evaluated the intervention process . Furthermore, in a separate published paper, we have discussed a range of implementation issues related to the MWS project (Smedegaard et al., 2017). These prior reporting's gave valuable insight into the project black box and what happened at the schools. More studies on implementation process and impacts will contribute to a better understanding of possible links between implementation and primary outcome variables (Gray et al., 2015).

Conclusion
The aim of this paper was to assess the degree of implementation of a multicomponent physical activity intervention in order to explore the association between implementation and effect on PSW. Three components were implemented at all schools to a varying degree. For PE, half of the performed lessons were from the program, which was in line with the protocol aim. Classes with a higher PE implementation had a significantly higher PSW at baseline. An average of 5.8 brain breaks was implemented per class per week, and no significant differences were established between implementation and change in PSW. Recess implementation was the hardest for most schools to implement, but schools with a high level of recess implementation had a significant increase in PSW at follow-up.
This indicates that if schools and teacher prioritize recess, help students plan, and initiate inclusive activities it may be possible to significantly increase PSW.

Funding
The study was funded by a donation from the non-profit foundation TrygFonden, Denmark. The funding body has no role or authority in the conduct of the research project, except acceptance of adjustments to the research plan.

Availability of Data and Materials
The datasets collected and/or analysed during the current study are available from the corresponding authors upon reasonable request.

Authors' Contributions
SS initiated this paper and wrote the draft. All of the authors contributed to writing the manuscript and critically reviewing its content. All authors approved the final version before submission.

Ethics Approval and Consent to Participate
Prior to the data collection, students and their parents received information about the nature and procedure of the study. Consent took the form of written passive consent from the parents and both written and oral passive consent from the students. It was emphasized that participation was voluntary, that respondents could withdraw consent at any time, and that data would be treated confidentially and anonymously. These points were also emphasized at the beginning of each questionnaire session. All data are stored and treated in accordance with Danish law for data protection, and the project is notified and approved by the and is on the ISRCTN registry (DOI https://www.isrctn.com/ISRCTN12496336).

Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.