Assessment of a Downloadable Application with Avatar Guidance for PT-Prescribed Home Exercise after Total Knee Arthroplasty: A 30-Day Usability and Feasibility Study ()
1. Introduction
Total knee arthroplasty (TKA) rates have grown exponentially in the United States in the last 15 years, with 1.7 million primary TKA procedures projected for 2030 and 3.5 million projected annually by 2040 [1]. Estimated rehabilitation costs per primary and revision TKA range from $5000 to $11,000 (in 2013 $) [2]. Given the rapid rise in TKA and attendant rehabilitation costs after hospital discharge, there is a critical need to develop rehabilitation capacity and resources to optimize movement and quality of life for this growing population. Healthcare shortages in rural and underserved areas [3], the lack of transportation services [4], the intermittent inability to attend therapy in person (e.g., quarantine due to COVID-19) [5], and increasing pressure to contain health care utilization costs after TKA [6] make the need for access amid limited resources even more acute.
Computers, interactive applications, and tele-rehabilitation represent technologies that can be leveraged to increase rehabilitation capacity and drive improved outcomes [7]. The Health-in-Motion® application developed by Blue Marble Health Co. (Altadena, California) is a downloadable computing platform that offers preoperative education, personalized avatar-guided home exercises, and remote exercise adherence monitoring capabilities for rehabilitation of the TKA population (Figure 1). In addition to home exercises, Health-in-Motion® captures patient-reported outcomes (PROs), functional outcomes, adherence data, and patient satisfaction measures. Digital health technologies have grown exponentially in recent years. Despite this growth, few studies have investigated applications specifically designed for rehabilitation after TKA [8] [9], and even fewer have assessed user experience by measuring ease of use, learnability, and efficiency of the technology [10]. Moreover, most studies have used technology primarily for communication and reminder systems rather than tools for exercise, or they were limited by requiring additional, specially designed hardware [8] [9] [10].
The purpose of this exploratory study was to determine the usability and feasibility of the Health-in-Motion® application (app) for guiding home exercise rehabilitation after TKA, and to explore changes in outcomes relative to app use. Our hypotheses were that the Health-in-Motion® app would meet criteria for usability and feasibility, and that app-guided home exercise (APP) would lead to similar outcomes compared to the standard-of-care, paper-guided home exercise (CONTROL) at 30 days after TKA.
2. Methods
Study Design, Participants, and Setting
This was a randomized controlled usability and feasibility app assessment that used a two-group parallel design. Participants were consecutively recruited through an existing network of orthopedic surgeons at three clinics in the (city deleted for blinding) metro area from July 2019 to September 2020 and one clinic in the (city deleted for blinding) metro area from November 2019 to December 2020. Participants were eligible if they were awaiting a primary, unilateral TKA for end-stage knee osteoarthritis (OA), aged 50 to 85 years, medically stable, with planned discharge to a home setting, and with Wi-Fi access in their home. Participants were excluded if they had limited weight bearing for conditions other than knee OA, an acute neurological or cardiovascular condition such as stroke or acute heart surgery, or amputation of the contralateral lower extremity. They were also excluded if they were undergoing current and active cancer treatment, had a neurologic disorder that precluded following a typical TKA rehabilitation protocol or precluded having the upper extremity and hand function needed to interact with their computing device. Informed consent was obtained from all participants. Assessments were performed either at two university-based rehabilitation laboratories at (institutions removed for blinding) or in participants’ homes. Public health precautions related to COVID-19 forced the research team to oversee app-based evaluations remotely for the final eight enrolled participants at the University of Colorado Anschutz Medical Campus.
The study was approved by the Colorado Multiple Institutional Review Board (COMIRB).
Procedures
Following consent, eligible participants were randomized using a 1:1 allocation ratio to APP or CONTROL group by a computer-generated random allocation sequence implemented by a blinded, impartial member of the research team not involved in clinical assessments. The app was downloaded to the participant’s tablet, smart phone, laptop or desktop computer, or a loaner tablet was provided for use in the study when a participant did not own a device, preferred not to use their personal device, or when their device’s older operating system was incompatible with the app.
Intervention
The APP group used the app, which was optimized for desktop, laptop, tablet, or mobile devices and featured an animated avatar for verbal and visual instructions to guide physical therapist-prescribed home exercises for 30 days after surgery. Only the APP group used the app to perform daily on-screen, avatar-guided home exercise in a gamified environment with visual pacing cues and timely messaging for safety, breathing instructions, and motivation. The avatar guided exercises for the APP group participants. The app displayed an on-screen visual representation of each prescribed exercise (Figure 1), including speed of movement, start and stop positions, the number of repetitions, sets, and exercise sessions each day, and summary reports of exercise performance. Reminders displayed intermittently on the app screen ensured safe performance of each
![]()
(a) ![]()
(b)
Figure 1. Examples of the Health-in-Motion mobile application. Used by permission from Blue Marble Health Co.
exercise at home. The app also provided positive, motivational messages to encourage adherence throughout the exercise sessions. By contrast, CONTROL group participants were provided with traditional, standard-of-care handouts with pictures and printed instructions to guide the daily home exercise program. The selection, sequence, and timing of exercise progressions for both groups were at the discretion of the treating physical therapist.
Conversion of Prescribed Paper-based to Digital Home Exercise Programs
Following surgery, each participant’s treating physical therapist emailed the home exercise program (HEP) to the research team member who used the app’s administrative portal to select the digital representation of the prescribed HEP. The HEPs for APP group participants were updated in the app as often as directed by their treating physical therapist. Prescribed changes were updated by the research team. Treating physical therapists saw participants in both groups on an outpatient basis (at one-to-three-week intervals as appropriate) through the 30-day study period to ensure exercises were performed correctly and to advance the exercise progression as dictated by participant status. CONTROL group participants performed daily exercises based on the paper exercise HEP guidance.
Application-based Outcomes Assessment
Assessment timepoints and outcomes criteria are summarized in Table 1. Both APP and CONTROL participants were assessed by an unblinded assessor using the Health-in-Motion® built-in app-based standardized testing sequence 1 - 14 days prior to TKA in their homes, at the University of Colorado Anschutz Medical Campus and University of California, San Francisco, or by secure, remote videoconferencing after the COVID-19 pandemic restricted in-person assessments. Whether in-person or remotely, the unblinded assessor confirmed functional outcome scores by timing with a stopwatch or counting repetitions as required by each test. For remote testing full visibility of the participant was ensured prior to the beginning of remote testing. Both APP and CONTROL groups performed the personalized, physical therapist-prescribed exercise program for 30 postoperative days. At the conclusion of the study period, outcomes were assessed in a second app-based testing session. Both groups used the app for baseline and 30-day outcomes data collection. The 30-day time point was chosen to focus on the primary outcome of usability and patient experience, allowing the APP group participants to engage with the app daily for 30 days so that measures of usability in this group would be based on substantive daily experience.
App-based testing used to guide functional assessments for both APP and CONTROL at preoperative and 30-day time points included three functional performance measures: Timed Up-and-Go test (TUG) [11] [12], Single Leg Stand Test (SLST) [13] [14], and Thirty-second Sit to Stand test (30STS) [15] [16] [17] [18]. In addition, participants completed four patient-reported outcome measures (PROs): the Knee injury and Osteoarthritis Outcome Score, Joint Replacement (KOOS JR) [19], the Veterans RAND 12-item Health Survey (VR-12) [20], the Modified Activities-specific Balance Confidence Scale (mABC)
![]()
Table 1. App-administered outcome assessment time points and criteria.
KOOS JR: Knee injury and osteoarthritis joint replacement outcome score; PROMIS: Patient-reported outcomes measurement information system—physical health; VR-12: Veterans Rand 12-item Health Survey-Physical; mABC: modified Activities-specific Balance Confidence Scale.
[21] [22], and the Patient-Reported Outcomes Measurement Information System, Physical Function subscale (PROMIS) [23]. These assessments, detailed below, were administered to participants in both APP and CONTROL groups using the app’s built-in electronic data collection capabilities. For timed functional tests study personnel cross-validated app-collected data with standard practice timing with stopwatch or by visually counting repetitions.
System Usability: Usability was measured in the APP group using the System Usability Scale (SUS), a widely used measure of usability for computing technology [24]. The SUS is a 10-item Likert scale questionnaire that provides a user experience index across four domains of user experience: efficiency, learnability, ease of use, and user satisfaction [25]. The sum of the 10 ranked item raw scores (1 = “strongly disagree”; 5 = “strongly agree”) was converted to a 100-point index scale. Drawn from non-parametric data, the SUS result represented an index score. A raw score of 68 is considered “average”, above 68 is “above average”, and above 80.3 is “excellent” [26]. Further empirical work by Bangor et al. [24] [27] identified meaningful cutoffs for adjectives applied to this questionnaire and found a raw SUS index score of 72 was indicative of “good” usability.
Feasibility: Feasibility of downloadable application use was determined by examining the percentage of portable device ownership and study use among potential and actual study enrollees in both APP and CONTROL groups, as well as the percentage of reported technology-related barriers to participation among non-enrollees. The feasibility of user experience was further determined in the APP group-only because this group used the app daily for postoperative exercises. This was done by examining the percentage of participants in this group who completed all pre- and post-operative functional assessments.
Functional Performance Outcomes: Functional performance outcomes included the Thirty-second Sit to Stand (30STS) test, Timed Up-and-Go test (TUG), and Single Leg Stand Test (SLST). The participant independently operated the app to complete all app-guided tests. All testing instructions were provided to the participant through the application.
The 30STS test measured sit-to-stand activity, lower body strength and dynamic balance. This test exhibits good reliability [15], responsiveness [16], and validity [15] [17] [18]. The digital version of this test, validated in this platform [13], begins when the participant taps the “Go” button on their device as they perform full, unassisted sit-to-stand cycles as many times as they can in 30 seconds. Each full sit-to-stand cycle was counted. At 30 seconds a “Stop” signal from the application signified the end of counting. The score was the total number of stands performed in 30 seconds.
The TUG test is a valid [11] and reliable measure of strength, agility, and dynamic balance, is recommended by the Osteoarthritis Research Society International [12]. A preliminary validation study of the Health-in-Motion® testing platform demonstrates high correlation with standard testing [28]. When measured early after TKA, the TUG predicts long-term functional performance [29]. TUG measures the time to stand from a chair, walk 3 m, turn around and walk back to the same chair, and sit down without physical assistance. Using the Health-in-Motion® app, the participants positioned themselves within reach of their digital device next to a 17" high sitting surface, tap or click the on-screen “Go” button to start the timer. On returning to the chair and sitting down they tapped/clicked the on-screen “Stop” button. The score of the test was measured in seconds. The smallest amount of change considered above the threshold of error at the 90% confidence interval (minimal detectable change; MDC90) for the TUG test is 2.49s in the first six weeks after TKA [30]. The participant was given one practice attempt followed by two test attempts. The shortest time was used for analysis.
The SLST measures the ability to stand and maintain balance on one leg without assistance [14]. It is a predictor of injurious falls and has been validated with this downloadable health platform [13]. For this app-based assessment the participant stood near their device and tapped/clicked the on-screen “Go” button as they lifted one foot off the floor without touching any surface to assist balance. The test was stopped before 30 s by tapping/clicking the on-screen “Stop” button if the lifted foot touched down or a hand touched a support surface. Standing time was recorded up to a maximum of 30 s, and the maximum time in seconds was used as the score of this test for each leg separately. The participant was offered up to three attempts to achieve the 30 second maximum score. The highest score was used for analysis.
Patient-Reported Outcomes: Scores of the four PROs administered through the app-based testing sequence (KOOS JR [19], VR-12 Health Survey-Physical [20], mABC [21] [22], and PROMIS Physical Health [23]) were automatically calculated and stored by the Health-in-Motion® testing platform. The KOOS JR provides Pain, Function, and Quality of Life scale scores and a summary knee impact score. The VR-12 Health Survey-Physical domain is a health questionnaire that provides health-related quality of life information pertaining to role limitations due to physical problems. The mABC quantifies fear of falling by assessing a person’s confidence that they will not lose balance while performing 16 activities of daily living. The PROMIS Physical Health form is an efficient short-form assessment tool that captures outcomes related to general well-being in addition to the status of the knee.
Statistical Methods
Primary Usability Endpoint and Criterion: Usability (SUS) questionnaire index scores for APP at postoperative day 30 were categorized according to Bangor et al. [24] [27] as either 1) at least 72 “good usability”, or 2) less than 72. This participant-level dichotomous variable was the primary usability endpoint chosen for this study. The a priori criterion used to judge app usability was that at least 75% of APP participants must rate the app with an index score of at least 72. The percentage of SUS index scores of at least 72 was used to determine whether the 75% usability criterion was met.
Primary Feasibility Endpoints and Criteria: The percentage of potential study candidates as well as study participants who owned a tablet, smart phone, or laptop for downloading the app, as well as the percentage of screened but non-participating individuals self-reporting a lack of technology literacy as a barrier to participation in this study were used to determine whether the 75% criteria were met in summarizing each of these outcomes. Additionally, a 90% criterion for completion of both preoperative and postoperative functional testing was used to evaluate the feasibility of outcomes testing using the app.
Exploratory Outcomes: To determine initial comparability of app versus paper guidance of home exercises at 30 days, an analysis of the between-group mean differences in change of the 30STS, TUG, and SLST as well as the KOOS JR physical subscale, VR-12 Health Survey physical subscale, mABC, and PROMIS physical subscale was conducted. For each variable, a change score between baseline and the 30-day follow-up period was calculated and used as the value for comparison between APP and CONTROL groups using two-sample t-tests. Descriptive statistics compared CONTROL and APP group demographics. Stata/IC Version 14.2 (College Station, TX, USA) was used for all statistical analyses.
Power and Sample Size: Quantitative outcomes for this study were powered as a noninferiority trial contingent on a two-sided alpha = 0.05 for significance and a power level of 0.80. Based on an established SD of 2.47 for the 30STS [31] and a noninferiority margin of 2 sit-to-stands based on estimates of MCID after joint arthroplasty providing an effect size of 0.625, the necessary sample size was determined to be 26 patients per group.
3. Results
For each group, participants were randomly assigned to each mode of exercise delivery as depicted in the study flow diagram (Figure 2). No differences were found in baseline characteristics between CONTROL (N = 31) and APP (N = 26) (Table 2).
![]()
Table 2. Participant baseline characteristics.
TUG: timed up and go test; SLST: single leg stand test; 30STS: 30 second sit-to-stand test; KOOS JR: Knee injury and osteoarthritis joint replacement outcome score; PROMIS: Patient-reported outcomes measurement information system—physical health; VR-12: Veterans RAND 12-item Health Survey-physical; mABC: Modified Activities-specific Balance Confidence Scale.
![]()
Figure 2. Study flow diagram. Reasons for withdrawal included partial rather than total knee arthroplasty, postoperative infection, two-week delay on starting home exercise, apprehension about secure app download from the web, the onset of the COVID-19 pandemic, and home WiFi too slow or sporadic.
Outcomes
Usability and Feasibility: The results of usability and feasibility analyses are shown in Figure 3. With respect to usability, we measured mean SUS scores in APP group participants. Twenty-four individuals in the APP group completed postoperative SUS scores, and 19 rated the usability of the application as “good” or better with a SUS score of at least 72, satisfying our a priori criterion for this study’s primary outcome.
With respect to feasibility by technology-based barriers to participation, 26 individuals were eligible to participate in the study but chose not to (Figure 2). Among those, 12 identified some form of barrier including three reporting a lack of technology literacy (n = 3), insufficient Wi-Fi connection (n = 1), or visual discomfort with screens (n = 1) as barriers related to technology. Nine of 12 individuals reported no technology-related barriers to participation, satisfying this a priori criterion.
Regarding feasibility by personal computing device ownership and use, 84 of 100 individuals assessed for eligibility were available for analysis. Among these individuals personal computing device ownership was 71.4% (Figure 3). The inclusion of loaner tablets in this study contributed to personal device use below the 75% a priori feasibility threshold.
![]()
Figure 3. Usability and feasibility outcomes.
Finally, regarding differences in outcomes between groups, 208 functional assessments were available for analysis in the APP group (N = 26) from two app-guided assessment sessions of four tests each and including outcomes from both operated and non-operated legs SLST. Data from 190 of 208 possible assessments were analyzable; an additional nine observations could not be analyzed because only one leg was tested with the SLST. In spite of this we are confident the participant was present and performing all functional assessments. Therefore, 199 of 208 tests were performed yielding a finding of 95.6% completion of functional assessments against the 90% functional assessment feasibility criterion.
Outcomes change from baseline: Within-group mean change from baseline and between-group mean differences in change for secondary outcomes are shown in Table 3, together with 95% CIs. No significant between-group mean difference in change from baseline scores were found for any functional or patient-reported outcome measure.
4. Discussion
This is the first study to report usability measures for a downloadable app developed for mobile tablet/smartphone or laptop/desktop avatar-guided rehabilitation after TKA. SUS scores for usability of the Health-in-Motion® app reflect the degree of ease, satisfaction, and intuitive learning experienced by individuals in the APP group. The 75% criterion set for usability was satisfied in that 79.2% of the APP group scored the app has having usability of 72 or higher on the SUS questionnaire. Moreover, among the 79.2% who gave the app at least a “good” rating of 72, nearly half exceeded that rating with an “excellent” score (>85%) [32]. These findings are consistent with the satisfaction scores reported by Higgins
![]()
Table 3. Change in secondary outcomes.
KOOS JR: Knee injury and osteoarthritis joint replacement outcome core; PROMIS-PH: Patient-reported outcomes measurement information system—physical health; VR-12: Veterans RAND 12-item Health Survey-physical; mABC: Modified Activities-specific Balance Confidence Scale.
et al. who observed 83% of 31 participants reporting a “good” or higher level of satisfaction after six weeks of app use after ACL reconstruction [8]. To our knowledge only Chughtai et al. [10] evaluated the usability of their modular, non-downloadable system in 157 individuals undergoing knee arthroplasty, including 18 undergoing TKA. Investigating a self-contained tabletop digital health delivery system rather than an app for smart phone, tablet, or laptop, these investigators used the SUS questionnaire to measure usability. The findings of both Higgins et al. [8] and Chughtai et al. [10] agree with our observation that individuals rehabilitating at home after knee surgery are able to assimilate computing technologies to assist with postoperative recovery.
The feasibility criterion of 75% computing device ownership and study use among potential study enrollees proved to be a high mark. Where other studies, including Chughtai et al. [10], delivered an eHealth system to all participants [9] [10], we were interested in pre-existing computing device ownership and study use in the TKA population as an indicator of the potential for immediate adoption and implementation of the Health-in-Motion® app. While the percentage of computing device ownership and use among potential enrollees was fairly robust at 71.4%, it failed to meet our criterion of 75%. However, our findings hold promise for two reasons. First, the 65+ year-old population at large is becoming increasingly literate with technology, presenting positive attitudes toward digital devices. Moreover, older adults tend to adopt mobile computing devices over computers [33]. Second, the average age of individuals receiving TKA is getting progressively younger [34], and this population is therefore exhibiting progressively higher technology literacy and comfort. Considered together with the currently estimated 3.8 billion smartphone users worldwide [35], we expect growing usable device ownership in the TKA population to continue, minimizing this potential technology-related barrier.
Seventy-five percent of respondent individuals who qualified for but did not participate in the study did not identify technology-based barriers. This feasibility criterion was therefore satisfied. Of the 25% of screened non-participants reporting technology-based barriers, the discomfort with technology may have been related to lower self-efficacy regarding perceived ability to learn to navigate new technologies [33]. App developers are challenged with finding ways to overcome self-efficacy issues, especially for technologically averse adults [36]. A screening tool has recently been developed to identify older adults who are most likely to need assistance with mobile app proficiency [37]. Development strategies may need to include settings that allow opt-in for help with on-boarding or features such as enhanced tutorialization that adapt to variable technology literacy among users.
Change from baseline scores on functional and patient-reported outcomes were comparable between APP and CONTROL groups, with no meaningful differences in postoperative outcomes produced via paper-based home exercise or app-based home exercise education materials. Our comparable results are in alignment with the randomized trial by Fleischman et al. [38] who found no differences in knee range of motion at six months post-TKA between unsupervised home exercise implemented either by paper or web-based delivery versus clinic-based physical therapy. In a recent randomized trial, Bäcker et al. [9] found no long-term differences in outcomes between a six-week app-based rehabilitation program and a regular physical therapy program. There is a small but growing base of evidence that app-based rehabilitation methods may be feasible in the home environment. Our study adds to this evidence.
In summary, the Health-in-Motion® downloadable app meets usability as well as most feasibility criteria assessing technology-based barriers to participation as well as the completion of functional assessments before and after TKA surgery. However, feasibility was not demonstrated according to the criterion of >75% of the population opting to use their personal computing device. This is likely attributable to the availability of loaner devices to participants. Comparable end-of-study outcomes scores between participant groups suggests overall feasibility for using app-based evaluations and therapist-selected exercises guided by avatar in the gamified environment for patients after TKA.
Study Limitations
This randomized pilot study had limitations. While it provides important insights for usability and feasibility outcomes, the small sample size did not allow for in-depth comparisons between APP and CONTROL groups. Postoperative rehabilitation and exercise prescriptions were at the physical therapists’ discretion and may have introduced variability between groups. Assessors were not blinded to group allocation or individual outcome scores. Participants in the APP group had more familiarity with the app, and this may have biased their assessments in favor of the app. However, if the app had proven to be more cumbersome, this may have led to negative bias against a favorable assessment by the APP group. Home Wi-Fi access was required for enrollment in the study prior to group allocation. This was deemed an acceptable selection bias based on the aims of the study and the widespread and increasing use of downloadable technology by older adults.
Lower-than-expected personal computing device use was influenced by availability of loaner tablets. Some participants owned one or more computing devices but chose instead to use a loaner tablet. Participants who owned a personal mobile device but chose to use a loaner tablet expressed the desire to keep personal and research devices separate, had cyber security concerns, or chose the loaner tablet because it had a larger screen than their personal device. Future studies of downloadable app usability and efficacy for personally owned computing devices should carefully address these interrelated factors.
Precautions related to COVID-19 during the late study period forced the research team to oversee app-based evaluations remotely for the final eight enrolled participants. Despite this change, the final feasibility criterion examining the percentage completion of all pre- and postoperative functional testing by both APP and CONTROL was met. Successful remote testing oversight with the final eight participants in this study provides impetus to investigate app-driven remote testing in a future clinical trial.
5. Conclusions
The Health-in-Motion® platform for home exercise rehabilitation after TKA surgery demonstrated good to excellent usability for 79% of users as measured by the SUS questionnaire. The app-guided patient assessment capability also demonstrates preliminary feasibility for guiding and administering functional and self-reported outcomes assessments, meeting two of three predetermined, a priori criteria for feasible use in the TKA population—those pertaining to technology-based barriers of participation and completion of functional testing sessions. The high satisfaction experienced by participants with the use of technology-guided exercise after arthroplasty in prior studies [8] [10] together with the high usability index scores observed, the Health-in-Motion® app warrants further study for the implementation of post-TKA home exercises.
Outcomes from app-based home exercise were comparable to traditional, paper-based exercise at 30 days, a further indication of app feasibility. Larger studies are needed to explore the efficacy of app-guided vs. traditional standard-care home exercise rehabilitation after TKA. Moreover, studies should explore the validity and reliability of app-based functional testing in remote and asynchronous contexts, and the feasibility of remote therapeutic monitoring of home exercise adherence compared to traditionally delivered unmonitored home exercise prescriptions.
Acknowledgements
The authors would like to acknowledge Brianna Roat, BS, PRA and Andrea Marshall for laboratory and data support, Charlotte Kurchian, BS and William Pu, MGH and Harrison Manacsa for data support, and Jeri Forster, Ph.D. for statistics support. We also wish to thank all study participants for their time and commitment.
Funding
SBIR Grant Number 1R44AG055341.
Abbreviations
App: Mobile/Laptop/Desktop Application
HEP: Home Exercise Program
TKA: Total Knee Arthroplasty
SUS: System Usability Scale