A pilot exploration of staff and service-user perceptions of a novel digital health technology (Virtual Engagement Rehabilitation Assistant) in complex inpatient rehabilitation

Abstract

Purpose

Digital health technologies have the potential to advance rehabilitation. The Virtual Engagement Rehabilitation Assistant (VERA) is a digital technology, co-designed to increase service-user engagement and promote self-management. This qualitative study explored staff and service-user perceptions of implementing VERA on a UK complex inpatient rehabilitation ward.

Methods

Purposively sampled service-users were allocated to VERA for up to six weeks. The Non-adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework underpinned service-user post-intervention interviews and staff focus groups, and structured analysis of the data. Seven service-users were interviewed. Nine staff contributed to focus groups.

Results

A framework analysis identified themes (and subthemes) structured by the NASSS framework domains: 1. Nature of Clinical Condition, 2. Technology (Ease of Use, Holding Information/Resources in a single Digital Location, Appointments), 3. Value Proposition (Structuring Time, Feedback, Unexpected Benefits) 4. Adopters (Confidence in using Technology, Usefulness), 5. Wider Organisation. Ease of use and storage of key information in a single location were beneficial. Reliability, and provision of accurate and timely feedback to staff and service-users, were identified as essential.

Conclusions

A blended approach is required to meet staff and service-user needs. The potential for VERA in a community setting was identified and requires further investigation. Learning from VERA will support development of other digital technologies and their implementation.

IMPLICATION FOR REHABILITATION

• Digital health technologies have the potential to positively impact rehabilitation but may not be suitable for all service-users.

• Digital health technologies for rehabilitation must be easy to use and reliable.

• Relevant and informative feedback from the digital health technology was considered essential by both staff and service-users.

• Utilising a theoretical framework that focuses on key components of implementation was instrumental for development and evaluation of Virtual Engagement Rehabilitation Assistant (VERA).

Keywords:

• Digital health technology
• rehabilitation
• qualitative
• co-design
• implementation

Introduction

Studies of inpatient rehabilitation units show that service-users spend more than half their time inactive [1–4], yet high intensity of therapeutic activity has been found to improve functional and motor recovery following neurological injury [5–8]. Alongside the need for intensity and frequency of therapeutic activity, there is growing evidence that guided self-management, defined as the ability to respond to the physical and psychosocial impact of one’s own condition [9], has the potential to improve quality of life and self-efficacy for people with neurological conditions [10, 11]. Against the background of this evidence, the Virtual Engagement Rehabilitation Assistant (VERA) was conceived and co-designed [12, 13] by a team comprising service-users, carers, university researchers, and healthcare professionals based at a specialist regional UK National Health Service (NHS) Complex Rehabilitation Unit (CRU), who then worked alongside a commercial software design company to develop VERA. The co-design process [13] involved regular group meetings over two years to establish the brief, develop a first prototype, and consider logistics to facilitate implementation of VERA. This novel and innovative digital health technology (DHT) aimed to provide opportunities for inpatient service-users to increase activity in a ward environment outside of formal therapy, by promoting guided self-management.

The VERA technology is a mobile portal comprised of two elements: 1. a web-based VERA staff portal to manage and upload information and digital resources for individual or groups of service-users; 2. a VERA App. downloaded onto a tablet which enables service-users to access and interact with the digital resources (first prototype designed for use on an iPad Pro 11); in this study we refer to this as the ‘VERA Unit’. The VERA technology has been designed to enable staff to upload a variety of file types through the web-based staff portal, with the aim of providing a versatile tool that can be adapted in real time. The content includes bespoke instructions or videos of a service-user’s personalised programme of therapeutic exercises and activities, and information about their individual rehabilitation goals. Service-users can set reminders to undertake these activities. A personalised timetable showing rehabilitation appointments is available. Staff can add games, additional information and validated assessments to support rehabilitation. To ensure that personal information was held securely, a Data Protection Impact Assessment was undertaken prior to the introduction of the VERA technology to the CRU. This incorporated secure cloud-based storage and password protection of the VERA Units to ensure access to service-user data was restricted to those with authorisation.

Implementing DHTs into healthcare practice is problematic; the majority of digital products are not adopted or used sustainably and often fail to deliver benefit [14]. Recent studies in general healthcare settings have concluded that factors including the design of the DHT, the behaviour of the users and the wider organisational context, are important in determining if a DHT is used [14]. However, these factors are often insufficiently considered in projects that design and implement DHTs [15].

Implementing DHTs in complex rehabilitation offers a particular challenge, with a need to understand how people with physical and cognitive impairments can interact with the DHT, and the behaviours required by staff and patients to promote regular use of DHT. In light of these challenges, exploration of the factors influencing adoption is fundamental to any DHT development.

This study aimed to explore staff and service-user perceptions of VERA during the implementation of the first prototype onto an NHS complex inpatient rehabilitation ward (the CRU).

Materials and methods

Study design

This qualitative evaluation of implementing the VERA digital technology in a complex inpatient rehabilitation setting was informed by a structured tool for the assessment of technology implementation (the Non-adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework) [14].

Research perspective

The study is underpinned by the theoretical perspective of social constructivism [16, 17], in which learning is understood to occur during social interactions such as those in the planned collaboration between service-users and health care professionals using the VERA technology on the CRU.

Ethical approval

This research was approved by the Health Research Authority (293744) and the University of Central Lancashire (HEALTH 0233).

Setting

The setting was a single-site 30-bed Complex Rehabilitation Unit (CRU) within the UK NHS. It comprises 20 beds (rooms) for service-users with Level 1B Rehabilitation needs (mixed disability, highly specialist tertiary rehabilitation), and 10 beds (rooms) for those with Level 2 needs (local specialist rehabilitation) [18]. All CRU service-users are provided with individualised treatment programmes which are updated regularly. Patient-centred care and self-management strategies are promoted. During the study period, approximately 50 medical, nursing and allied health professions staff spent time working on the CRU.

Intervention

The intervention was the introduction of the VERA technology onto the CRU to support service-users’ rehabilitation goals. Staff introduced and used the technology during the study period. Service-user participants were allocated a VERA Unit (i-pad with an app. configured to access the mobile portal) to use for up to six weeks, or until discharge from the CRU, if sooner. Six VERA Units were available. The evaluation followed the experiences of staff and service-users over 30 weeks from February to September 2022.

Sampling and recruitment

Service-users

Service-users on the CRU were eligible for inclusion in the implementation study if they: 1., had capacity to give consent to take part in the study (assessed by clinical staff following usual NHS practice utilising the five principles in the Mental Capacity Act 2005 Code of Practice [19]), 2., had complex rehabilitation needs (requiring a highly-trained multidisciplinary team (MDT)), 3., had rehabilitation goals that could be addressed using the activities and resources accessible through the VERA technology, 4., had been assessed by a speech and language therapist as able to interact effectively with the VERA Unit, and 5., were able to understand English, as the prototype used English language only.

The clinical team reviewed all inpatients at the start of the study. In collaboration with the Chief Investigator (KJ), eligibility criteria and clinical reasoning were used to identify service-users to invite to take part. Purposive sampling [20, p.306] was employed. The clinical team were asked to consider inclusivity to maximise variation in condition, gender and ethnicity in the study sample.

Service-users were approached by a member of the clinical team. Written information about the evaluation was provided. A video version was available, but not accessed.

Staff

All CRU health professions staff in a qualified or support role were invited to participate in the evaluation, if they had worked for more than seven hours a week with service-users trialling VERA. The Principal Investigator at the CRU distributed participant information to staff by email. Staff expressed an interest by contacting a university researcher.

Consent

All participants gave consent. Staff consent was recorded on a Microsoft Form on the secure university network, accessed through the staff member’s work email account. Service-user consent was audio-recorded using Microsoft Teams during a meeting with a university researcher. Supplementary audio-recorded verbal consent was obtained from service-users before each individual interview.

Data collection

Data collection was carried out by two researchers (JC, KJ). All participants were allocated a Participant Code to anonymise their data. Responses from two staff Focus Group Discussions (FGD) are combined to prevent identification of the small number of staff participants from each profession.

Participant characteristics

Service-user characteristics

Service-user participant demographic data was recorded by clinical staff via the online Qualtrics Survey Platform, accessed through the service-user’s VERA Unit: (1) age; (2) medical condition/s; (3) reason for admission; (4) date of injury/start of condition; (5) date of admission to ward (to give an indication of whether being established on the ward influences VERA usage); (6) independent/supported communication (by people or equipment); (7) if supported communication, the form and type.

The online Qualtrics Survey Platform enabled service-users to self-report gender identification and ethnicity through their VERA Unit. They were also asked to report their general self-perceived assessment of technology use (low/moderate/high) and information and communication technology (ICT) use, based on a list adapted from the annual Eurostat survey [21, 22]. This information provided a subjective assessment of everyday technology use and was used in describing the participants in the study.

Staff characteristics

Staff participants were provided with a link to self-report personal data through the online Qualtrics Survey Platform: (1) age; (2) gender identification; (3) ethnicity; (4) profession and band/grade; (5) length of time working in rehabilitation; (6) length of time working on the CRU. They were also asked to report their general self-perceived assessment of technology use (low/moderate/high) and ICT use, based on a list adapted from the annual Eurostat survey [21, 22]. This was also used to describe the participants in the study.

Service-user interviews

Semi-structured interviews were undertaken by a university researcher (JC), these were video-recorded using Microsoft Teams. Interview schedule questions (Appendix 1) were underpinned by the NASSS framework, and explored the barriers and enablers, and benefits or disbenefits of using the VERA technology.

Staff focus group discussions

Focus groups were selected to collect staff data. This method aligns well with the social constructivist perspective, enabling dialogue between staff participants who regularly use team communication to develop their own understanding and learning [23]. This method enabled staff to express views and reflect on these in the context of the perceptions and experience of others. Two FGDs were undertaken. 1. early in the intervention (face-to-face at the CRU, audio-recorded on an encrypted digital recorder), and 2. at the end of the intervention (online, video-recorded through Microsoft Teams). Both were led by a university researcher (KJ), co-facilitated by another (JC). Both took field notes. The FGD schedule (Appendix 2) was underpinned by the NASSS framework, and explored the barriers, enablers, and benefits and disbenefits of implementing the VERA technology on the CRU.

Data analysis

Descriptive analysis of the service-user and staff demographic data, including general self-perceived level of technology and ICT use, was undertaken to provide context for the qualitative evaluation.

The interviews were transcribed, checked and analysed by two university researchers (KJ, JC). The data were analysed based on the domains of the NASSS theoretical framework, using framework analysis [24–26], which has five stages: familiarisation, developing a theoretical framework, indexing, charting, and synthesis.

The researchers familiarised themselves with the data using the transcripts and the original video recordings. Data from the transcripts were coded (indexed) to the NASSS domains using Microsoft Excel. Each interview was independently coded, and discrepancies discussed before a decision was made. One researcher (KJ) grouped the codes to chart the findings within the NASSS domains, which was reviewed and agreed by the second researcher (JC).

Following charting, the data from the service-users’ experiences of using the VERA Unit were synthesised. This enabled examination of similarities and differences in experiences and perceptions of the development and implementation of the VERA technology into the inpatient rehabilitation setting.

The audio-recordings of staff focus groups were transcribed, checked and coded independently by the two researchers who facilitated the FGDs. Framework analysis mirrored the process undertaken for the service-user interview data.

Researchers kept a reflexive journal, field notes and an audit trail of decisions during analysis of the interviews and FGDs.

Results

Service-user and staff participants

Service-users

Sixteen service-users consented to take part in the evaluation, none had been involved in the co-design of VERA. Due to quick discharge, nine participants had limited (if any) use of the VERA Unit and/or were unable to take part in an exit interview. Seven service-users were interviewed; three were receiving rehabilitation for a neurological condition, two for a non-neurological condition, two were missing data. Six interviewees self-reported as male and one as female. All seven were White British. Two were 35-44 years of age, one 55-64, two over 65, two missing data. Five self-reported as self-perceived moderate users of ICT and technology, and two as high users. The number of ICT activities undertaken from a list of 12 ranged from four to nine, with five service-users reporting seven or more activities.

Staff participants

Nine staff provided demographic information and participated in one of the two FGDs. Four of these staff participants had been involved in the co-design of VERA, contributing to the group meetings to design, develop, and implement the VERA technology in readiness for this pilot evaluation. Four professions were represented: medicine, and three allied health professions - occupational therapy, physiotherapy, and speech and language therapy. One staff participant was 18-24 years of age, four were 25-34, two were 35-44, two were 45-54. Three had been working in rehabilitation for two years or less, and six for more than nine years. Five participants self-reported as being moderate users of technology, and four as high users. The number of ICT activities undertaken from a list of 12 ranged from eight to 12, with six participants reporting nine or more activities.

Staff and service-user experiences

The analysis was based on five of the seven domains of the NASSS theoretical framework. Domains 6. Wider System, and 7. Embedding the Technology over Time, did not feature in the service-user or staff narratives due to the early stage of the prototype evaluation. Therefore, in line with a framework analysis methodology, the findings are presented as themes which draw on the NASSS terminology: 1. Nature of Clinical Condition, 2. Technology, 3. Value Proposition, 4. Adopters, 5. Wider Organisation.

Nature of clinical condition

This theme explores the interplay between the service-users’ clinical condition and their use and acceptance of the VERA technology [14].

The service-users noted some condition-based challenges when interacting with VERA. One indicated that the weight of the iPad had caused difficulties, ‘The iPad’s a bit heavy I must admit, but I am very weak.’ (SU3), and two noted that memory impairment was a potential barrier:

If you pick something up, you can forget it the next day, it’s gone. (SU4)

I have been doing a few exercises … I cannot get the exercises to stay in my brain. That’s one fault I have got … I think if you got into a habit of using it, I think it would sit in there a bit better. (SU6)

Other challenges that impacted service-user engagement included fatigue, therapy or medication, and low-mood:

I sleep that much that I’m finding it hard to use VERA. You might think you’ve got all the time in the world to use it, but you haven’t actually cause you’re busier than what you think. … Soon as you get back to your room, the only thing you can think about is catching up on your sleep. It’s just like, do a session, do a physio session. You come back to your room. And you wouldn’t think about, ‘Oh yeah, I must do VERA now.’ You’re worn out. (SU3)

I have felt a bit glum, I think the phrase is. Fed up, fed up and bored. And I just haven’t given everything a try. (SU6)

There was a strong indication that whilst service-users were generally able to use VERA, they thought it might be more challenging to other patients:

My injuries were mainly physical so it wasn’t too bad for me, but anybody who had maybe a more serious brain injury, might find it not as easy to get round. (SU7)

Staff noted that further work is required to ensure that VERA is accessible to those with acquired language impairment:

Looking at VERA going forward, there will always be a significant number of patients on any rehab unit with communication problems. And I think it’s quite language heavy. (FGD)

Technology

This theme contains three sub-themes: Ease of Use, Holding Information/Resources in a single Digital Location, and Appointments.

Ease of use

Service-users reported that they liked the appearance of the VERA App. on the iPad (the VERA Unit), and found it easy to navigate.

And once you logged in you just clicked it, clicked on the programmes or whichever box that you wanted, and the information came up. So it was pretty straightforward. (SU7)

I found, as you use it, you do get into it a little bit more … Find a way around the screens and that, but yeah, it’s … pretty self-explanatory, it’s pretty good. (SU6)

It’s a simple enough layout. You know, so it’s very self-explanatory which makes it easy, very easy to use. (SU5)

It looks good. Yeah, it’s set out OK. You know, it’s explanatory what each box was. You know what it was for. (SU4)

However, staff felt that the VERA portal could be improved to make it easier for them to use:

The staff portal still feels really quite difficult … even just like from using it and trying it, there’s still a lot of stuff that needs to be ironed out on there. (FGD)

Another noted; ‘If electronically it’s slicker, then that would be better’ (FGD).

The VERA staff portal appeared to become easier to use with practice, and uploading resources was found to be straightforward. One staff participant who had been using VERA regularly noted:

I found that process actually the easiest part of it, creating the programme the same way you would do using an online service and then just uploading the PDF. That’s really, really good. (FGD)

Holding information/resources in a single digital location

Staff appreciated the VERA technology’s ability to hold information in a single location:

The main benefit has been collating everything together so patients who have a multiple programme, so a bed-based programme, a stretching programme and then a standing programme to complete. Having those three sets of paper just floating around makes it quite challenging for them to keep track of. When they move rooms, they go missing. (FGD)

However, new challenges were identified in transitioning to the digital technology. When service-users were discharged home, staff found it difficult to download support information from VERA. Resources previously available as a paper copy were now held only in a digital form.

You get families all familiar with it and they’ve got all this information and resources to work through with their family member. And obviously, we have some that move very quick through the pathway. They’ve then got nothing the next day or for the week. And so, some in the office was trying to find a way of printing off all the exercises before they [the service-user] are discharged because you don’t want to have all these resources and then the family have nothing … Because ultimately obviously with a paper copy, they could be taking it with them. But all the information about their injury and things doesn’t just become void when they move on from VERA. (FGD)

Service-users could also see a potential benefit from holding information in an accessible central location, but this transition was not in place for this first evaluation, leading to gaps in information:

I was given a leaflet on fatigue. I took that home and left it with [partner] to read, it might have been handy to have been uploaded onto VERA … so I could read it over again and get it firmly in my mind. (SU7)

Appointments

The appointments function was designed for service-users to see all their upcoming rehabilitation sessions.

Staff felt that the feature was an important function of the VERA technology:

It’s a massive focus of patients’ days … ‘What time is it?’ … Yeah, they’re really on board with that. (FGD)

This was supported by service-users, who wanted to access this information independently:

I’ve always needed to be pushed around, or taken to places I needed to be. So, I can imagine for people that are in the same situation, being able to find what appointments they’ve got rather than go to the [notice] board, which is out of the room. (SU5)

I spoke to my physios and the OT people, and they actually put a programme up in my room so I could look at any time. But I felt that would be helpful because normally without the programme, without VERA, you have to sort of ask whether my physio appointments and when are my OT appointments the next day. You know, that is something that would be helpful … because sometimes people don’t get washed before they go to physio, or there’s a problem there. I mean, that has happened to me at least once. But I always like to try and have a wash, before I go to physio. (SU1)

However, during this evaluation, the appointments function did not work reliably:

Some of the dates, they wouldn’t go onto the calendar properly …. Occupational therapy times come up at lunchtime and things like that. (SU7)

Yeah, they had a problem with the dates and times, so they were not going on to it. (SU2)

Staff recognised that inaccurate timetabling could influence service-users’ confidence in the VERA technology, and thus their adoption of it:

I think the patients would really benefit from that [timetable] because they always keep wanting to check it. ‘Oh, you know, my appointments are on there’. And then if one’s not correct for whatever reason, ‘Oh, I didn’t trust it because it was wrong last week’. (FGD)

… they then don’t go to the iPad to check it, they go to the board, because they’ve tried it once and it was incorrect … It will always be important the information on VERA is accurate. We’re trying to change culture. If they start sneaking back to the old ways, so the tiny thing’s important, isn’t it? (FGD)

Value proposition

The Value Proposition domain of the NASSS relates to the perceived value the technology holds for the stakeholders [14]. This theme contains three sub-themes: Structuring Time, Feedback, and Unexpected Benefits.

Structuring time

Service-users reported that the VERA functions had potential to support structuring their days, for example planning visitors around scheduled rehabilitation appointments:

SU5: You could sort of forward plan with the VERA in terms of seeing what appointments are where during the week. If you’ve got visitors, you can then work around that.

[Interviewer: Of course. Yeah, because the visiting has been quite restricted as well. So I guess you’ve had to think about that quite a lot?]

SU5: Yeah. So the VERA certainly eliminates those sort of problems.

Service-users also noted that the VERA activities were helpful in filling time, and reducing boredom and isolation:

Yeah, it definitely does help break the boredom up a little bit. (SU6)

So I like the fact that they were on there and there’s the option to have games … also keeps [service-users] busy. If you get a little bit of boredom kicking in as well. (SU5)

Because I know from talking to some other people that have used it. They’ve said it was quite good when they were bored or they were lonely, they didn’t have anyone to talk to. (SU2)

Feedback

There was a clear message from service-users that VERA did not, and should not, replace a therapist:

When you’re doing the exercises, you need the expertise of a technician. They know – are you doing the actual exercises? … If you’re not doing them in the correct manner, say if something’s out of align you can be doing your body more harm than good. (SU3)

The original design brief stated that the VERA technology should provide feedback. The first prototype enabled service-users to mark an activity from their programmes as completed, but at this stage of development the information was not visible through the VERA staff portal.

The service-users indicated that this function was important for their motivation, and they needed to know that staff were aware of what activities had been completed.

If you’re trying to see if people are actually doing them [the exercises], going into the programme and checking them. And there’s no way of sort of tracking them sort of things. (SU5)

Staff agreed it was important that the VERA technology provided them with this information about service-users’ engagement:

As long as it gives me feedback it gives me the number of times, for example, this patient – did they do exercise on Saturday and Sunday? (FGD)

Unexpected benefits

Service-users reported some benefits that were unexpected by the VERA Project Team. One service-user indicated that VERA had increased family involvement in their rehabilitation, and connections with younger children:

I have found it useful, especially when I’ve gone home at weekends. My youngster gets involved and can help me a little bit. (SU6)

One service-user awaiting discharge noted the potential benefit of VERA for those transitioning to the community rehabilitation team:

It would have been quite good to have that to go home with. Knowing that I’ve got a six-week wait before I have physio, when I go home, it would have been handy to have something in between. (SU2)

One service-user suggested developing use of the quantitative measures, which had been incorporated specifically for the evaluation:

I think the questionnaires were good that we did together, the emotions and like how you’re feeling and stuff. It was good probably for them to get an idea of how we were feeling, and it made us speak about how we’re feeling more. ‘Cause there’s a lot of things, like you wouldn’t say to them how you’re feeling, or about your pain so much because you just go and do your sessions. And no-one would really ask how you’re feeling in yourself. So I think that was a good thing. (SU2)

Adopters

The analysis considered both staff and service-users as potential adopters of the VERA technology. This theme contains two sub-themes and explores the ways in which it was adopted, and factors that impacted this.

Confidence in using technology

Service-user adoption was impacted by general confidence in using the technology, with divergent experiences. For some, using the VERA Unit was straightforward:

Anybody that’s used a tablet or a mobile phone should have a reasonable idea of how to get round it. (SU7)

Oh, if you’re used to using an i-pad it’s fine. It’s quite user-friendly. (SU3)

But others struggled:

I think it was just me. And it’s just me and my confidence, you know, with using it. (SU4)

This was echoed by the staff:

[a patient] got a bit frustrated sometimes with some buttons and things … wasn’t very used to technology … didn’t have a touch screen phone or anything, so it kind of goes hand-in-hand with that. (FGD)

Usefulness

One service-user expressed a preference for paper resources:

I probably looked more at the paper than on the tablet, and probably found it easier that way … they’ve given them the same time as I had it on the VERA, so it was the same stuff … So I think I’m more into the paper side of things … because it’s all in one thing. Instead of separate exercises, it was just on one sheet (SU2)

Two could rely on memory due to the nature of their conditions and therapy:

As I went on further into the study, I wasn’t using the iPad as much for that because I know the exercises literally off by heart now. (SU5)

Rather than bring it all up on VERA, it was just nearly as easy to do it from memory and that’s what I did, to be honest. (SU1)

Another service-user said they needed to set additional reminders on their phone to prompt them to use VERA:

I’m in an exhausted condition 90% of the time, and it doesn’t take much to make you even more exhausted and want to go back to your room and just fall asleep. You fall asleep in the day – but it impinges on your sleep at night. Just circle. Question of VERA is OK. But you need to like set reminders on your phone to remind you to do it. (SU3)

Staff adopters appreciated that they could update the personalised exercises through the VERA staff portal in real time during a therapy session:

I like that you can take it down and adapt it within [a] session. Paper-based you wouldn’t be able to do that because we would have to go back [to the computer] and do it, whereas you could have somebody doing 10 minutes on the bike whilst you tweak the exercises, take the photos, film - it’s more interactive within the session. (FGD)

This member of staff had adopted the photo/video facility, but overall, staff reported that they did not use the video upload very frequently. Staff expressed some uncertainty around this, raising concerns about using the iPad to take videos in a way that protects data/other patients.

So, obviously our gyms are quite busy, so you just need to time it right to be able to fit in. And if it’s something like walking, it’s quite unusual that if you had somebody walking up and down the gym that there wouldn’t be anybody else. (FGD)

Service-users felt the VERA technology would be more useful if further resources were available:

A lot of stuff wasn’t on it, like the games and stuff, but things like that I probably would have gone on to more, as well as doing the exercises. (SU2)

Organisational factors

The Organisational Factors theme addresses how an organisation, its structure, routines and readiness to embrace innovation can influence the adoption of technology [14]. Some organisational factors were considered in the FGDs.

Staff explored how allied health professionals (from occupational therapy, physiotherapy and speech and language therapy) adopted the VERA technology in their practice:

I think the therapy team has taken full ownership and full involvement in the VERA rollout. (FGD)

There were a number of therapist staff champions who used the VERA technology regularly and became advisors to those with less experience:

If it was like a bigger systems issue then it would just be [VERA Project Team members]. But if it was just an inputting thing, that would probably be somebody from within the office. Everyone who is in the office knows who’s on VERA. (FGD)

The therapists recognised that it took time to learn how to use the system:

I think with any new skill, it takes a while to master. And I think if we can put more into it in the sense of like content for the patient, then we can potentially buy that time back elsewhere. (FGD)

It also required an investment of time to set up VERA for a service-user and then keep it updated:

I just think the practicality of finding time to do it … out of my time getting all those appointments on, physio appointments are there, exercise programmes. It was quite time consuming (FGD)

Therapist participants emphasised the challenges of implementing VERA in the context of limited time and high therapist workload in their clinical setting, indicating that for VERA to be feasible the technology would need to be integrated with the electronic records system:

We will probably still want to write into the electronic records and have it talk to VERA.

It would almost be good if you could have it so that you write on the electronic records and then have a box that then says, ‘send to VERA’. (FGD)

One staff participant reflected on the features of the VERA technology in relation to the expectations and requirements of the organisation:

I think there is always that assumption that technology will make things slicker. And I think VERA, because it’s still in the very beginning stages, definitely there are elements that don’t make things quicker … And when you present it to the patient, when you’re describing what features it can do, it reminds you every time that these features actually are very good. I think if it was slicker and less time-consuming, it would be really, really useful in all of the elements of it. (FGD)

This comment encapsulates the overall findings, suggesting that whilst the features from the initial design brief were desirable, the VERA technology requires further refinement during the subsequent design process to meet its envisaged potential.

Discussion

Digital Health Technologies are increasing in popularity [27], and have the potential to enhance rehabilitation [28, 29]. However, there are few examples of the use of qualitative service-user experience feedback and co-design of DHT in the rehabilitation improvement literature [30]. This study explored staff and service-user perceptions of VERA during the implementation of the first prototype onto an NHS complex inpatient rehabilitation ward (the CRU). The findings highlight factors relating to the technology, the value the technology offers, the behaviour and needs of adopters, and the influence of the wider organisation upon implementation.

All service-users self-reported at baseline that they were moderate or high users of digital technology, completing at least four different tasks using ICT in their daily activities. These participants said that the VERA App. was well-laid out and self-explanatory, but some still expressed a preference for alternatives to the DHT (using paper or memory). A blend of such options has been recommended to increase accessibility and respect users’ choices [31]. In line with previous studies [32–34] service-users also noted that problems with memory, fatigue and mood, as well as the physical requirements of interacting with VERA impacted their use of the DHT. With digital access and skills increasingly recognised as a social determinant of health [31], our participants’ experiences reflect the importance of considering the needs of users when developing DHTs such as VERA, and underlines the importance of inclusive design. In the UK, 92% of people aged 16-75 have access to a smartphone [35], and 64% have access to a tablet [35] (p3). It was therefore assumed that most service-users would be familiar with and able to use the app-based VERA technology. This assumption was largely supported in this study, but one service-user (contrary to their self-reported technology-use), said that they did not routinely use digital technology and struggled to embrace the VERA App. Despite service-users reporting that VERA was relatively easy to use, therapy staff and service-users noted that the current iteration of VERA would be unlikely to be acceptable to those with language and comprehension difficulties. This is supported by the wider literature, as people with impairments are reported to experience significant challenges when using digital products, with 66% of those with learning or memory challenges, and 55% with vision or hearing issues struggling to attain basic levels of digital competency [36].

Both staff and service-users reported that VERA was relatively easy to use, and the storage of information in one place positively contrasted with traditional paper-based methods of information provision. They also noted that the ability to see appointments on VERA was a beneficial feature, although this was not always reliable. Previous studies [37–41] have identified the importance of the reliability of the system to support implementation of technology.

The inability of VERA to provide feedback was a frustration reported by both service-users and staff, who noted that feedback was particularly important to generate and maintain service-user motivation. Feedback is a recognised technique to support changes in behaviour [42], and is particularly important in a rehabilitation setting where a DHT aims to change the behaviour of service-users, for example, to increase activity or promote self-management. The importance of feedback that is accurate and measures progress has been highlighted in other DHT research [34, 41, 43]. Future iterations of VERA should prioritise the provision of high-quality feedback to service-users, alongside inclusion of other well-documented techniques to change behaviour (such as self-monitoring, reinforcement and reward) [44].

This study also considered the changes in behaviour required for staff to introduce and adopt the technology into their usual working practices [45, 46]. Health professionals’ behaviour is rarely considered in the development and implementation of complex interventions and so the inclusion of their views adds to the novelty of our work [47]. Staff users valued several aspects of VERA and were keen to expand its functionality, but some features were underutilised (e.g. video recording functions). Staff reported that using VERA required time to learn, and it also took substantial time to identify, produce and upload appropriate individualised resources for service-users to access, which could challenge the feasibility of VERA’s wider implementation. This could be mitigated by developing a library of resources, containing standard information that can be combined with individual exercises, to both support personalised rehabilitation and reduce the staff time required for these tasks.

Some staff participants had been involved in the co-design of VERA, and developed expertise in its use, providing a valuable resource for others who were less familiar with DHT and VERA. This underlines the value of readily available technological expertise, which has been reported by others to support the implementation of new technologies [48, 49]. To improve the feasibility of implementing VERA on the CRU, staff felt that the VERA technology would ultimately need to be integrated with the NHS electronic records system to avoid duplication. This would have additional ramifications such as necessitating a further Data Protection Impact Assessment to ensure the continued safety of personal information.

Whilst the VERA technology was designed for use in the inpatient setting of the Complex Rehabilitation Unit, staff and service-users identified features of the VERA technology that could be useful for service-users living in their own homes or other community settings. With a growing emphasis on the provision of rehabilitation from home [39], this was identified as a priority for future development. Collectively, this study highlights that future iterations of VERA, and the development of other technologies for people with complex rehabilitation needs, should consider the spectrum of skills, abilities and location of service-users.

Limitations

We recognise the potential bias in the staff participant data, implicit with four staff participants being involved in the co-design of the VERA technology. This approach is in keeping with a co-design partnership. The four staff participants were key members of the clinical team at the time of the study, therefore, excluding these staff would have led to a detriment in the depth of findings. This bias was not present in the service-user sample, as no service-users had been part of the co-design process or had any experience of the VERA technology before commencing the study.

Only medical staff and therapists responded to the invitation to take part in this evaluation, so other staff views have not been explored. It may have been beneficial to more actively involve e.g. nursing staff in the implementation and evaluation, but further work is required to explore what impacted the decisions of staff who did not engage in the co-design process or the adoption of the VERA technology.

Short stays on the CRU meant not all service-users were able to use the VERA Unit for the intended six weeks. We were not able to collect detailed usage data in this initial evaluation, and this should be considered in future studies. We did not have ethical approval to actively follow-up service-users once they had left the CRU, and none contacted us. This impacted the number of exit interviews. Future work should study the use of VERA over a longer timeframe.

The nature of this study meant that the sample of service-users was small and comprised people with a limited range of conditions and ethnicities who tended to be previous moderate to high users of technology. There are indications that less reliance on the written word, and the ability to adapt for easier use in the presence of cognitive and visual impairments will increase accessibility and should be features of future iterations of VERA. Maximising participant variation is a challenge experienced by others [30]; however, we need to need to find creative ways to address this limitation and facilitate the recruitment of a diverse range of people, including low users of technology.

Conclusion

This study explored staff and service-user perceptions of the VERA technology in an inpatient complex rehabilitation setting. It contributes important data to inform future iterations of VERA and highlights several important factors that influence the use of rehabilitation technologies more widely. Enablers were the appearance of the App., and ease of use. There was enthusiasm for expanding the DHT through integration into the electronic record system. The potential to transfer use to a community setting was also identified. Key challenges included the low reliability of some features of the technology, service-user confidence, and the time it took staff to use it, particularly in the initial stages.

Iteration is inherent in co-design [45, 50], as end-users establish what works well and what requires further development [45]. This study highlights the importance of the continued involvement and engagement of all stakeholders in co-design. It underlines the value of co-design as an ongoing process, rather than a single event, when developing rehabilitation, to ensure technologies meet stakeholders’ needs and can be adopted into practice.

Acknowledgments

We thank the staff and service-users at The Walton Centre NHS Foundation Trust for their support of this study.

Disclosure statement

VERA was developed through a co-design process. Citrus Suite and The Walton Centre will benefit financially if VERA becomes available commercially. The researchers from the University of Central Lancashire have no conflicts of interest.

Data availability statement

Data is available from the corresponding author upon request.

Additional information

Funding

This work was supported by the following funding scheme: MedCity’s Collaborate to Innovate in conjunction with the Stroke Association, under Grant C2N-SA.06.A

Notes on contributors

Kathryn Jarvis

Dr Kathryn Jarvis is a Senior Research Fellow and Occupational Therapist in the Stroke Research Team at the University of Central Lancashire. Her research interests include the implementation and use of digital health technologies to enhance neurological recovery.

Julie Cook

Dr Julie Cook is a Research Fellow in the Applied Health Research Hub at the University of Central Lancashire. She has a special interest in the ethics of qualitative research with participants with intersectional vulnerabilities.

Ganesh Bavikatte

Dr Ganesh Bavikatte is a Consultant and Clinical Lead of Rehabilitation Medicine at The Walton Centre NHS Trust and a Senior Clinical Lecturer at the University of Liverpool. He is actively involved in various research and innovation projects in the field of neurorehabilitation following acquired neurological injuries.

Nicola Branscombe

Nicola Branscombe is an Occupational Therapist who works at The Walton Centre NHS Trust and has specialist interest and knowledge in neurology and complex rehabilitation.

Steve Donovan

Steve Donovan is the Chief Operating Officer at Citrus Suite, where he leads the design and strategic execution of complex software projects. He has played a pivotal role in steering the company’s operations, with a keen focus on enhancing efficiency, ensuring top quality, and driving innovation in software development.

Jo Haworth

Jo Haworth is a Clinical Specialist Physiotherapist who works within the Cheshire and Merseyside Rehabilitation Network hosted by The Walton Centre NHS Trust. She has a special interest in the treatment of patients with complex rehabilitation needs and complex neurological diagnoses.

Charlotte Lawrence

Charlotte Lawrence is a Speech and Language Therapist who works at The Walton Centre NHS Trust and has specialist interest and knowledge in neurology and complex rehabilitation.

Chris Morland

Chris Morland is the CEO and Founder of Citrus Suite, a creative software company based in Liverpool, UK. With a passion for digital health innovation and technology for good, Chris leads a team renowned for developing bespoke software systems, mobile apps, and web platforms.

Rachel C. Stockley

Dr Rachel C. Stockley is an Associate Professor and Physiotherapist in the Stroke Research Team at the University of Central Lancashire. She undertakes programmatic research to understand how to improve recovery, particularly of the upper limb, after stroke, and how technologies can be used to improve stroke rehabilitation.

Appendix 1.

Interview schedule

1. I have lots of detailed questions, but firstly I would like to know your overall impressions of VERA?

2. I would like to take you back to the beginning and think back to training? Who provided the training? (prompt: who showed how to use VERA?)

3. Can you remember what they told you?

4. Were you able to use this information?

5. How have you used VERA over the last 6 weeks (or less if discharged or stopped using VERA)?

6. Was VERA easy to use?

7. What were your thoughts about the way VERA looked? And what was it like to handle?

8. What did you use it for? (prompts: exercises, information, timetabling…)

   1. Was your rehabilitation better with VERA than before VERA?

   2. What rehabilitation like now … (probably post-VERA)?

9. Was there an aspect (or aspects) of VERA that you particularly liked?

10. Were there elements that you like or didn’t use? If so, why?

11. Were there things that made VERA easier use?

12. Were there things that made VERA more difficult to use? Did you find a way around these? Did you need support to do this?

13. Would you want VERA back again if this was an option?

14. What advice would you give to someone else being offered the use of a VERA?

15. Have you got any advice for the trainers/to develop the training?

16. If we were going to develop it further what would we need to do?

Appendix 2.

Focus group schedule

1. I have lots of detailed questions, but firstly I would like to know your overall impressions of VERA? (reflection: consider use of show of hands)

2. Is there any feedback about the training of VERA? (what worked well, what didn’t work so well and are there suggestions for developing?)

3. Did the training help you use VERA in your clinical practice/job?

4. What were your thoughts about the way VERA looked? And what was it like to handle? And use?

5. What did you use it for? (prompts: exercises, information, timetabling…)

6. Did this alter change your job? If so, how?

7. Was there an aspect of VERA that you particularly liked?

8. Were there elements that you didn’t like or didn’t use? Why not?

9. Were there things that made VERA easier use?

10. Were there things that made VERA more difficult to use? Did you find a way around these? Did you need support to do this?

11. Have you got any advice for the trainers/to develop the training?

12. If we were going to develop it further, what do you think we should do?