“Qualitative evaluation of a digital software solution for documentation and training in 24-hour home care”

ABSTRACT

Owing to its cost-efficiency, 24-hour home care is increasing and represents a cornerstone in the care of elder people in Austria. A software solution to support 24-hour caregivers in their daily routine was developed facilitating a user-centered design process. It includes electronic documentation, an e-learning platform, emergency management, and a communication tool. To evaluate the usability and acceptance of the developed software solution, a qualitative survey including focus groups (FG), a group interview (GI), and a usability walkthrough (UW) were conducted. This paper describes the findings of the qualitative survey which indicate that the e-learning platform as well as the e-documentation support 24-hour caregivers in their daily work with their clients. Participants with less technical affinity needed support to use the tool proficiently. 24-hour caregivers appreciate the fact that the solution unites all the needed information in one place.

KEYWORDS:

• Active and assisted living
• digital software solution
• qualitative study
• quality assurance
• 24-hour home care

Introduction

The Austrian model of 24-hour home care supports more than 33,000 individuals in need of care and facilitates an important alternative to informal or residential care. The model provides 24-hour home care by caregivers, mostly migrants from Eastern Europe, commuting between their home country and Austria in fortnightly shifts (Schmidt, Leichsenring, Staflinger, Litwin, & Bauer, 2020). They provide full-time personal care to older adults in their own homes, including support with everyday needs such as assistance with cooking and housekeeping up to nursing duties like the ingestion of food (Haslinger-Baumann, Lilgenau, Binder, & Gugenberger, 2019). This care model offers extensive support, especially in terms of time, something which existing local mobile care providers are not able to offer affordably (Bauer & Österle, 2013). In times of an aging society, increasing demand for a cost-efficient model of 24-hour home care is to be expected in the coming years.

Due to the wide range of illnesses of the persons to be cared for, caregivers also have to take over nursing tasks for which they are not trained. Undertaking these tasks requires proper training and written delegation by a registered nurse (RN). In many cases, this however does not take place (Haslinger-Baumann, Lilgenau, Binder, & Gugenberger, 2019).

This leads to extremely challenging and exhausting working conditions for caregivers (CG), including weeks of isolation with a sick person. Caregivers are mostly migrants from countries such as Slovakia, Romania, Hungary, and Bulgaria, often lacking German language skills, which leads to language barriers between the person in need of care and his/her family. This fact reinforces the feeling of isolation and poses challenges within the work setting (Haslinger-Baumann, Lilgenau, Binder, & Gugenberger, 2019). Digital tools for care documentation save time and ensure quality of care (Röhm, 2018). Therefore, a modern digital solution that meets at least some of these challenges and the needs of 24-hour caregivers is needed.

The digital software solution “24hQuAALity”

Based on the findings of a user survey including 18 currently employed 24-hour caregivers, 14 relatives, ten individuals in need of care, and three RNs, a software solution that meets the requirements of all those people affected by the 24-hour care setting was developed (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022). “24hQuAALity” provides a wide range of support options in 24-hour home care, containing digital care documentation, an e-learning platform, and an emergency management system. Furthermore, a communication platform, as well as additional tools such as a translator and recipe app, are included (see Figure 1) (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022). To address the language needs of most 24-hour caregivers, the contents are available in four languages. In addition to German, all apps can be used in Slovak, Hungarian, and, Romanian (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022).

Figure 1. 24hQuAALity app.

The e-documentation system developed represents the first profound and individually adaptable documentation for 24-hour care services. Conducted care activities can be entered efficiently, mainly by clicking/tapping. This also serves to avoid potential translation challenges common within written text. In addition, free text fields are available if qualitative inputs are needed. Another important feature of the e-documentation platform is the indication of care activities requiring delegation from a RN. If such an activity is entered, the caregiver receives a note. To legally secure the care activity, permitted delegations can be stored in the e-documentation system (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022).

An emergency management system, which joins all important information and phone numbers needed in case of an emergency, is integrated into the e-documentation app. Therefore, existing emergency numbers were put together along with information for questions asked by the call center in case of an emergency. Moreover, the name and address of the client as well as their insurance number and medical history are displayed at one glance (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022).

The e-learning tool is based on the existing learning management platform “Talent LMS”(Talent Lms 2022). A total of 33 courses and 35 videos were developed and are offered on this platform. Learners can select the courses in any order and repeat them as often as they like. At the end of each unit, there is a short quiz; if passed a certificate of attendance can be downloaded. Learning contents are available in Slovak, Hungarian, and Romanian. The video voiceover is in German with selectable subtitles in the aforementioned languages. This provides a learning possibility for caregivers, which can be used independently of time and location and can also be repeated whenever needed. The learning content is customized to typical care settings in 24-hour home care. It provides topics about handling the most common illnesses among older adults, how to support them in personal hygiene, and housekeeping. Moreover, one unit addresses the legal matter of working as a (self-employed) 24-hour caregiver in Austria. In addition to the emergency management system, the e-learning platform provides courses on training emergency skills for 24-hour caregivers (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022).

The social media platform”Facebook” was installed on the tablet and a moderated Facebook group was created. This allowed participating 24-hour caregivers to communicate with each other and ask questions regarding the study or their care setting itself. Furthermore, the messenger “Signal” was provided on the tablet, to offer a secure way to communicate with the exchange colleague or the relatives individually or in created groups. The well-known app “Chefkoch” is included as an online tool with a collection of recipes for making use of leftovers and suggestions for meal preparation (Hauser, Kupka-Klepsch, Haslinger-Baumann, & Werner, 2022).

AAL technology

AAL stands for “active and assisted living” and addresses assistance systems for healthy and independent living. This implies using new technologies in a social environment, including concepts, services, and products to increase the quality of life, especially for older adults (AAL Austria, 2022). AAL technology supports the control of health problems in all dimensions of the older population and can therefore contribute to an improvement in quality of life (Siegel & Dorner, 2017). The 24-hour care model utilizes the unique element of personal care around the clock in private homes of persons in need of care, which makes it especially individual and specific (Haslinger-Baumann, Lilgenau, Binder, & Gugenberger, 2019). By supporting caregivers in their daily work via a structured documentation solution, improved care education, and emergency management, an improvement in the care setting can be achieved. Therefore,“24hQuAALity” meets the requirements of AAL technology.

Methods

Design and participants

This paper describes the qualitative survey conducted to evaluate the usability and acceptance of the developed software solution and experience of use. T Therefore, focus groups (FG) and a group interview (GI) were conducted after using 24hQuAALity for approximately three months as well as a usability walkthrough (UW) with eight participants.

Participants from different target groups were recruited to attend the study. The primary target group included older adults who received 24-hour care. The secondary target group included 24-hour caregivers themselves, as well as registered nurses (RN) and relatives of the person in need of care. Care organizations involved in the project “24hQuAALity” recruited participants from the different target groups in the environment of their professional sphere of 24-hour home care.

These included large local care organizations and smaller 24-hour home care agencies. If a participant was recruited, all relevant people (relatives, RNs, and persons in need of care) from the respective household were invited to participate in the study. Participating households received a tablet with “24hQuAALity” and incorporated it in their daily care routine.

A total of 25 people took part in the qualitative surveys. Due to the corona pandemic, no participants from the primary target group of elderly people could be involved in the qualitative surveys in order to avoid the risk of an infection. Also, since this target group is distinguished by its high age and therefore often a low technological affinity, interviews via Zoom or telephone were not feasible.

All participants were informed comprehensively about the study and gave their written informed consent. Interview guides were used to structure the dialogue. They were tailored to each target group and along with the contents of the developed software solution (e-documentation, e-learning, emergency management, and networking platform). The group interview and focus groups were held via Zoom and were conducted from March 2021 to July 2021. They lasted from 60 up to 110 minutes. The usability walkthrough took place in person with a trained testing person. One iteration of the usability walkthrough procedure took about two to three hours. See Figure 2 for an overview of methodological approaches within the qualitative surveys evaluating “24hQuAALity.”

Figure 2. Methodological approach evaluating the 24hQuAALity app.

Focus groups and group interview

Two focus groups, one with eight 24-hour caregivers and one with seven registered nurses, were conducted. Moreover, a group interview with two relatives of care receivers has taken place. Therefore, a semi-structured interview guide, tailored to each of the target groups, including questions regarding the benefits of the solution, the integration into the daily work routine as well as questions on necessary improvements regarding content and usability, was set up. Examples of questions from all target groups interviewed along the different topics can be found in Table 1. All interviews were held and recorded using Zoom, they were also moderated by two researchers. Written informed consent was obtained in advance for participation and recording.

Table 1. Example questions from all target groups interviewed along the different topics.

Category	RN (registered nurse)	24-hour Caregivers	Relatives
Usage	How did the work process change since the applications were used by 24-hour caregivers?	How has daily work changed since you started using the applications on the tablet?	How does the 24-hour caregiver integrate the provided applications into the care of your relative?
E-Learning	Which of the provided content supports the work of 24-hour caregivers and why?	How does the e-learning content support your work as a 24-hour caregiver?	What added value does the e-learning tool offer 24-hour caregivers from your point of view?
E-docu	\What content do you see supporting the work of24-hour caregivers and why?	What content is supporting your work setting and why?	What changes did you notice in the care of your relative since the 24-hour caregiver started using e-documentation?
Emergency management	What content do you see supporting the work of24-hour caregivers and why?	How did emergency management help you in case of an emergency with your client?	Which aspects of emergency management are useful in case of an emergency with your relative and why?
Communication platform	How do you use the communication platform for correspondence with the 24-hour caregiver?	How does the 24-hour caregiver use the communications platform to correspond with you?	How do you use the communications platform?
General questions regarding the applications	What changes have you noticed since the 24-hour caregivers started using the applications on the tablet?	Where do you need more support for a comprehensive use of the software solution?	What else does the provided digital solution need to further improve the care of your relative?

Usability walkthrough

Participants in the usability walkthrough (UW) were recruited through convenience sampling by a project partner. Test users attending the usability walkthrough had to be introduced to the “24hQuAAlity” system first. This was conducted by using specifically developed manuals and instructional videos. Moreover, the interviewer gave a short introduction on how to use the tablet and its content. After that, the test user worked on the tasks given and provided structured feedback to solve the task (see example in Table 2). Items were dichotomous (yes/no) and asked on the one hand if tasks could be solved by participants and on the other hand for their rating of the task from “very good” to “very poor” regarding how well they could get along. Additionally, they could leave further comments. Furthermore, the tester made additional notes. Provided tasks included issues from e-documentation (6), e-learning (1), and emergency management (1). Participants were mainly women, only one person was a man. The individual testers shared the same native language as the respective participants.

Table 2. Task sample from the usability walkthrough.

Due to her dry skin, Mrs. Riegler needs to be creamed with body lotion once a day. Note in the e-documentation that you applied lotion to her skin today.
Task solved?	Yes/No
If not, why not?
How did you get along with the task?	Very good/good/bad/very bad

Data generation and analysis

Interviews were transcribed verbatim and analyzed according to Mayring’s qualitative content analysis (Mayring, 2015) using MAXQDA Software (Version 2020). To enable a structured description of the collected data, a deductive category application according to Mayring (2015) was used. For this purpose, each application of the developed app was used as a category and the material was systematically extracted. In addition, new categories were generated inductively throughout the content analysis. Data that emerged from the usability walkthrough were evaluated along with Polit et al. quantitative recording plus classification content analysis (Polit, Beck, & Hungler, 2004). Table 3 gives an example from the analysis procedure of the usability walkthrough for one topic.

Table 3. Example usability walkthrough.

Main category	Sub-category	Value	Quotation	Compaction	Count
E-Learning	content	interesting detailed topics; videos; lesson “first aid”	“Courses are great, especially when someone starts. I think first aid is the most important topic.” UWPB_5	suitability of the content	13
		lesson “mobilization” is not realistic; hard tests questions	“The presentation of the mobilization from bed to chair is very well presented, but in reality, people can’t move that well. With my patient there is much more to take over.” UWPB_5		4
	display	content was easy to find; log-in was easy	“Found the lesson straight away.“UWPB_3	orientation and readability	2
		lessons layout unclear; wording of course names; font too small	“Unfortunately, it is not possible to see exactly how many topics/videos are in one chapter. Therefore, it is difficult to find something specific later.” UWPB_1		3
	benefit	information centralized in one place; certificates available for download; improves German language skills; helpful for refreshing basic knowledge and further training; motivates me to complete the course	“E-learning is very good, helps me a lot in my work, it’s all together in one place, don’t have to click here and there on the internet.” UWPB_7	comprehensive learning opportunity	13
		contents are also available over the internet	„ … not sure, if I would pay money for it. It‘s all on the internet as well.“UWPB_3		1
	other topics	downtime, if answers are wrong	“When you give wrong answers, it takes too long to get on, can happen that I don’t continue learning.” UWPB_4		1

Ethics

Ethical approval was obtained from the ethics committee of the Evangelical Hospital Vienna. The selected ethics committee has a special focus on complex research projects in extramural healthcare. Participation in the study was voluntary. Beforehand, interested people were informed about the study comprehensively and had the opportunity to ask questions. After that, written informed consent was obtained from all participants in the study. All data that emerged from the study were anonymized, and by reading the results, no conclusions about individuals can be drawn.

Results

Hereafter, findings of the qualitative survey regarding usability, acceptance, and impact of the software solution developed are described along the parts of the digital tool. Moreover, throughout the content analysis, the “teach-in” phase arouses inductively across these themes and is related to the use of the digital tool in general. Further, all of these topics are described in detail. Anchor quotes were selected from the analysis to illustrate the results. To make them accessible to international readers, they have been translated from German to English. The abbreviations “RN” denote statements from registered nurses, “CG” from caregivers, and “R” from relatives.

E-learning platform

Caregivers pointed out, that courses in the language of origin improved their German language skills. Interviewed caregivers especially emphasized the fact, that courses can be attended repeatedly, which supports knowledge expansion for current topics raised in the 24-hour care setting: “You can repeat it if you have forgotten something. That way you can refresh your knowledge. To me this is a great possibility.” (FG, CG: 500–502). Interviewed nurses reported that it is noticeable that the learning contents are already being put into practice by 24-hour caregivers. The learning contents are stated as suitable for the needs of the daily work for 24-hour home care. Further topics came upon, such as specific care techniques, knowledge of confidentiality regarding personal data, and data privacy in general. These themes can be considered in the further development of the e-learning content. However, not all care-related topics can be learned through an online format. There was consensus across the target groups, that issues such as positioning, mobilization, and transfer of a bedridden person have to be trained in person: “You can’t replace everything with e-learning. How do I deal with a patient who is becoming or has become physically sluggish in terms of mobilization, that requires personal training.” The same applies to emergency training (see emergency management).

E-documentation

Regarding the e-documentation solution, interviewees pointed out that, after getting used to handling the technical tool, the application is appropriate to document in higher detail and saves time as well as other resources like paper. ”At the beginning, it was a bit more to do, what do you have to fill in and what not […]. But now it’s going well, it’s quite easy.” (FG, CG: 120–122). RN mentioned that during their care rounds, time is too short to support 24-hour caregivers to train on the documentation and how to use the tablet. They suggest regular online meetings with the caregivers for training, exchange of experience, and networking among each other. Moreover, RN emphasized the fact that the e-documentation has better legibility, and they expected less effort for daily documentation once they got used to it. Overall, the participants stated, that the developed e-documentation is an innovative tool that stores all important information in one safe place. In particular, for RN, it was important that the digital documentation provides remote access, is compatible with ELGA (Electronic Health Records, 2022) and has interfaces to attending practitioners. This allows nurses to use the documentation to define long-term care benefit levels and quality management: “…if an assessment for care benefit classification is necessary it can be printed out and sent to the relevant offices. I think it’s easier for all of us. It benefits not only the care workers but also the client.” (GD, RN: 348–354). A fact that was confirmed in the interviews with the relatives as well. Within these discussed topics, data security was an issue that arose frequently across all target groups.

Emergency management

Emergency management turned out not to be used over the time of the study. Participants of the usability walkthrough procedure could not see the advantages of the emergency management system in comparison to a phone call. Interviewed relatives emphasized the fact, that besides emergency numbers, other important contacts (e.g. household ailments) can be stored there: “ … it is good, that these emergency numbers are stored there, it can be extended to include the plumber if the washing machine is leaking so that you can also call private emergency numbers.”(R1, GI: 814–819) As mentioned above, especially interviewed relatives pointed out, that emergency training cannot be replaced by a digital tool.

Communication platform

The provided secure communication platform “Signal” was not used during the trial. Participants stuck to the communication structures they were familiar with, such as WhatsApp or phone: “They talk to each other on the phone […]. I think that networking with each other via the tablet doesn’t happen.” (R1, GI: 744–745) Some of them did not even know about the platform. Because caregivers used the same tablet, applications that required a password, such as a login to the personal Facebook account, were unattractive to use over the tablet. However, 29 caregivers joined the moderated Facebook group and used it mainly passively to consume content.

“Teach-in” phase

24-hour caregivers are mostly older people themselves, and working with a digital tool is not a matter of course. Therefore, awe in using the tablet and fear of doing something wrong are customary among the participating 24-hour caregivers. In addition to the training materials offered, an intensive “teach-in” phase at the beginning of using the tablet and its applications seems to be indispensable: “I think that more time needs to be taken when handing over the tablet. The initial training should be more intensive, with repetitions at short intervals, so that initial fear can be taken away. This must be continued until it becomes a matter of course” (R1, GI: 860–865). Interviewed registered nurses pointed out, that they do not have the resources to train the caregivers on using the tablet. Caregivers appreciate personal training to use the applications, preferably in their language of origin. Participants from the cognitive walkthrough in particular mentioned that the “teach-in” phase needs to go on until the tablet can be used smoothly.

Discussion

This paper describes the results of the qualitative findings from the evaluation of the usability, acceptance, performance, and impact of the software solution developed. Findings show that usability and actual use of the software solution depend on the technical competencies of the individual caregiver. Technical skills and perceived benefits influence acceptance of AAL technologies. People with low levels of technical self-efficacy are more reluctant to accept AAL technologies (van Heek, Ziefle, & Himmel, 2018). There is agreement across all target groups that data privacy is an important issue and that there must be transparency regarding the data stored.

Data storage and access for third parties are relevant aspects and potential barriers to AAL technology acceptance (Offermann-van Heek & Ziefle, 2019). The developed e-learning content was stated helpful for everyday life in 24-hour care. The content is suitable for the predominant care settings and available as text and subtitled videos in different languages. Provided courses can be repeated as often as needed.

Learning a language via subtitled videos can help learners to expand their vocabulary over multisensory channels (Harji, Woods, & Alavi, 2010). Therefore, the e-learning platform supports 24-hour caregivers in their German language skills. Moreover, the possibility of repeating the courses gives caregivers the opportunity to continue their education at any time and revisit topics when new questions about the care situation arise. Limitations of e-learning became apparent regarding the course contents to be offered. Topics such as positioning an immobile person and acting in an emergency can only be adequately practiced through face-to-face training.

Digital care documentation can support carers with standardized documentation and guide them through the planning of interventions (Röhm, 2018). At the time of writing, structured documentation is uncommon among 24-hour caregivers (Haslinger-Baumann, Lilgenau, Binder, & Gugenberger, 2019). With the distributed e-documentation, participating 24-hour caregivers documented their work in a very detailed and organized way, several of them for the first time in this extensive form. Therefore, it must be considered that such detailed documentation was new for many caregivers. Even if they have already documented on paper before, using a digital tool is unfamiliar.

The adoption of technology requires learning new skills and new ways to perform familiar tasks (Brosnan, 1999). This fact was also mentioned in the interview with the RN: ”They are doing the documentation by writing it down on paper. If they have time, they work with the tablet and move on to electronic documentation. They’ve been familiar with writing on paper for years since and are well versed in it. They can do it half asleep, they write it fast so that there is anything for the nurse” (RN, FG: 206–210). 24-hour caregivers are often older people, mostly women.

This target group has a higher level of computer anxiety (Lee, Czaja, & Sharit, 2009). Reducing anxiety leads to rating the digital tool as more useful (Brosnan, 1999). Older people need access to technology training programs, designed to their learning limitations. They prefer group formats to learn and share experiences with others (Lee, Czaja, & Sharit, 2009), a relaxed and supportive environment, sufficient time, and not feeling rushed (Fisk, Charness, Czaja, Rogers, & Sharit, 2004). The interviewed RN also made these suggestions: “But it’s about getting in touch with them. A way to ask questions and knowing that I’m not alone” (RN, FG:549–550).”Would be good to give the caregivers the opportunity to make an appointment every fortnight, three weeks, and that everyone who is interested and who is working with the tablet, can join to exchange how they are doing. That is more effective…” (RN, FG: 519–521). Encouraging feedback during training leads to a sense of success for the learners (Brosnan, 1999). Therefore, caregivers need enough time before having to use the e-documentation as well as sufficient training, as one interviewed caregiver gets to the point: “I can imagine working with it, but I will need a lot of help and training, especially at the beginning” (CG 8, UW).

For the implementation of digital tools, it is helpful to know the basic attitude of users and to build trust in the new technology (Lohmann et al., 2021). The results give an insight how participants handle the digital tool within their work and what their view on the potential benefits of the digital application is. This marks an important factor for acceptance and future use. The fact that all information needed in terms of 24-hour home care is included in one digital tool was highly appreciated by the participants. For a comprehensive use of “24hQuAALity”in practice, a longer training process is necessary. Once safe handling by the users is ensured, the advantages of the developed software solution can take full effect. Thus, the developed app can make a major contribution to a better quality 24-hour care.

Limitations

There are some limitations concerning the sample and the interview settings. Certain groups, such as the one with relatives, were too small to break down into a basic population. The participants of the UW all had a similar cultural background. Therefore, statements are limited to that fact. Participants in focus group discussion may have known each other. This could have led to an inhibition threshold for critical statements or uncertainties.

Participants of the primary target group could not be integrated, as they could not be reached due to the pandemic situation. Hence, their point of view is unrepresented, however to a certain extent constituted by the statements of their relatives. Participants in the RN focus group have only worked with “24hQuAALity” for a short time and therefore were not yet fully familiar with it. A possible language barrier arises for participants for whom the interviews were not conducted in their native language. Therefore, questions and statements may have been misunderstood. Overall, the pandemic situation may have influenced the results, as all people involved in the care setting were working under burdening conditions and experiencing higher stress than usual.

Contributorship

All authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by EKK, CH, FW, and EHB. The first draft of the manuscript was written by EKK and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Evangelical Hospital Vienna (REC number: 01/2019/No 012019)

Guarantor

EKK

Acknowledgments

We would like to thank all participating caregivers, clients, and their relatives as well as the registered nurses for their trust they have placed in us and their time. It was only through the contribution of their expertise to the study that it was possible to develop the “24hQuAALity” app and generate the present results.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The research project was funded by the Austrian Research Promotion Agency [grant number: 868222].