Formative and Validation Human Factors studies of a new disposable autoinjector for subcutaneous delivery of chronic disease therapies

ABSTRACT

Background

Use-related risks related to autoinjector devices have been previously identified. To minimize these problems, the identification of potential use errors is a critical task during device development.

Methods

This article presents iterative human factor studies, which aim to assess user interaction with the tested push-on-skin BD Intevia™ 1 mL Disposable Autoinjector, across a wide range of indications, and a broad user population.

Results

Through the different human factor studies, use errors were recorded when the participants completed the critical tasks, but their occurrence continuously decreased. First, the incidence of use errors was reduced when the participants read the IFU. In addition, the IFU updates and design change implemented contributed to improve the usability performance. During the validation study, some use errors were still observed, mainly during the first uses. Nevertheless, providing a training to the participants almost fully eliminated the remaining use errors.

Conclusion

Thus, these results demonstrated that this new autoinjector can be safely and efficiently used for its intended uses and under the expected use conditions by all tested user groups.

KEYWORDS:

• Autoinjector
• push-on-skin
• single-use
• human factors engineering
• usability
• simulated testing
• subcutaneous injection
• validation study

1. Introduction

A number of treatments for chronic inflammatory diseases are delivered subcutaneously via prefilled syringes or auto-injection devices [1–4]. Contrary to intravenous infusions, subcutaneous injections provide the convenience of self-administration, which enhances patient experience, decreases treatment burden, and lowers costs for both patients and healthcare systems [5–12].

However, several technological barriers remain that must be resolved to provide drug delivery devices for self-injecting patients with dexterity issues. Hand function is often impacted by chronic autoimmune diseases, such as rheumatoid arthritis and multiple sclerosis. Therefore, manufacturers have developed novel drug delivery devices to promote easy and safe self-injections for patients diagnosed with these diseases, as well as supporting caregivers and health-care professionals. Safe, effective, easy to use, and satisfactory autoinjectors have been developed since the 1990s to improve injection experience, and thereby improve patient adherence to therapy [13–18]. Human Factor evaluations are thus essential to inform and assess the design of devices throughout the entire design and development process to ensure effective, efficient, and safe use by minimizing potential use errors and resulting harm [5,19–26].

By allowing patients to self-inject, the use of autoinjectors offers considerable advantages over manual subcutaneous injections, which include increased patient adherence and compliance with the treatment of rheumatoid arthritis and multiple sclerosis [13–18]. To further improve the user’s experience with autoinjectors, push-on-skin autoinjectors have been developed to be more convenient since they fewer user steps than fully automated button-activated devices [27]. However, safety issues related to the use of autoinjectors have been reported during different human factor studies and via public information. The most commonly reported use errors include holding autoinjectors upside-down or failing to hold the device in place after injection [21,28–31].

In this context, iterative human factor processes have been conducted during the development of a new single-use and push-on-skin autoinjector to reduce the occurrence of use errors as much as possible (Figure 1).

Figure 1. BD Intevia™ 1 mL push on-skin and single-use Disposable Autoinjector before use (left) and after use (right)

This article aims to describe the iterative Human Factor studies that have been evaluated the target user’s understanding of this autoinjector’s use, and the reduction of use error occurrence to mitigate potential residual risks to an acceptable level for health-care professionals, caregivers, and patients.

2. Methods

2.1. Study autoinjector

Participants simulated injections using the BD Intevia™ 1 mL Disposable Autoinjector, which contained a BD Neopak™ 1 mL Glass Pre-fillable Syringe long staked needle (29 G needle diameter; ½ inch) (Figure 1). The syringes were filled with either water for injection or an aqueous solution of polyethylene glycol of varying viscosities (10 cP, 20 cP or 30 cP solutions).

This device is intended to deliver up to 1 mL of medication into the subcutaneous tissue. To correctly use the autoinjector users have to remove the cap position it perpendicularly onto the skin and firmly press down on the device to start the injection, which is indicated by a first click. The user is informed on when to remove the device from the skin by interpreting the end of dose indicators (audible, visible, and tactile) prior to discarding it into a sharp container. As illustrated in Figure 2, in addition to standard audible (clicks at the beginning and end of injection) and visual indicators (visible plunger rod inside the drug viewing window), a visible and tactile feedback has been incorporated in the top of the autoinjector to indicate the end of injection (three light blue features visible from different angles rise and trigger a vibration under the thumb).

Figure 2. Visual, audible and tactile feedback indicators of BD Intevia™ 1 mL Disposable Autoinjector. The drug viewing window is empty before the activation of the device (a). Once the injection has started, indicated by a first click when pushing down the needle cover (b), the blue indicator progressively appears in the viewing window (c). The end of injection is signaled by three indicators, audible, visual and tactile at the top of the device (d). After the complete dose delivery, the drug viewing window appears fully blue (e)

2.2. Human factors procedure

In compliance with United States Food and Drug Administration (FDA) Guidance (2016) [5], and AAMI/ANSI-HE-75:2009 Part 9.3.2/Table 9.2., a minimum of 5 or 15 participants per user group were recruited for the formative and the validation studies, respectively. In each study, the participants were recruited in different countries/cities. In addition, they were eligible if they were not enrolled in a study conducted to assess the use of an injection device within the year preceding the study. Prior to completing any human factor activities, all participants signed an informed consent. For the studies conducted in the US (Human Factors Formative study 2 and Human Factor Validation study), an approval by an Institutional Review Board was obtained. These studies were considered as non-significant risk studies where no therapeutic intervention was performed.

These human factor studies included Health-Care Professionals (HCP), caregivers, and self-injecting patients. Two years of experience working with syringes or autoinjector devices were required for Health-Care Professionals. Patients and caregivers were distinguished by two sub-populations, naïve participants who have never used an autoinjector, and experienced ones who have used push-on-skin or button-activated autoinjector devices for at least 10 injections or for more than one year. Some of the patients were diagnosed with Rheumatoid Arthritis or Multiple Sclerosis which affect their finger and/or hand dexterity. Their hand impairment was characterized by a Cochin score at the recruiting stage for all studies and additionally on the day of testing for the validation study [32,33].

An iterative human factors process consisting of three main formative studies and one validation study is presented in this article. The first and third Human Factors formative studies were conducted in France whereas the second formative and the validation studies were conducted in the US. During each study, single one-on-one testing sessions were performed in facilities which were customized to mimic a home environment. During the formative and validation studies, foam pads, into which participants simulated injections, were directly attached to the self-injecting patient’s abdomen or thigh or were placed on mannequins for caregivers and health-care professionals. Design of the presented Human Factors studies is illustrated in Figure 3.

Figure 3. Summary of formative and validation study design. Abbreviations: HCP, Health Care Professionals; IFU, Instructions For Use

The first Human Factors formative study aimed to assess the minimum and maximum acceptable forces required to start the injection as well as the acceptance of the holding time required to maintain the autoinjector on the pad using different forces.

During the second and third Human Factors formative studies, the usability of this autoinjector was evaluated among the targeted end-users and naïve participants, respectively (Figure 3). In the second Human Factors formative study, the population was divided into two subgroups, in one the participants did not have access to the IFU and in the second one the participants read the IFU (Figure 3). This designed aimed to assess the intuitiveness of the device or the impact of the IFU on the usability. During the third Human Factors formative study, all participants had access to the IFU and were free to read it or not, as in real-life conditions (Figure 3).

The usability of the final design of the autoinjector and the final Instructions For Use (IFU) developed by BD were evaluated during the Human Factor validation study (Figure 3). The study population was divided into two arms: a trained one and an untrained one, to assess the impact on such training on the usability performance. The untrained participants had access to the IFU and were free to read it or not, as in real-life conditions. For the trained participants, a 45-minute training was provided by one independent trainer with a rigorous script to limit study bias. During the training session, the trainer provided a short overview of the autoinjector and the steps required to perform an injection. The participants were then asked to fully read the IFU, ask the trainer any questions they might have, and finally use the autoinjector to perform one simulated injection into a pad placed on a table in front of them. A training decay period of one day was required between the training and usability assessment.

2.3. Usability assessment

During all the formative and validation studies and for each simulated injection, a moderator observed and recorded any use errors, success with operational difficulties, close calls and successes, if use was according to the IFU, and the performance of the critical tasks (Table 1) related to use of the autoinjector. As per FDA Guidance, the critical task is defined as a task that, if performed incorrectly or not performed at all, would or could cause serious harm [5]. Risk analysis approaches, such as failure modes effects analysis (FMEA) was the tool used for the identification of the critical tasks. A use error was defined as an action (or lack of action) that leads to a result that is not intended or expected by the user. A success with operational difficulties was defined as a case where the user struggles to complete the intended action but was ultimately resolved. A close call was defined as a use error that was recovered by the user (avoiding harm which would not have occurred) or a use error nearly occurred. A task could be marked as ‘Not Applicable’ in the case of prior use error which prevented the user from performing the task. The tasks required to use of the autoinjector which could or would compromise medical care were considered critical, as defined per the FDA Guidance are outlined in Table 1 [5]. Usability data are described as categorical data (N) and proportion frequencies (%). Following the simulated uses, the moderator debriefed all use errors, close calls, and operational difficulties encountered during the session to identify the related root causes. At this stage, all comments expressed by the participants, both positive and negative, were recorded into databases.

Table 1. List of essential and critical tasks performed by users during an injection. Critical tasks are highlighted in orange

The usability evaluation with the assessment of the remaining use-related risks was used to validate the effectiveness of the IFU and training.

2.4. Usability perception

2.4.1. Acceptance of forces to activate and to maintain the device in place (Human Factors (early) formative study 1)

The participants from study 1 simulated two injections using seven prototypes with different forces of activation: 5 N, 7.5 N, 10 N, 15 N, 20 N, 25 N, and 30 N. First, each of the seven prototypes were presented to the participants in a randomized order. For the second simulation, the prototypes were given to the participants according to an increasing order of activation force (from 5 N to 30 N). A total of 14 trials, two for each force, were performed by the participants.

Three prototypes with different forces to maintain the device in place for 15 seconds were used by the participants to simulate injection: 3.2 N, 5.8 N, 7.8 N. First, each of the three prototypes were evaluated in a randomized order and then in an increasing order of maintaining force (from 3.2 N to 7.8 N). A total of six trials, two for each force, were performed by the participants.

For each prototype, the participants evaluated the acceptance of the force used to activate the device and the force needed to maintain the device in place for 15 seconds using Likert scales, ranging from ‘not acceptable at all’ to ‘very acceptable.’

2.4.2. IFU and training (human factors validation study)

After the usability assessment phase, all the participants were asked to read the IFU. Subjective feedbacks of both IFU and training understandability were evaluated and recorded using a questionnaire with a Likert scale. The 86 participants rated how easy or difficult it was to understand the instructions on how to use the autoinjector. Similarly, the 16 trained participants evaluated the training. The following questions were asked to the participants ‘how easy or difficult is it to understand the instructions/training on how to use the auto-injector?.’ Both Likert scales were defined from ‘very difficult to understand’ to ‘very easy to understand.’

2.4.3. Ease of use and acceptance (human factors validation study)

The overall ease of use, the ease of activation and the ease of handling the tested product was evaluated by the 86 participants of the validation study by asking three questions as follows: ‘how easy or difficult it is to use the auto-injector?’; ‘how easy or difficult was it to push to activate?’; ‘how easy or difficult was it for you to handle the autoinjector?.’ Answers were recorded via Likert scales after the usability assessment phase. Likewise, the acceptance toward the use of the autoinjector was evaluated, with the question ‘how will you rate your experience of using this auto-injector?.’ Likert scales were defined from ‘very difficult/very unacceptable’ to ‘very easy/very acceptable.’

2.4.4. Confidence on product use (human factors validation study)

To assess the confidence of using the tested autoinjector the following question was asked to the participants: ‘how confident or not confident are you in your ability to use the auto-injector?.’ All answers were recorded through a Likert scale from ‘not confident at all’ to ‘very confident’ after the usability assessment phase.

2.4.5. Usability perception on feedback indicators (human factors validation study)

The participants’ perception on the use of the autoinjector due to its feedback indicators was evaluated by asking the following question: ‘would you say that this feedback enhanced or didn’t enhance your experience using this autoinjector?’. All answers were recorded through a Likert scale from ‘not enhanced at all’ to ‘very enhanced’ after the usability assessment phase. Among the 86 participants only those having a prior experience using autoinjectors without audible click indicating the end of injection were included.

2.4.6. Injection time (human factors validation study)

The impact of the injection time on the acceptance and ease of use was assessed through Likert scales questionnaires defined from ‘very difficult/very unacceptable’ to ‘very easy/very acceptable.’ At the end of the validation study a total of four viscosities (1 cP, 10 cP, 20 cP, 30 cP) were tested to assess different injection times (~2.5 sec, ~4 sec, ~6 sec, ~9.5 sec) by the nine participants enrolled in the Sharps injury prevention feature study. The ease of use and acceptance were assessed after the simulations performed with each viscosity configuration with the following questions: ‘was the device in this configuration easy or difficult to use?’ and ‘was the overall time required to use the device acceptable or unacceptable?.’

2.4.7. Control point feature (human factors validation study)

The question ‘how confident or not confident are you regarding the fact that you can control the start of the injection?’ was asked to the participants to evaluate their confidence of controlling the start of injection when pressing the product on the pad with to the control point feature. All answers were recorded through a Likert scale questionnaire, defined from ‘very confident’ to ‘Not confident at all.’

2.5. Sharps injury prevention feature

To meet the criteria set up by the ISO:23,908 standard and the FDA guidance on sharps injury prevention feature of zero failures out of 500 tested devices, nine participants performed 44 additional simulated injections on pads and under conditions that simulate the critical clinical variables (dry and wet hands, with and without gloves), to simulate clinical use of sharps injury prevention feature, and evaluate its performance (396 devices tested) [34,35]. In addition to these 396 tested devices, the third and fourth devices used by the 86 participants during the usability assessment of the validation study were kept before discarding into the sharps container. The performance of the safety feature of 172 devices was then tested by moderators. Thus, a total of 568 devices were included in the evaluation of the sharps injury prevention feature.

3. Results

3.1. Patient demographics

The device under evaluation may be used by self-injecting patients, healthcare professionals, or lay caregivers who give injections to patient in the context of chronic disease therapies. As the autoinjector will be employed across different medical indications, a broad user population has been considered, including users with some impairments. Therefore, results recorded for the Human Factor studies described herein reflect as much as possible differences in the abilities of potential end users.

Summary of participants’ characteristics involved in formatives and validation studies is presented in Table 2.

Table 2. Demographic characteristics of participants. HCP: Health-Care Professionals

	Formativestudy #1		Formative study #2		Formativestudy #3		Validationstudy
Characteristic	Patients (N = 26)	Caregivers(N = 8)	Patients (N = 22)	Caregivers(N = 12)	Patients (N = 15)	Patients (N = 50)	Caregivers(N = 20)	HCPs (N = 16)
Female, N (%)	19 (73%)	5 (62.5%)	17 (76%)	6 (53%)	7 (47%)	36 (72%)	16 (53%)	15 (94%)
Age (years),mean [min – max]	47.3 [19–70]	44 [25–66]	47.3 [19–70]	55.3 [21–80]	45 [19–65]	54 [26–78]	49 [27–71]	39 [23–57]
Dominant hand, N (%) Left Right Both	6 (23%) 19(73%) 1 (4%)	7 (87.5%) 1 (12.5%) -	1 (5%) 44 (86%) 2 (9%)	11 (92%) 1 (8%) -	2 (13%) 13 (87%)-	5 (10%) 45 (90%)-	1 (5%) 17 (85%) 2 (10%)	0 (0%) 15 (94%) 1 (6%)
Hand impairment, N (%)	17 (65%)	-	4 (36%)	-	1 (7%)	13 (26%)	-	-
Sub-group, N (%): Training IFU	--	--	-11 (50%)	-6 (50%)	15 (100%)	12 (24%) 50 (100%)	4 (20%) 20 (100%)	-16 (100%)
Prior experienced, N (%)	13 (50%)	0 (0%)	14 (64%)	0 (0%)	0 (0%)	24 (48%)	9 (45%)	11 (69%)

3.2. Human factors (early) formative study 1

The first formative study was intended to assess different forces to activate and to maintain the device in place for 15 seconds to provide input specifications for the final design of the autoinjector.

During the first formative study, all participants were able to activate the prototypes whatever the activation forces including patients with hand impairments. However, the most acceptable forces (‘acceptable’ and ‘very acceptable’ ratings) to activate the autoinjector were 5 N, 7.5 N, and 10 N, respectively for 62% (N = 21/34), 68% (N = 23/34) and 76% (N = 26/34) of participants, regardless of their hand dexterity (Figure 4). Regardless of the user population, the activation forces of 25 N and 30 N were the less satisfactory with only 15% (N = 5/34) and 6% (N = 2/34) of acceptable scores (‘acceptable’ and ‘very acceptable’ ratings) and 29% (N = 10/34) and 41% (N = 14/34) of not acceptable scores (‘not acceptable at all’ and ‘not acceptable’ ratings), respectively (Figure 4).

Figure 4. Activation forces acceptance when using BD Intevia™ 1 mL Disposable Autoinjector during the formative study 1. Reported acceptance toward different tested forces to activate the BD Intevia™ 1 mL Disposable Autoinjector during the formative study per participant (N total = 34). N, number of participants who rated activation force as ‘not acceptable,’ ‘neutral opinion’ or ‘acceptable.’ Note: The ‘very acceptable’ and ‘acceptable’ ratings are the two highest ratings on the Likert scale

The ‘very acceptable’ and ‘acceptable’ ratings are the two highest ratings on the Likert scale.

In addition, all participants succeeded in holding the autoinjector prototypes on the pad for 15 seconds regardless of the force (from 3.2 N to 7.8 N). Better acceptance for forces below 5.8 N was perceived by the participants. Eighty-eight percent of acceptable scores were recorded for a force of 3.2 N while 79% and only 35% of the participants rated acceptable the forces of 5.8 N and 7.8 N, respectively. These results provided user-centric specifications, including disease-specific limitations, during the autoinjector development.

3.3. Human Factors formative study 2

Once design input specifications toward the forces required to start the injection and to maintain the autoinjector on a pad have been defined, a second formative study was conducted to assess the usability and identify potential risk of the autoinjector during the early development process.

Among the 816 simulated injections performed by the participants, a total of 156 use errors were observed during the evaluation of the six critical tasks (19%). Participants experienced the most use errors during the first simulated injection, when no participants had access to the IFU (Figure 5a). Before the second simulated injection half of the participants were asked to read the IFU (user group ‘IFU+’) while the other half continued without any instructions (user group ‘IFU-’). During the second to the fourth use, the participants who read the IFU encountered 22 use errors (7%; N = 22/306) while in the group of participants who did not have access to the IFU 20% of use errors were still recorded (N = 61/306). From the first to the fourth simulated injections without the IFU the rate of success was significatively increased from 45% (N = 70/154) to 88% (N = 172/196), suggesting a potential learning effect. In addition, the increased success in performing critical tasks was even more pronounced when participants read the IFU. Once the participants have access to the IFU more than 88% of these critical tasks were correctly performed (88% (N = 88/100); 91% (N = 91/100) and 92% (N = 94/102) for simulated injections 2, 3 and 4, respectively).

Figure 5. Usability of BD Intevia™ 1 mL Disposable Autoinjector during the early and late formative studies. a. Overall critical tasks completion rates per simulation for 34 participants of the formative study 2 performing six critical tasks and having read the IFU or not (N total of evaluated tasks per IFU group = 102). b. Overall critical tasks completion rates per simulation for 15 naïve self-injecting patients of the formative study 3 performing six critical tasks (N total of evaluated tasks = 90).

Rates of success, success with operational difficulties (SOD), close call and use error are represented. N, number of critical tasks performed.

*No participant read the IFU before the first simulation.

** NA for Not applicable was used for the tasks not performed because a blocking event on previous task

Focusing on each critical task, this study highlighted some residual risks related to the recorded use errors.

3.3.1. Task ‘remove the blue cap by pulling it straight off’

During the first use and without the IFU participants failed to remove the cap in 47% (N = 16/34) of the simulated injections since they assumed it as an activation button. While participants who did not receive the IFU still encountered similar use errors during the second and third simulated injections (12% and 18%, respectively), no use error was observed after IFU reading. These use errors were caused by a misunderstanding on how or when to remove the cap, and not due to physical disability. Therefore, the risk of delay in treatment would be very limited.

3.3.2. Task ‘place the white needle cover of the autoinjector perpendicularly on the skin’

During the first use, when no participant read the IFU, holding the device upside down was observed in 31%. In the group of participants who did not have access to the IFU during the entire simulated-use session, one misorientation event was still observed during the second simulated-use. However, all users who were asked to read the IFU before the second simulation successfully positioned the autoinjector without exposing themselves to risk of needlesticks. These results demonstrated that providing the IFU to users was efficient in helping them to correctly position the autoinjector.

3.3.3. Task ‘press firmly the autoinjector down on the skin to start the injection’

The injection was triggered correctly in 94% of the simulated injections. The only observed use error was encountered during the first simulated injection. The participant thought that the cap was an activation button due to a negative transfer from their previous experience with a pen device.

3.3.4. Task ‘continue to hold the autoinjector’

During the first use and without the IFU participants prematurely removed the device in 69% (N = 18/26) of cases. The risk of underdosing due to premature removal dropped to 18% (N = 3/17) at second use and 0% during the last simulated injection after reading the IFU, while 40% (N = 6/15) to 19% (N = 3/16) of use errors were still recorded during the second and fourth simulated injections when the participants did not read the IFU, respectively.

3.3.5. Task ‘continue to hold the autoinjector against the skin for a few more seconds after the « CLICK 2 »’

At first use and without the IFU, the device was prematurely removed in 33% of the simulation. The instructions on the IFU appeared to be not clear enough to fully prevent the risk of underdosing, which might be limited when removing the device immediately after the second click, since 6% (N = 1/17) and 15% (N = 2/13) of use errors were encountered by the group of participants with and without the IFU, respectively.

3.3.6. Task ‘dispose the used autoinjector in an appropriate sharps disposal container after use’

Finally, after the first simulated injection, 50% (N = 51/102) of the autoinjectors were not correctly disposed of in a sharps container because they thought the device should be disposed of in the trash. These types of errors still occurred in 19% of cases during the second to fourth simulated injections for the group of users who did not have access to the IFU (N = 9/48). Providing the IFU was enough to help users in correctly disposing the autoinjector in the appropriate sharps container to prevent accidental needle stick injuries since users who read the IFU correctly dispose the autoinjector without use error.

Results of this formative study demonstrated that without the IFU and mainly during the first use a substantial rate of use errors, which could result in needle stick injuries or underdosing, were observed. To mitigate the remaining use errors some recommendations were suggested related to design (i) increasing the contrast between the body (gray) of the autoinjector and the needle cover (white), or instructions (ii) adding/clarifying the tested IFU in regards to the holding time and the indicators of injection (Table 3).

Table 3. Identified hazards and their mitigation during development of the BD Intevia™ 1 mL Disposable Autoinjector for subcutaneous administration

Finding	Changes implemented during autoinjector development
	Formative #2	Formative #3	Validation
‘Place the white needle cover perpendicularly on the skin’: Risk: accidental needlestick	Initial design: white autoinjector’s body and needle cover	Darker grey color of the autoinjector to contrast with needle cover	Design changes from Formative #3 were kept
‘Continue to hold the autoinjector’: Risk: underdosing	Initial drawing	Holding indication added in the IFU	Holding time added in the IFU
	Initial drawing	Indications of each feedback signaling the end of injection in the IFU	Indications of each feedback signaling the end of injection in the IFU
‘Continue to hold the autoinjector against the skin for a few more seconds after the ‘CLICK 2’’: Risk: underdosing		Step of holding the autoinjector after the 2nd CLICK added	Time to wait indicated in the IFU

3.4. Human factors (late) formative study 3

A new formative study was conducted to assess the impact of the mitigations implemented based on the recommendations raised during previous formative studies on the usability of the BD Intevia™ 1 mL Disposable Autoinjector. As the identified use-related risks during prior studies were limited and mainly faced by naïve patients only this user group was included in this formative study (Figure 5b). The IFU was provided with the devices, and the participants were free to read it or not.

A total of 32 use errors were recorded among the 60 simulated injections during the evaluation of the six critical tasks (9%; N = 32/360) (Figure 5b). Based on the overall usability results we observed that the rate of success was improved across the four simulated injections, from 68% (N = 61/90) to 90% (N = 81/90), with more than 80% of successes since the second simulation (Figure 5b).

3.4.1. Task ‘Remove the Blue Cap by pulling it straight off’

The patients successfully removed the cap in 78% of the simulated injections (N = 47/60). Misunderstanding on how to remove the cap of the autoinjector resulted in two errors and seven successes with operational difficulties (no removal of the cap to simulate the injection or recapping the device) for the 15 naïve patients during the first simulated injection. Although the IFU was provided in the packaging some of the participants started to perform the simulation before reading the IFU. However, for the seven users who faced difficulties they had to read it to succeed in removing the cap. Four additional successes with operational difficulties were recorded. Three occurred because a patient twisted the cap a little to remove it instead of pulling it straightforward. One patient succeeded but found difficult to remove the cap during the second simulated injection. However, this participant correctly understood this task and did not raise a similar concern for the other simulated injections.

3.4.2. Task ‘Place the white needle cover of the autoinjector perpendicularly on the skin’

All participants correctly positioned the device. One patient hesitated on the orientation of the autoinjector and two of them nearly misoriented it but they self-corrected by reading the IFU. Therefore, participants did not experience needlestick and possible injection in the wrong tissue.

3.4.3. Task ‘Press firmly down on the skin to start the injection’

One hundred percent of participants succeeded in starting the injection. Only one success with operational difficulties during one simulated injection occurred because the participant, a right-handed user, had difficulties to trigger the injection since he placed the pad on his left abdomen (N = 1/60).

3.4.4. Task ‘Continue to hold the autoinjector’

Three participants prematurely removed the autoinjector from the injection site during the first (N = 2/14) and second (N = 1/14) uses (one patient did not complete the task due to a previous error). After checking the IFU no more use errors were recorded.

3.4.5 Task ‘Continue to hold the autoinjector against the skin for a few more seconds after the « CLICK 2 »’

Many of the use errors recorded during this study occurred because participants directly removed the device after the second click (N = 9/55; five tasks were not performed due to errors that occurred on previous tasks). These use errors were caused by participants who did not understand they had to wait a few seconds after the « CLICK 2 » as they did not, or not carefully, read the corresponding IFU section (N = 6) or by a participant with hand impairment and loss of feeling in fingers who was surprised by the triggering of the top indicators (N = 3).

3.4.5. Task ‘Dispose the used autoinjector in an appropriate sharps disposal container after use’

Although the design of the autoinjector protects the user by a locked needle cover once the injection is completed, 17 use errors were recorded among the 60 simulated injections because the participants did not read the instructions related to the disposal and thus recapped the evaluated autoinjector prior to disposal.

In summary, most of the difficulties were encountered during the first simulated injections and before the participants read the IFU. Between the first and fourth simulated injection, the rate of use errors was reduced from 12% to 7% (Figure 5b). Interestingly, the risk to use the autoinjector upside-down, which could lead to an adverse event (needle stick injury and possible injection in the wrong tissue), was found to be limited in this study as the participants were able to correct themselves by self-referring to the IFU in comparison to the previous study.

Besides the changes implemented between the second and third formative studies, some additional information was incorporated in the IFU for the validation study to mitigate the risk of premature removal (Table 3): the estimated injection time (i), the nature of the feedback indicators (audible, visible, and tactile) (ii), and the approximate holding time required after the « CLICK 2 » which indicates the end of the dose delivery (iii).

3.5. Human factors validation study

3.5.1. Usability evaluation

Final design of the autoinjector and its IFU were evaluated in terms of usability and user’s experience during the validation study across the targeted end-user populations (Table 2).

During the validation study 89 use errors were recorded across the 344 injections performed by the 86 study participants for the six evaluated critical tasks (Figure 6a). Most of the use errors recorded occurred during the tasks ‘Hold the autoinjector firmly against the skin until the end of injection’ (N = 21/344, 6% of use errors) and ‘Continue to hold the autoinjector against the skin for a few more seconds after the « CLICK 2 »’ (N = 36/322, 11% of use errors) (Figure 6a and 6b).

Figure 6. Usability of BD Intevia™ 1 mL Disposable Autoinjector during the validation study.

a. Overall injection success, success with difficulties (SOD), close call, and use error rates across the four simulated injections in the untrained and trained arms during the validation study. The six critical tasks were performed by 70 untrained and 16 trained participants. Two untrained participants performed an additional simulated injection due to product replacement (N total of evaluated tasks = 282).

b. Success, success with difficulties (SOD), close call, and use error rates per critical tasks across the four simulated injections during the validation study for 16 trained participants and 70 untrained participants performing six critical tasks. Two untrained participants performed an additional simulated injection due to product replacement (N total of evaluated tasks = 64 for the trained arm and N total of evaluated tasks = 282 for the untrained arm).

N, number of critical tasks performed.

* NA for Not applicable was used for the tasks not performed because a blocking event on previous task.

** Task 11: 172 tasks for simulated injections 3 and 4 (32 tasks from the trained group and 140 tasks from the untrained group) were not evaluated and rated NA. These 172 devices were included in the Sharps Injury Prevention Feature study

A prior training improved the usability performance. The occurrence of use errors was strongly reduced in the trained arm (N = 2 use errors over 64 injections; 3%) over the untrained arm (N = 87 use errors over 282 injections; 5%) (Figure 6a).

The usability results for each critical task were analyzed for the untrained arm (Figure 6b and Supplemental Figure S1 a-f).

3.5.2. Task ‘Remove the Blue Cap by pulling it straight off’

Without training five participants directly checked the IFU and corrected themselves to succeed in removing the cap (close calls 2%; N = 5/282) and eight participants (caregivers and patients) either attempted to recap the autoinjector or to inject without fully removing the cap during the first use, but they did not reproduce these use errors once they carefully read the IFU for the following simulating injections (use errors, 3%; N = 8/282) (Figure S1a). While patients suffering from rheumatoid arthritis or multiple sclerosis did not face use errors due to their hand impairments, three of them had some difficulties to remove the cap at first use only (2%; N = 5/282) (Figure S1a).

3.5.3. Task ‘place the white needle cover of the autoinjector perpendicularly on the skin’

The untrained participants encountered seven use errors (N = 7/281, 2%) when placing the autoinjector to trigger the injection, of these errors three occurred at first use (Figure 6b). These participants were more likely to misorient the device during the first use (N = 3/70; 4% of use errors) than during the fourth use (N = 1/70; 1% of use errors) (Figure S1b). For these seven use errors, the moderator intervened prior any potential activation upside-down and no adverse event occurred. Among the three users (caregiver, patients) who incorrectly placed the device, one patient repeated the errors during the third and fourth simulation and one patient inverted the device during the second and third uses due to nervousness, and did not encounter difficulties for the other two simulated injections.

3.5.4. Task ‘press firmly down on the skin to start the injection’

All untrained participants (N = 280/280, 100%) successfully triggered the injection without difficulties (Figure 6b and S1c).

3.5.5. Task ‘continue to hold the autoinjector’

Untrained participants prematurely removed the device before the end of injection in 21 of the simulated injections (7%; N = 21/280) (Figure 6b). However, the use errors that could result in an incomplete injection mainly occurred during the first use (N = 12 use errors), and only one use error was observed during the last simulated injection (N = 1 use error) (Figure S1d). Participants expected a shorter injection time based on their previous experience (with autoinjectors or syringes). However, focusing on this task in the IFU helped the users to succeed in holding the autoinjector until the second click and prevented the risk of underdosing.

3.5.6 Task ‘Continue to hold the autoinjector against the skin for a few more seconds after the « CLICK 2 »’

11% (N = 34/258) of use errors were observed for the untrained participants (Figure 6b). The overall rate of failure remained constant across all simulated injections for all user groups. Participants did not clearly understand or see and remember in the IFU that the autoinjector must be maintained against the skin for a few additional seconds even after the second click or they removed the device just after the second click as they would do with their previous or currently used autoinjectors (N = 3 for health-care professionals; N = 4 for naïve caregivers; N = 8 for experienced caregivers; N = 9 for naïve patients; N = 12 for experienced patients).

3.5.6. Task ‘dispose the used autoinjector in an appropriate sharps disposal container after use’

One hundred percent of health-care professionals correctly disposed of the device in a sharps container. Thirteen percent of use errors were recorded and caused by caregivers and patients who attempted to recap the device (N = 8/17) or who did not discard the device or threw it in the regular trash (N = 9/17) (Figure 6b). Nevertheless, the risk of needlestick injury associated to the incorrect performance of this task was low due to the safety feature of the device.

3.6. Impact of the training on the usability

Without training a majority of the use errors were encountered by the participants during the first uses (Figure S1a-f). To address the main remaining use errors and difficulties, a training was provided to 16 participants prior performing the simulated injections. Tenfold less use errors were committed in the trained arm (2 use errors out of 64 simulated injections) than in the untrained arm (87 use errors out of 287 simulated injections). Additionally, zero use errors were recorded in the trained arm after two simulated injections were performed with the autoinjector (Figure 6b). The only two use errors were caused by one naïve patient who prematurely removed the autoinjector after the second click. They assumed that the simulated injection was done after the first click for the two first simulated injections. Nevertheless, this participant correctly waited two seconds after the click for the two last simulated injections (6%; N = 2/64). Consequently, the risk of underdosing at this step is very limited as in the worst case, after using twice the autoinjector, all the participants were able to wait the supplemental seconds to ensure the full delivery. Moreover, with the training the participants did not encounter difficulties coded as success with operational difficulties and close calls (Figure 6b). Therefore, the training enhanced user’s safety by mitigating the risk of needlestick injuries by avoiding upside-down issues. Hence, the analysis of the usability results highlighted a strong added value of the training which improved the performance by reducing or eliminating residual risk.

3.7. IFU and training evaluation

The IFU and the training on the use of the autoinjector were rated as easy or very easy to understand by 90% (N = 77/86) and 94% (N = 15/16) of users, respectively (Figure 7).

Figure 7. Reported understanding of the IFU and training during the validation study. N, number of participants who rated the IFU (N total = 86) or the training (N total = 16) as ‘very easy to understand’ and ‘easy to understand,’ ‘neutral opinion,’ or ‘difficult to understand’ and ‘very difficult to understand.’

Note: The ‘very easy to understand’ and ‘easy to understand’ ratings are the two highest ratings on the Likert scale.

Therefore, the IFU and training were greatly understood by most of the participants and no major concern was raised, which enabled the IFU and training to be validated.

3.7.1. User feedback

3.7.1.1. Ease of use and acceptance

Overall, very positive subjective feedback was recorded for most of the participants. 98% of them perceived the tested autoinjector as easy or very easy to use (Figure 8a). Consistently, the handling of the device was positively evaluated by 82 out of 86 users (95%) (Figure 8a). Very positive feedback was recorded concerning the ease of activation, with 100% of ‘easy’ and ‘very easy’ ratings (Figure 8a). Consequently, 95% of participants found acceptable or very acceptable the use of the autoinjector, regardless their user groups, their experience, and even their potential hand disability (Figure 8a). The confidence of using the tested autoinjector was rated in a consistent way with 99% of ‘confident’ and ‘very confident’ scores (Figure 8a). Among the experienced users of autoinjectors without audible click indicating the end of injection 97% (N = 31/32) found their experience enhanced or very enhanced with the autoinjector due to its feedback indicators (Figure 8a).

Figure 8. User-feedback scores on usability questions asked at the end of the validation simulated-use test. a. Ease of use, ease of handling, ease of activating, acceptance, confidence of using and experience due to the feedback indicators when using the BD Intevia™ 1 mL Disposable Autoinjector. Ease of use (b) and acceptance (c) toward different injection time when using the BD Intevia™ 1 mL Disposable Autoinjector. d. Reported confidence in controlling the start of injection during the validation study

3.7.1.2. Injection time

The injection time did not impact the perceived ease of use for the tested autoinjector (Figure 8b). No difficulties were perceived by participants to simulate the injections with the autoinjector filled with 10, 20, or 30 cP solutions (Figure 8b). The injection time was well accepted with 94% (N = 17/18) and 89% (N = 8/9) of users that found acceptable or very acceptable the respective injection time for corresponding viscosities of 1 cP and 30cP (Figure 8c).

3.7.1.3. Control point feature

To allow users to control the start of injection using the push on skin technology rather than a button, the design of the autoinjector has incorporated a control point feature. Ninety-eight percent of the entire population (N = 84/86) was confident or very confident that they can control the start of the injection when simulating it on a pad (Figure 8d).

3.7.1.4. Sharps injury prevention features

Among the 568 tested autoinjectors zero failures of the protection feature were reported. The needle cover was correctly deployed during all simulated injections and remained locked-out for 100% of the tested devices. Consistently, no needlestick injuries or adverse events were reported. Therefore, this new autoinjector met the criteria of zero failures for both the needle cover deployment and lock-out features among the total number of products recommended by the FDA and the ISO 23908 standard [34,35].

4. Discussion

Throughout these successive studies, the rates of use errors recorded during the evaluation of the critical tasks continuously decreased. During the second formative study, the IFU improved the proportion of critical tasks successfully performed by the participants. During the second use and for the subgroup of participants read the IFU, 9% of use errors were observed while without reading the IFU the participants encountered 36% of use errors during the first simulated injection. In the following formative study, the IFU were provided to all participants and free to read or not. Some updates in the IFU were implemented as well as a design change (Table 3) and continued to make improvement in reducing the occurrence of use errors compared to the previous formative study. During the validation study, when the IFU was free to read or not, 87 use errors were observed out of the 282 simulated injections throughout the completion of the six critical tasks. However, in the study arm who received a training before the simulated testing the majority of the use errors were eliminated and only two use errors out of the 64 simulated injections throughout the completion of the critical tasks were still recorded.

In the literature, the most frequently reported use errors with the use of autoinjectors were related to a failure to remove the device’s safety cap prior to an injection, holding the device upside down, and failure to hold the device in place until the end of injection [24,36].

During the validation study, all participants, even those suffering from hand disability, succeeded to remove the cap. The only use errors reported were caused by participants attempting to recap the autoinjector. Although a slight delay could be caused by these use errors, the impact may not be significant to the patient in a chronic context. In addition, this risk was fully eliminated by a training, or at the second use.

The second known risk often attributable to push-on-skin autoinjectors is related to the misorientation of the autoinjector, by holding it upside-down [31,36]. Such use error might cause injection into the thumb of the person using the autoinjector which would prevent administrating the full dose of drug to the patient and cause harm in case of injection delivered by a caregiver or a health-care professional. The design and IFU changes reduced the occurrence of inverted use of the autoinjector between the formative studies and the validation study. During the validation study some self-injecting patients or caregivers, who did not receive the training, oriented the autoinjector upside-down during the first simulation which required assistance by the moderator to prevent from needle-stick injury, although no adverse events occurred. No use errors were recorded for health-care professionals and trained participants. Therefore, in a context of real injections, health-care providers will give information needed to inject properly that will prevent misorientation of the device.

Finally, with autoinjector devices a final common use error was reported, which was the failure to continue holding the device in place for a few additional seconds after the end of injection [22,24,25]. Although the risk of underdosing is very limited when removing the device immediately after the second click, it is also dependent on the drug being used and this risk may need to be evaluated through clinical studies. Health-care professionals, patients, and caregivers with training were able to successfully deliver the full dose in a safe and effective way, without the risk of underdosing.

In the literature, similar rates of use errors were reported for the first use of equivalent autoinjectors [23,24]. However, in the validation study, the occurrence of use errors was strongly reduced when the participants were trained to use the autoinjector. The iterative process and the different study designs demonstrated the importance of the instructions and training. The second formative study was designed to assess the usability performance within an IFU group versus a no IFU group. The participants who read the IFU experienced more successes than the participants who did not read it. The IFU reading improved the usability performance for the targeted end-users. In the next formative and validation studies, the IFU was provided with the autoinjector and was free to read or not, to mimic real-life. As, previous studies demonstrated that, from the first use, a training on how using the device minimized use error ratings [22]. During the validation, a subgroup of participants received a training. These participants almost succeeded in performing all the critical tasks without errors. In addition, the completion of successive simulated injections also contributed to reduce the occurrence of use errors and the rate of successes was increased across all user groups from the first to the last simulated injections. Therefore, in a chronic context and under real injection conditions, this low rate of use errors would even be reduced when using this new autoinjector.

Besides minimal usage errors throughout the injection process, the overall perceived usability was positively rated in terms of acceptance and ease of use. Moreover, participants reported high scores for ease of handling and confidence. These results are aligned with previously published data about ease of use, confidence, and comfort perceived by participants with similar autoinjectors [24–26]. The development of new biotherapeutics that push the traditional boundaries of volume or viscosity may impact the injection time [37–39]. While holding the device against the skin for a longer injection duration could be a challenge for the user, especially in case of hand impairment, most of the participants found the time of injection from 2.5 to 9.5 seconds to be acceptable. These results are supported by a recent simulated-use study in which participants were able to hold the autoinjector against the skin for a longer injection duration [40].

The reported human factors studies include some limitations, such as the absence of a comparator, the small sample size, and the qualitative rather than quantitative nature of the outcomes. However, simulated-use testing is a recommended method by the FDA guidance and IEC 62366 standard to assess the safe and effective use for the development of a new autoinjector device, through the record of any use-related errors [5,19,41]. In addition, the succession of a minimum of four simulated injections during the Human Factor studies, that do not reflect reality, might impact users’ feedback and usability performances. Moreover, the occurrence of some use errors could be reduced as users would be more attentive when administrating a medication via subcutaneous injection to themselves or other persons than when simulating the injection into a pad. In case of the occurrence of use errors during the first use, users may be likely to request some external help of a doctor or a pharmacist not to repeat such errors.

5. Conclusion

Successful development of a new push-on-skin autoinjector for subcutaneous chronic drug delivery was achieved through iterative design and testing. Specifications were defined according to the user’s acceptance toward the force required to activate the start of injection and to maintain the autoinjector in place. Throughout the following formative studies, the occurrence of use errors was reduced when the participants read the IFU, and thanks to the design changes and IFU updates. During the validation study, while use errors were still observed when the participants had access to the IFU, providing a training prior the simulated testing almost eliminated the remaining use errors. Only two use errors were recorded in the trained arm, and these use errors were never experience from the third simulated injections. In addition, the evaluated autoinjector provides users with convenience, ease of use, confidence in controlling the start of injection and needle safety feature. Therefore, this newly developed autoinjector can be safely and efficiently used by health-care professionals, caregivers, and patients including hand impaired users, in both a clinical and non-clinical environment. These findings support an interest expressed by the target end-users for this new autoinjector.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Declaration of interest

C Lageat, A Combedazou, C Ramus, K Guerrero, C Frolet, and S Glezer are employees of Becton Dickinson and Company. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Author contributions

C Lageat, C Ramus and K Guerrero, contributed to the conception, and design of the human factors testing. C Lageat, C Ramus and A Combedazou analyzed the data. A Combedazou wrote the paper with the inputs from all authors. All authors reviewed the manuscript. C Frolet and S Glezer supervised the project.

Supplementary material

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was sponsored by BD Medical Pharmaceutical Systems.