Abstract
Intravitreal injection (IVI) is the most commonly performed intraocular procedure worldwide. Several manufacturers have developed glass prefilled syringe (PFS) devices to increase the ease of performing IVIs and reduce the complications associated with medication preparation. This formative human factors study assessed a novel, polymer PFS alternative to glass syringes to support development of a usable, silicone-free delivery platform for IVI. Thirteen retina specialists (RSs) with experience preparing a minimum of ≥10 IVIs per week completed the study. RSs were presented with the concept device and prototype instructions for use and completed hands-on tasks to simulate IVI. They then evaluated the concept device for ease of use, comfort, safety, and overall preference versus the IVI devices they are accustomed to using. The primary objectives were to assess the ease of use and acceptability of the proposed syringe design, evaluate the corresponding instructions for use (IFU), and identify any potential usability issues. The secondary objectives were to evaluate a new tamper-evident cap design and compare several externally printed dose marking designs. There were 130 total opportunities for use errors that deviated from the IFU. Of these 130 steps, 110 were a Success, 17 were Incomplete or Incorrect, 2 were Resolved, and 1 was the result of a Study Artifact. All 13 participants completed 3 Essential Tasks successfully and at least 10 participants completed each of the 4 Safety-Critical Tasks successfully. A total of 20 errors were made throughout the test simulation, most of which were rooted in unfamiliar use steps or transference behaviors. Overall, the concept device was found to be usable, acceptable, and safe for IVI by experienced RSs. RSs preferred the concept device to IVI products supplied in vials, but there was no notable preference for the concept device design compared to current glass PFSs used for IVI. The unique features of the concept device, including absence of silicone oil and break-resistance, were mostly recognized by participants and may offer an improvement to currently available systems for IVI.
Introduction
Intravitreal injection (IVI) is the most commonly performed intraocular procedure worldwide and a cornerstone of retinal care (1, 2). In 2016, an estimated 6 million IVIs were administered in the United States alone (3). Numerous medications are currently delivered via IVI to treat diseases such as diabetic macular edema (DME), diabetic retinopathy (DR), neovascular age-related macular degeneration (AMD), macular edema after retinal vein occlusion (RVO), uveitis, and myopic choroidal neovascularization (mCNV), and many more therapies are under investigation (4). Among the most important agents are the vascular endothelial growth factor (VEGF) inhibitors, which function to reduce ocular angiogenesis and associated vascular leakage (5). At the time this study was conducted, there were three FDA-approved VEGF inhibitors available for IVI: pegaptanib (Macugen; Eyetech, Cedar Knolls, NJ), aflibercept (Eylea; Regeneron Pharmaceuticals, Tarrytown, NY), and ranibizumab (Lucentis; Genentech/Roche, South San Francisco, CA). Although it is not FDA-approved for IVI, bevacizumab (Avastin; Genentech/Roche, South San Francisco, CA) is also commonly employed in retina practices (6). The available packaging configurations for these products at the time of study are provided in Table I.
Packaging Configurations for Marketed VEGF Inhibitors
IVI is typically performed by a trained retina specialist (RS), as it requires aseptic manipulation and precise delivery of extremely small dose volumes (e.g., 0.05 mL) into the delicate structures of the eye. Although the procedure is generally safe, improper technique can result in serious adverse effects such as endophthalmitis, intraocular inflammation, retinal detachment, intraocular pressure elevation, and ocular hemorrhage (7). Some of these complications have been associated with medication preparation (8⇓⇓⇓–12), and particular concern has been raised regarding introduction of silicone oil (SO) droplets into the eye from silicone-lubricated syringes used during the procedure (13⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓–25), especially when conventional hypodermic syringes are used. These issues have spurred the development of delivery devices intentionally designed for IVI, such as the recently approved ranibizumab prefilled syringe (PFS) (26). Use of PFSs to deliver IVIs offers the potential to reduce injection time, lower the risks of contamination, endophthalmitis, intraocular air bubbles, and SO droplets, and increase dosing accuracy (8, 27⇓⇓⇓⇓⇓⇓–34).
To date, marketed IVI medications are provided in glass primary containers (either vials or PFSs). Because of the potential for glass defects and the need for SO lubricant (35), novel polymer PFS devices are currently under development. The FDA recommends that manufacturers conduct human factors testing during the development of new medical devices to ensure they are safe and effective for the intended users, uses, and use environments. This process typically begins with formative testing, which is intended to inform the design of the device-user interface, reduce or eliminate potential use errors, determine training and labeling requirements, and help define the structure of validation (summative) testing (36). We present the results of a formative human factors study to support development of a usable, silicone-free delivery platform for IVI.
Methods
Study Materials
PLAJEX is a SO-free, prefillable syringe system with a polymer barrel composed of cyclo-olefin polymer (COP) and a butyl rubber plunger stopper coated with i-coating technology. The i-coating provides lubricity at the stopper-syringe interface without the need for silicone inside the syringe barrel. The studied PLAJEX syringe provides a maximum 0.5 mL fillable volume, an extended backstop for ergonomics, a traditional male luer for connection to an ISO 80369-7 standard injection device, a novel tamper-evident tip cap design that is compatible with an integrated luer lock collar, and an internally molded dose mark to aid in dose setting (Figure 1). In addition to these features, six externally printed dose marking designs (two different line thicknesses and three different circumferential patterns) were evaluated for user preference (Figure 2) to compare these designs against those that are currently used on marketed IVI PFSs.
Concept device design and features.
Dose marking design alternatives.
The syringe is supplied with a tamper-evident tip cap that employs a mechanism whereby a rubber stopper that provides container closure is trapped within a white plastic housing with a clear molded observation window. When the tip cap is removed (unscrewed), the rubber is freed within the tip cap; if it is reinstalled, it will “pop up,” providing feedback to the user that the cap has been removed and subsequently reinstalled (Figure 3). Prototype instructions for use (IFU) were developed to incorporate all of the design elements being evaluated (Figure 4). The use steps and wording of the study IFU were patterned after the FDA-approved ranibizumab PFS IFU.
Tamper-evident tip cap design.
Prototype instructions for use.
Study Design
The three primary objectives of the study were to: 1) assess the ease of use and acceptability of the proposed syringe design according to RSs who perform IVIs; 2) evaluate the corresponding IFU for the design; and 3) identify any potential usability issues with the design. The two secondary objectives were to: 1) evaluate a new tamper-evident cap design; and 2) compare several externally printed dose marking designs. Although participants were asked to evaluate the concept device against existing products during the study, this was intended to establish benchmarks and identify opportunities for design improvement. The goal of this study was not to compare the relative value of different delivery devices.
A total of 13 RSs completed the study. All participants prepared a minimum of 10 injections per week and had experience injecting aflibercept, ranibizumab (both vial and PFS), and bevacizumab. Only 9 of the 13 participants had experience with pegaptanib. Recruiting was performed via commercially available, nationwide panels, and a purposive sampling approach was taken to capture a range of demographic characteristics and minimize bias where possible. Participant demographics and their most frequently administered IVI products are provided in Table II. This research was conducted according to the principles of the Declaration of Helsinki and the Marketing Research Association’s Code of Marketing Research Standards; all participants granted their written informed consent.
Participant Demographics
Study Procedure
The study was completed at two US research facilities, each equipped with a two-room suite containing a one-way mirror. Two study moderators administered the usability test sessions and recorded test data, and all interview sessions were video recorded from multiple angles to capture use errors, operational difficulties, and close calls on Essential Tasks and Safety Critical Tasks. A head model with a representative, anatomically correct injection eye was configured by each participant to the correct height, side (left or right injection hand with head to the left or right of the physician), and injection posture (sitting or standing) at the study outset. This ensured appropriate position based on actual injection preference/practices. In addition, the study room was equipped with examination gloves, masks, and sterile drapes for participants to use if they did so in practice.
Each test session was initiated by introducing the participants to the test environment, explaining the test purpose, and asking demographic-related questions. RSs were then presented with the concept device without any packaging, an unopened 30-gauge, ½ inch injection needle (BD, Franklin Lakes, NJ), and a prototype IFU supplied on a printed sheet. Participants were instructed to use the IFU as much or as little as they’d like according to how they would approach using a new device in practice. The concept device was prefilled with 0.2 mL of placebo solution matched to the viscosity of ranibizumab. When they were ready, participants then completed the preparation and administration of one simulated IVI using the placebo solution and eye model. None of the participants received formal training or a demonstration before participating in the usability test session. After each task and during a post-task interview, the moderator interviewed participants about their interactions with the device using retrospective look-back techniques. Participants then evaluated the concept device for ease of use, comfort, safety, and overall preference versus the aflibercept vial, ranibizumab PFS, ranibizumab vial, and pegaptanib vial, based on their experience using these products. Bevacizumab was excluded from the preference evaluation, as it is not FDA-approved for IVI. Ratings were captured on a 7-point Likert scale (1 = “Completely Disagree”, 4 = “Neutral”, 7 = “Completely Agree”). Next, RSs were presented with a second prefilled concept syringe that included the externally printed dose marking alternatives and were asked to provide subjective feedback on preference. Finally, participants were prompted to provide subjective feedback on specific aspects of the concept device design (i.e., tamper-evident cap, plastic construction, lack of SO). This section began with a brief description of each feature before recording participant responses, as some features would not have been obvious during simulated use (e.g., lack of SO).
Essential and Safety-Critical Tasks
An evaluation of the intended use of the product identified a total of 10 use steps, of which 6 were categorized as Essential Tasks and 4 were categorized as Safety-Critical Tasks in accordance with best practices for human factors engineering of combination products. An Essential Task is necessary for successful use of the device for its intended purpose, but if missed would not lead to safety concerns. Safety-Critical Tasks, on the other hand, are tasks during which users could make errors that would have a negative clinical impact. Each task was then assessed as Successful with No Issues (S), Incomplete/Incorrect (I), Resolved (R), Operational Difficulty (OD), Close Call (CC), Study Artifact (SA), or Not Assessed (NA) according to the definitions provided in Table III. Essential and Safety-Critical Tasks with their corresponding assessments are summarized in Table IV.
Use Step Assessment Criteria
Summary and Assessment of Use Steps
Results
The enrolled RSs each completed 10 use steps, providing a total of 130 total opportunities for use errors that deviated from the instructions provided in the IFU. Of these 130 steps, 110 were a Success, 17 were Incomplete or Incorrect, 2 were Resolved, and 1 was the result of a Study Artifact (Table IV). No participant made the same error more than once, and no participants required assistance from the test administrator during the testing. All 13 participants completed 3 of the 6 Essential Tasks successfully (attaching the needle by twisting, removing the needle shield by pulling, and removing the syringe from the injection site), and at least 10 participants successfully completed each of the 4 Safety-Critical Tasks. A total of 20 errors were made throughout the test simulation, 12 of which were during Essential Tasks and 8 during Safety-Critical Tasks. Overwhelmingly, inspecting the tip cap was the most common Incomplete or Incorrect step, with only four participants completing it successfully. All use errors are summarized in Table V and each is explained in the following section.
Summary of Use Errors
Use Errors
Did Not Inspect the Tip Cap
Nine participants removed the syringe tip cap before inspecting it, missing the first Essential Task on the IFU. After root cause analysis, it was determined that 8 of the 9 participants would not normally read the device IFU in practice and therefore did not know to inspect the tip cap. One participant read the IFU, but because of his more frequent experience with aflibercept vials and ranibizumab PFSs, he was not accustomed to inspecting the tip cap of the syringe before attaching the needle. This likely affected his interactions with the concept device, regardless of the IFU. Moreover, five of these participants expressed a lack of concern about device tampering in practice, which may have decreased their likelihood to check for tampering during the study.
Did Not Remove the Tip Cap
One participant attempted to attach the needle onto the syringe without removing the tip cap. In a scenario where the syringe was not provided to the user in an externally sterile package or was provided in a sterile package but was not used on a sterile field, this error could lead to a breach in sterility. As a result, it was classified as a use error. After root cause analysis, two contributing factors were identified: 1) the participant did not read the IFU and therefore did not read the step that instructed to twist off the cap; and 2) the participant did not recognize that the syringe had a tip cap that needed to be removed. This RS exclusively injects aflibercept in his practice and was therefore not familiar with this type of tamper-evident device (the disposable syringe provided in the package alongside the aflibercept vial does not have a tamper-evident tip cap that needs to be removed).
Did Not Consolidate Air
Three participants failed to successfully consolidate the air bubbles in the syringe before expelling them. These errors were attributed to a single root cause: although the three participants had experience removing air bubbles when preparing aflibercept vials, ranibizumab vials, and ranibizumab PFSs, each stated that it was not in their current practice to consolidate the air bubbles before injection.
Did Not Expel Air
One participant failed to expel the excess air in the syringe before injecting into the eye model. At first, the participant saw no issue with this and behaved nonchalantly during the IVI preparation and administration process. Once probed, however, he noted that he “was not sure if that was something that you had to do since it was prefilled.” Although he was familiar with the ranibizumab PFS, which requires a similar step, it was not clear to him that the concept device required expulsion of air before administration, suggesting an uncertain understanding of PFS use.
Did Not Set the Dose Properly
Two participants did not attempt to set the dose on the concept device despite the instructions provided and their experience with the ranibizumab PFS. On root cause analysis, it became evident that the participants did not see the internal dose mark at first without reading the instructions.
Pulled the Needle Out of the Eye Prematurely
One participant removed the needle from the eye model before the injection was complete. This removal resulted in significant leakage from the needle tip. Despite this, the participant showed no sign of concern and did not indicate this behavior was unexpected or disconcerting post-injection. After probing, the root cause for this error was identified, and the participant stated that the “[IVI] drugs I’ve been using, they’re so potent…so if a tiny bit less is getting into the eye, I do not really think it matters.”
Resolved—Close Calls
Only one participant encountered a close call during the hands-on tasks. This participant first tried to snap the tamper-evident cap off instead of twisting it as per the IFU. However, he quickly realized the cap had to be twisted and recovered without consequence. This error was rooted in his more frequent experience with the ranibizumab PFS, which is supplied in a glass syringe with a snap-off, tamper-evident cap (V-OVS; Vetter Pharma). Given his current familiarity with IVI PFSs, the participant transferred this snap-off behavior to the concept device.
Resolved—Operational Difficulties
Only one participant encountered an operational difficult during the hands-on tasks. While holding the device at eye level as indicated in the provided IFU, this participant second-guessed whether or not he set the dose properly. However, after looking at the IFU, he was reassured that he did it correctly. Similar to the preceding, this error was rooted in the participant’s prior experience with other injection devices, namely the ranibizumab PFS and the disposable syringe supplied with the aflibercept vial. Both of these devices have bold black dose mark lines provided on the exterior of the syringe.
Study Artifacts
One participant experienced a study artifact during syringe disposal. This RS completed the simulated injection and, rather than disposing of the syringe in the provided sharps collector, proceeded to the table set up for study debrief and discussion with the syringe in hand. This event was considered directly related to the study procedure and would not have occurred during actual use.
Subjective Feedback
Ease of Use
Overall, participants found the concept device easy to use from preparation to completion of the injection (Figure 5). Ratings demonstrate that removing the tip cap was viewed as the easiest task, though one participant reported that removing the tip cap was not intuitive. This could be explained by this particular participant’s frequent experience with the V-OVS cap on the ranibizumab PFS (explained previously) and lack of familiarity with the new tip cap design on the concept device. Setting the dose was viewed as the least intuitive task to complete, mainly because of the participants’ previous experience with externally printed dose marks. Regardless, only one participant reported that he was not confident that the dose was set successfully. Though this use step was seen as challenging, it did not impede most participants from finishing the task, with 10/13 completing it successfully.
Participant evaluation of ease of use.
On evaluation of the prototype IFU, all participants were found to be able to clearly read, interpret, and understand the instructions provided. No participant reported that any information was missing or poorly presented, and no use error was attributed to IFU design.
Comfort and Safety
In terms of comfort, participants rated the device highly, with 11/13 reporting that it was very comfortable and 10/13 reporting that they felt in control of the injection and were able to maintain their normal practice. The concept device was also nearly universally considered safe and acceptable, with only one participant (the same from above) claiming that he did not feel the device was acceptable or safe because of the lack of visibility of the internal dose mark (Figure 6).
Participant evaluation of comfort and safety. IVI = intravitreal injection.
Dose Marking Alternatives
Of the six externally printed dose marking designs, participants had a clear preference for the near-circumferential thin line. Nine of the 13 participants preferred the thin circumferential line (“Line type I – Thin” shown in Figure 2) over all others, and no other patterns of preference were observed. This dose marking style is the most similar to what is presented on currently marketed IVI PFSs. Several participants expressed concern that thicker lines or lines with larger breaks could have a negative impact on dose setting accuracy.
Device Preference
Regarding preference, 10/13 participants reported they would use the concept device if given the option, and 6/13 participants rated the device as more comfortable than the IVI devices they currently use (Figure 7). Overall, most participants were neutral in terms of preference for a prefilled plastic device compared with glass, although more participants viewed the plastic device as less likely to break compared with the glass syringes they currently use. Participants did perceive value in having a SO-free syringe before they were told the concept device was SO-free, with 9/13 participants rating that they had some concern about the SO in the syringes they currently use for IVI. When prompted about these ratings, one participant stated: “silicone oil bubbles are horrible; patients have them and they hate them…it is a permanent floater that you give them.” Another explained: “my goal is not to put silicone oil in the eye, my goal is to give them the medication.” Moreover, participants preferred the concept device over current IVIs that require a vial during preparation. Preference for the concept device versus the ranibizumab PFS was evenly split, but fewer participants preferred the concept device compared with the pegaptanib PFS (Figure 8).
Participant evaluation of specific use steps. IVI = intravitreal injection.
Participant preferences for IVI devices. IVI = intravitreal injection.
Discussion
This formative human factors study sought to evaluate several features of a novel PFS device for potential ophthalmic applications. Overall, the concept device was found to be usable, with the majority of use errors concentrated around inspection of the tamper-evident syringe tip cap. Because the new tamper-evident cap design was one of the features assessed in this study and one that could be considered necessary for its intended use if brought to market, tip cap inspection was considered an Essential Task and specifically included in the prototype IFU. However, this is rarely the case with existing products used for IVI, and established practices related to tip cap inspection appear to be limited. Currently, the ranibizumab PFS is the only marketed device for IVI that features a tamper-evident cap. In the marketed product’s IFU, users are instructed to inspect the syringe and dispose of it if the cap is detached from the luer lock, the syringe is damaged, or particulates, cloudiness, or discoloration are visible (37). Notably, tip cap inspection was not considered an Essential or Safety Critical Task in formative or summative testing of the ranibizumab PFS (26). Most of the remaining use errors were related to lack of frequent experience with PFS devices for IVI, transference behaviors from existing products, or incorrect technique around the IVI procedure itself. Of note, further encouraging consolidation and expulsion of air bubbles should be a focus of design refinement, as injection of intraocular air bubbles may result in transient increases in intraocular pressure, albeit typically without serious sequelae (38, 39). Regardless, the errors reported in this study signal an opportunity to improve instruction and/or design language to prevent them in subsequent studies.
Participants generally rated the concept device as easy to use, comfortable, acceptable, and safe. The most frequent source of difficulty was visualizing the internal dose mark, which was largely attributed to transference from currently marketed devices. Still, even without any externally printed dose marking, most participants were able to set the dose successfully and confidently. Although incorporating the preferred external dose marking (the near-circumferential thin line) into the design has the potential to further improve usability, in its current embodiment, the majority of participants reported that they would use the concept device if given the opportunity, and about half expressed that the device was more comfortable to use than their current devices. These findings could be used to inform subsequent human factors evaluations that focus specifically on optimizing the dose marking design.
Some of the unique features of the concept device were evident to participants, whereas others were less so. Most participants expressed concern about the presence of SO in their current IVI devices, and the absence of SO in the concept device was viewed positively. The extrusion of SO droplets into the eye during IVI has been thoroughly described and is associated with floaters (14) and possible elevations in intraocular pressure over time (16). The risk of SO-related adverse effects is thought to be lower with the ranibizumab PFS (26), as it is manufactured using a baked-on siliconization process designed to reduce free SO levels (40). However, a recent laboratory analysis of ranibizumab PFSs, aflibercept vials, and two types of repacked ready-to-use bevacizumab plastic syringes revealed similar absolute amounts of SO microdroplets in all four products (21). In relative terms, SO levels were found to be higher in ranibizumab PFSs, which the authors attributed to the product’s storage in siliconized glass syringes and overall lower protein content compared to the others studied.
Beyond avoidance of silicone introduction into the eye, the absence of SO in the concept device may have other potential benefits related to the injection process. With traditional siliconized syringes, SO can migrate during product storage or agitation, resulting in an uneven distribution of lubricant within the syringe and/or at the plunger’s rest position (41, 42). As a result, siliconized syringes can require greater forces to initiate plunger movement (break-loose force) compared with those required to maintain movement (glide force), especially as syringes are stored over periods of time (43). Silicone migration can also produce “stick-slip” or “stiction” behavior, characterized by inconsistent glide force as the plunger is depressed and moves across the inside of the syringe barrel (44). Although these phenomena have not been directly associated with adverse effects in the setting of IVI, any factor that alters injection speed or precision could result in increased intraocular pressure or damage to eye structures (45). Lack of SO in the concept device allows for more consistent break-loose and glide forces compared with those of siliconized systems without exceeding the maximum permissible value for actual use (43). These characteristics have the potential to prevent issues related to differences in force application during the injection process. Moreover, the combination of high break-loose force and fine motor movement required for IVI dose setting may cause the clinician to “overshoot” the target dose mark, resulting in underdosing and drug waste (46), although this risk appears to be hypothetical.
Participants viewed the device’s COP composition as generally neutral, with some considering it to be less likely to break than traditional glass syringes. Although perhaps not perceptible to participants during the study, syringes made of plastic are known to have higher break resistance compared with that of typical glass syringes (35). This could potentially reduce the risk of breakage and associated complications during use or transport. Unsiliconized plastic also offers other potential benefits over siliconized glass, including decreased surface reactivity, reduced protein aggregation, and improved overall product stability (35, 43, 47). In addition, the potential for glass defects during manufacturing are eliminated, providing an advantage to drug manufacturers during processing in reduced inspection and drug product loss (48); neither of these benefits would be apparent to the end user.
The major limitations of this study were its relatively small sample size and lack of inclusion of ophthalmic technicians. Some retina practices rely on technicians to help prepare syringes for IVI, and it may have been valuable to observe their interactions with the concept device as well. Also, although the study was conducted in two separate facilities, one in the Northeast and one in the South, possible geographic differences in practice were not fully captured to the extent they exist.
Conclusion
In summary, the studied PFS design was found to be usable, acceptable, and safe for IVI by experienced RSs. Use errors observed in the study were largely rooted in unfamiliar use steps or transference behaviors from existing devices, both of which are addressable with improved instructions and design language. Minor difficulties related to dose setting were anticipated in advance of the study, and a potential external dose marking solution was identified by participants for incorporation into the overall design. RSs preferred the concept device to IVI products supplied in vials and half preferred the concept device to the ranibizumab PFS. The unique features of the concept device, including absence of SO and break-resistant COP composition, were mostly recognized by participants and may offer an improvement to currently available systems for IVI. Overall, the outputs of this study can be used to inform design refinements and future formative and summative human factors testing.
Conflict of Interest Declaration
This research was sponsored by Terumo Pharmaceutical Solutions.
- © PDA, Inc. 2022
References
In This Issue
Jump to section
Related Articles
Cited By...
- No citing articles found.