Abstract
Numerous companies have ordered fully automated inspections systems as a means of improving inspection performance and reducing costs. A common complaint made is that automated technology generates excessive rejection. A time-proven combination of the methods documented in this article will provide superior detection of rejects compared to manual inspection, while minimizing false rejection.
- Automated inspection systems
- Manual inspection
- Reject zone efficiency
- False rejection of containers in accept and gray zone
- Gray zone
Introduction
Numerous companies have ordered fully automated inspections systems as a means of improving inspection performance and reducing labor costs. When a new automated inspection system is installed online, it is not uncommon to experience an elevated rejection rate when making the transition from manual baseline to automated inspection. The increase in rejection rates may be due to a number of factors, including unique properties of the liquid being inspected, container shape, formation of air bubbles during spinning, Schlieren lines associated with terminally sterilized products, unique product characteristics, machine settings that are not optimized, and an unsound manual baseline. A number of preliminary steps should be taken before selecting machine settings. Although this study was performed on an Eisai 1088W Automated Inspection (AIM) system, many of the principles mentioned will improve inspection efficiency with all semi-automated or fully automated inspection systems.
Accurate Assessment of Manual Inspection Capability
According to the latest Parenteral Drug Association (PDA) survey, 69% of all participating companies stated that they were using a manual baseline for validation of automated inspection systems (1). A statistically sound manual baseline needs to be established by controlling the variables associated with manual inspections (2–8). Once the manual benchmark is established it is possible to begin developing optimal machine settings for detection of particulate matter, also called the reject zone efficiency (RZE), while minimizing costly false rejection in the accept and gray zones (RAG). The compilation of multiple studies reported by Shabushnig and Melchore at the 1994 PDA annual meeting demonstrated that manual inspection is reproducible when conducted under controlled conditions (9). This is important because validation of the automated inspection system will only be as sound as the manual baseline on which it is based.
Challenge Set Composition and Characterization
Challenge set containers should be thoroughly characterized as to the description of the particulate and, when possible, the size of the particulate. All types of particulates apt to be observed in a specific operation should be included in the challenge set.
If the methodology of Knapp and Abramson is used, the challenge container set should be composed of approximately 170 “good” containers, 40 gray zone containers, and 40 containers with visible particulates (2). The reject population of this challenge set is 16%, although some of Knapp's later publications suggest that the reject population should be ≤25% of the container population. Based on preliminary trials by this investigator, a reject population of 16–21% did not sensitize inspectors; however, increasing the reject population to 25% resulted in inspector sensitization. Manual inspection used for this study was performed using a 20.4% reject population and is within the Knapp guideline and preliminary studies performed in-house.
There has been some discussion on utilizing multiple particles per container. The disadvantage with this approach is that you will not know which specific particulate led to the rejection of a specific container (e.g., was it the glass of the fiber?). Also, at the 2007 PDA Visual Inspection Forum, Knapp addressed the undesirable multiplication effect when multiple particles increase the probability of rejection. Many investigators have performed studies using one and only one characterized particle per container (2, 3, 7–13).
Once a challenge set is assembled and characterized, the containers are randomized and engraved with a permanent identification number. The containers are then manually inspected multiple times to develop a statistical rejection probability for each container in the set. This data provides a statistical means to compare the manual inspection benchmark to automated inspection data.
The Knapp method provides a robust, time-proven means of comparing manual inspections to machine performance (2). The expectation is that automated inspections or any alternative inspection methodology must be statistically equivalent to or superior to manual inspections, without excessive costly false rejection (7).
Adjusting Spin, Brake, and Inspection Window Settings
The ability to observe the container as it is being processed, combined with electronic data generated by the Eisai AIM system, facilitates selection of initial machine settings for the inspection window, spin speed, and brake. These settings are determined prior to validation trials. Spin speed is selected by putting the liquid and any particle in the liquid into motion so that the particle will be in the inspection window during inspection. The spin duration is controlled by application of a brake to the container sometime before inspection. When the brake is applied it will stop container movement, but the liquid and any suspended particles will remain in motion. With experience, preliminary spin speed and brake settings can be determined by observation. Viscous liquids require spinning almost up to the point of inspection, while the brake can be applied much earlier for aqueous products. Containers should be spun with sufficient speed to put the liquid and any particles into motion without generating air bubbles and to ensure that the meniscus does not dip into the inspection view window at the time of inspection. With a little experience, observing the container after it has been spun and the brake has been applied is an easy way to determine if there was sufficient time for the meniscus to recover out of the inspection window before inspection. Adjustments to the brake and/or the inspection view may have to be adjusted if disturbance from the meniscus is being detected during inspection. Lastly, the inspection view is the area of the height of the container that is being inspected. The inspection view should be set so that it is free of disturbance from the recovering meniscus. For inspection of molded glass, a few of the bottom diodes must be inactivated to avoid interference from the uneven bottom of the glass container. Tubular glass does not present this problem.
The initial setting of these machine parameters becomes relatively simple as experience with similar products is gained. A recent article by Rathore et al. provides detailed information on Eisai operational basics and the fine-tuning of these settings (14) that confirm this investigator's findings. A complementary article by Singh et al. provides information on inspecting protein-based biotech drug products on the Eisai (15). These articles discuss adjusting two variables to improve performance for a given product when using an Eisai 587-2 (14, 15). The Eisai Static Division (SD) system utilized for this study had a diode array with independent light beams spaced 1 mm apart. The light is transmitted through the product container and collected on a sensor. The study reported in this article utilized an Eisai 1088W high-speed system with the same detection system arranged in a slightly different order as the 587-2 AIM system. To accommodate the increased inspection speed of the 1088W, two containers are inspected at once. At the first inspection station, container A is inspected by two light beams that are positioned at 45° angles to the container, and then container B is inspected with a light beam aimed directly at the container. The inspection sequence is reversed at the second inspection station so that each vial has been inspected from three different angles to accommodate the fast speed of this unit. The inspection of the vial from three angles is considered necessary for the speed of this unit, and collectively all three inspections comprise one inspection.
The key to all inspection methods is to optimize the spin setting required to transfer energy to the solution in order to put a particulate into motion. Once the spin is optimized, the brake is applied to stop container movement so that only the liquid and any particulate suspended in the liquid are in motion at the time of inspection. Particulates in solution will reduce one or more discrete light beams from the diode array being transmitted onto the SD sensor. The larger the voltage drop on the SD sensor, the larger the particle size at the preset sensitivity.
Inspection view, spin speed, and brake settings should be optimized before selecting a sensitivity setting. False rejection can be caused by the generation of artifacts such as air bubbles or turbulence in the inspection window. Spin and brake settings should be optimized to eliminate these artifacts while providing sufficient spin to place the suspended particle in motion at the point of inspection.
The reason some validation efforts have failed is because of an inability to minimize costly false rejection due to the failure to optimize the sensitivity setting. This article is meant to complement the latest articles on automated inspection by providing additional information on how to optimize the sensitivity setting for a specific product/container combination being inspected. Spending the time to optimize sensitivity settings and having a strategy to deal with elevated rejection is cost-effective. It is important to remember that the Knapp method provides a number of gray zone containers to provide extra security to ensure that the “must reject” containers are detected. Fine-tuning the sensitivity setting can reduce the number of gray zone containers that are rejected, while ensuring that the “must reject” containers are rejected.
The key to optimization of machine performance is to evaluate a range of sensitivity settings to determine which sensitivity setting provides particulate detection that is equivalent to or better than manual reject detection, while minimizing excessive false rejection.
The study reported in this article used seven sensitivity (voltage) settings. The containers were inspected 10 times at each sensitivity setting to develop the probability of rejection for each individual container. This was easily accomplished because the Eisai has a recirculation mode, which facilitates timely machine inspection. Plotting regression lines using the polynomial theorem enables evaluation of RZE and RAG data for each sensitivity setting. The RZE data is plotted using the left Y axis and the voltage (sensitivity) setting on the X axis. The RAG data is plotted using the right Y axis combined with the voltage setting on the X axis. The manual RZE is a constant drawn horizontally on the chart. The intercept between the RZE and RAG regression lines is very close to the optimal setting for validation, as shown in Figure 1.
Optimizing sensitivity setting.
A higher rejection rate with automated systems can be expected due to their greater sensitivity, which is capable of detecting rejects beyond the range of human visual detection. A challenge set has a number of gray zone containers that have rejection probabilities ranging from 0.30 to 0.7071. The rejection probability of each gray zone container is problematic, and rejection can occur on the initial inspection and not be repeated in a subsequent inspection. As described by Knapp, the gray zone containers provide a security zone ensuring that the “must reject” containers will be rejected with a high degree of certainty.
At the end of the initial inspection, the questionable containers are inspected a second time (cull inspection). Containers accepted after the second inspection are placed with acceptable stock, while rejected containers are declared rejects. Knapp defines gray zone containers as usable, but that they can be sacrificed to assure that the reject zone containers are rejected (16). The second inspection of the containers is performed at the end of the batch in order to allow for artifacts such as air bubbles or Schlieren lines. Re-inspection also reduces a portion of the gray zone population that has a low rejection probability.
Validation data should be generated to support this “accept in two” re-inspection procedure, as graphically portrayed in Figure 2.
“Accept in two” strategy to minimize false rejects.
Further security in this decision is provided by re-inspection of the reject population, known as the “accept in two” approach described by Knapp (17). This testing verifies that the “must reject” containers are detected after two consecutive inspections. In every instance, this investigator found that automated inspection performance is statistically equivalent to or greater than manual baseline. The re-inspection strategy is designed to reduce false rejection and can be built into validation. This method involves using a slightly greater sensitivity for both inspection and re-inspection of the questionable containers. The containers rejected after the initial inspection are classified as questionable. It is important to recognize that the containers initially rejected during the first inspection include true rejects, products containing air bubbles or other product artifacts, and gray zone containers that have a probability of rejection ranging from 0.30 to 0.7071. After letting the containers settle to dissipate air bubbles or other product artifacts, a second (cull) inspection will split the gray zone population.
In practice, a line clearance is performed and the questionable containers are returned to the line for re-inspection. After re-inspection, acceptable containers are placed with acceptable stock and those rejected a second time are classified as rejects. This decision is final and these rejected containers cannot be re-inspected manually with the intent to return the containers to acceptable stock. Performing manual inspection on these containers would negate validation of a superior inspection process by using a less capable process.
If the re-inspection strategy is used, it should be built into the validation strategy. The initial and second inspections are performed at a slightly increased sensitivity compared to single inspection. The re-inspection of the 51 reject vials in this study was performed at the same sensitivity as was used in the first inspection. Simply stated, two consecutive inspections at a sensitivity yielding 90% accuracy in detection should result in a detection rate of 81% after re-inspection. In practice, the realized result falls between 81% and 90% and consistently results in an efficiency that is equivalent to or better than the manual baseline.
The RZE is an important indicator of detection performance, but it does not tell the entire story of automated inspection superiority. Data generated by machine inspections is more decisive than manual inspection. This highlights the consistency of machine inspections (Table I).
“Accept in Two” Re-inspection of 51 Questionable Vials Using 20 Inspections of “Questionable” Containers
This supportive study indicates that re-inspection of the 51 containers ejected by the Eisai on the first pass maintained superior performance to manual inspection after an “accept in two” cull inspection on the Eisai 1088W.
Other Parameters To Consider
There is more to the story of the superiority of an automated inspection system performance compared to manual inspection. Unlike human inspectors, automated systems are not subject to fatigue or affected by overhead lighting. Machine data is more decisive and provides greater reproducibility than manual inspection. The data in Table II supports the graphic presentations of the decisiveness and reproducibility of automated inspection compared to manual inspection (Figures 3 and 4).
Reproducibility of Inspection: Manual versus Eisai
Decisiveness of automated inspection vs manual baseline.
Cosistency in classifying containers that were rejected or accpeted in 20 consecutive inspections.
The data in Figure 3 shows the decisiveness of automated inspection versus the manual baseline.
The graphic presentation in Figure 4 demonstrates the consistency of manual inspection versus the Eisai. As shown, 90 vials were declared either “good” or “bad” by manual inspection for all the 20 inspections performed. During inspections by the Eisai 1088W, 214 vials were consistently declared either “good” or “bad” in all 20 inspections. Overall, the Eisai was 2.4 times more decisive in classifying containers than the manual baseline.
In summary, great care must be taken in constructing a challenge set and following the basic practices for the conduct of manual inspection. If the manual inspection baseline is statistically sound, then the validation of an automated system will be successful without costly false rejection.
Summary
If the manual inspection baseline is statistically sound, validation of an automated system can proceed. Inspection view window, spin speed, and brake setting become relatively easy to set with the experience gained by observing the containers as they move in the machine and reading the electronic data generated by the system. Selecting machine settings using these procedures will optimize detection and minimize false rejection. The Eisai 1088W AIM combined with the “accept in two” strategy will outperform manual inspection in RZE, reproducibility, and decisiveness, and will reduce false rejection. In the words of Julius Knapp, “Both reject rates must be considered to achieve the desired product quality with an economically effective inspection” (17).
Inspection does not put quality into the product, but it is a good monitor of the overall manufacturing process (batching, preparation, filling, and capping). Improvements and standardization should be the goals toward inspection excellence. In order to reach the goal of manufacturing excellence, improvements need to be made to all phases of the manufacturing process. The expectation of absolute sterility assurance or the absence of particulate contamination is not within the capabilities of the present technology (17). Improvements such as barrier technology in the filling room will reduce particulates. Other improvements in upstream manufacturing operations will further reduce the number of particulates before inspection. Elimination of inspection variables through a universal guidance document will enable inspection to keep pace with manufacturing improvements.
- © PDA, Inc. 2010
