Abstract
Two sterility test methods, the ScanRDI® rapid sterility test and the United States Pharmacopeia/European Pharmacopoeia/Japanese Pharmacopoeia (USP/EP/JP) compendial sterility test, were compared with respect to the limits of detection for the presence of viable microorganisms in aqueous solutions at low inoculation levels. The ScanRDI® system employs a combination of direct fluorescent labeling techniques and solid-phase laser scanning cytometry to rapidly enumerate viable microorganisms from aqueous samples, whereas the compendial sterility test is a qualitative, growth-based method that uses a visual assessment of turbidity to indicate microbial contamination. Eight microorganisms were evaluated, seven compendial microorganisms (Clostridium sporogenes, Escherichia coli, Pseudomonas aeruginosa, Staphylococcus aureus, Bacillus subtilis, Aspergillus niger, Candida albicans) and the Gram-positive anaerobe Propionibacterium acnes. The number of viable organisms was estimated using the ScanRDI® method and the conventional sterility test method using most probable number methodology. The mean difference between the methods was computed and 95% confidence intervals around the mean difference were estimated. The ScanRDI® method was found to be numerically superior and statistically non-inferior to the compendial (USP/EP/JP) sterility test with respect to the limits of detection for all organisms tested.
Introduction
A common experimental approach to demonstrating functional equivalence between two analytical methods is to conduct a series of parallel, or side-by-side, experiments. Such studies are best suited to those quantitative microbiological methods that characterize populations. In these methods, the abundance of microbial cells allows for statistical evaluation of the experimental outcomes. However, the parallel testing approach is not well suited to compare methods that are qualitative in nature, or those that attempt to detect and quantify rare events (i.e., detecting microbial contamination at extraordinarily low concentrations). With these methods, the inherent statistical power of the microbial population is lost, and it must therefore be provided through increased repetition of the experiment.
A sterility test, in its most basic form, is a qualitative assay that is designed to detect the absence of viable microbial cells in or on a product. The expected outcome for such a test is zero, or no organisms recovered. In the current pharmaceutical environment, the likelihood of observing a sterility failure is exceedingly rare, generally on the order of less than 0.1%. As such, studies designed to demonstrate functional equivalence for these types of methods, using the qualitative presence/absence metric, will require inordinately large sample sizes, which become significantly larger as the differences between two methods shrink (1).
An alternate approach to evaluating equivalence between such qualitative methods is through the application of the most probable number (MPN) method. The MPN method is well-known and widely used for estimating the density of viable microorganisms based on the Poisson distribution (2, 3). A sample is diluted to such an extent that only a small proportion of sub-samples will contain organisms that will display positive growth when assayed. It is thus possible to use a series of qualitative assays, such as a sterility test, in an MPN series, to provide a quantitative measurement of the number of viable organisms in a solution. More importantly, the MPN method provides a means to utilize a quantitative non-inferiority criterion for a qualitative test method. Non-inferiority studies are designed to show that a particular test method is no worse than a standard method by a given margin. Non-inferiority testing further permits the conclusion of superiority.
In theory, the presence of a single viable cell should be sufficient to generate a positive result in a sterility test. Consequently, the limit of detection is the single most important attribute of a sterility test method. In this study we employed an MPN scheme to compare the ability of two sterility test methods to detect microorganisms when they were exceedingly rare. We compared a rapid detection approach using the ScanRDI® microbial analysis system and the reference sterility test method described in the current US/EP/JP pharmacopeias (United States Pharmacopeia/European Pharmacopoeia/Japanese Pharmacopoeia). We also evaluated the robustness of the methods by replicating the experiments using three discrete teams of analysts in three different laboratories. The data generated from these studies support the conclusion that the ScanRDI® sterility test method is non-inferior to the compendial sterility test with respect to the limits of detection.
Materials and Methods
Overview of the ScanRDI® System
The ScanRDI® microbial analysis system employs a combination of direct fluorescent labeling and solid-phase laser scanning cytometry to rapidly enumerate viable microorganisms from aqueous samples. The system has sufficient sensitivity to detect a single viable microorganism in 2–3 h, without the need for growth and multiplication. Cells are collected by filtration onto a single, 0.4-μm-porosity polyester membrane and treated with a proprietary combination of background and viability stains. The viability stain consists of a membrane-permeant, non-fluorescent substrate, similar to fluorescein diacetate, which freely crosses the cell membrane. The substrate is cleaved by nonspecific esterases into a membrane-impermeant chromophore. As a result, cells with intact membranes accumulate the chromophore in the cytoplasm while those with compromised membranes are unable to retain the fluorescent probe. A processed filter is subsequently transferred into the cytometer where the filter is scanned by a high-speed, 488-nm argon laser. The entire filter surface is scanned in a pattern that ensures that each section of the membrane is illuminated twice. Fluorescent light is detected by multiple photomultiplier tubes and processed through a number of discrimination parameters that enable the instrument to differentiate between valid signals (labeled microbes) and background noise (electronic, optical, and/or auto-fluorescent particles). The result of the scan is displayed in the form of a membrane scan map that identifies the position of each fluorescent event. Each event is verified by visual examination using an epifluorescent microscope with an automated motorized stage. Those events that possess morphology consistent with microbial cells are subsequently considered viable microorganisms. Amorphous particles and auto-fluorescing crystals are invalidated. The combined speed and sensitivity of the ScanRDI® system make it ideally suited for industrial applications, particularly for ascertaining the sterility of filterable solutions (4–6).
Sterility Test Methods
Organisms and Culture Conditions:
Organisms, ATCC identification, and growth conditions used in each MPN series are given in Table I. Organisms, with the exception of Propionbacterium acnes, were obtained as lyophilized Bioballs™ from BTF Precise Microbiology (Pittsburgh, PA). Bacillus subtilis, Clostridium sporogenes, and Aspergillus niger were obtained as preparations of spores, while the remaining organisms were vegetative cells. Bioballs were dissolved in sterile saline to yield an initial concentration of 30 cfu/mL−1. Subsequent dilutions were performed in saline to yield three dilutions: 10−1, 10−2, and 10−3. P. acnes was serially diluted in sterile saline to yield cell concentrations ranging between 0.03 and 3 cfu/mL−1. Culture conditions (media, temperature) were based on the compendial requirements for sterility testing and are indicated in Table I.
Microorganisms and Culture Conditions
ScanRDI® Test Method:
At each dilution, the contents of a tube were briefly vortexed and a 1-mL aliquot was filtered through a fluorassure integrated filtration unit (FIFU) (Chemunex 220-C2104-02). The filter was incubated aerobically on resuscitation media for 3.0 h ± 10 min at 30–35 °C prior to labeling with the viability reagent. This resuscitation step is required to activate the metabolic activity of dormant spores and to induce germination such that the spores can be labeled with the viability reagent (6, 7, 9). Under these conditions both aerobic and anaerobic bacteria spores as well as mold spores can be quantitatively detected by the ScanRDI® system with no effect on the vegetative forms. While longer incubation periods may improve the recovery and detection of spores, extended periods of incubation adversely affect the ability to efficiently label and detect other microorganisms, particularly those that have been physically damaged or subjected to metabolic and environmental stressors. As such, the parameters (time and temperature) of the pre-labeling step were optimized to achieve accurate recovery of all the test organisms, and by inference, the broadest spectrum of organisms possible. Following the pre-labeling step, the filters were aseptically transferred onto an absorbent pad saturated with the proprietary fluorogenic stain (Chemunex 201-R2007-03) and incubated (45–60 min at 30–35 °C, aerobic conditions). The processed filters were transferred into the cytometer and scanned using discrimination parameters determined by the manufacturer. Each fluorescent event was verified by visual examination using an epifluorescent microscope with an automated motorized stage at a magnification of 500×. Those events morphologically consistent with microbial cells were considered viable microorganisms and enumerated by the analyst. For the purpose of this study, the presence of one or more validated fluorescent events was scored as a positive sterility result.
Direct Sterility Test Method:
Test solutions were assessed for sterility using a direct inoculation test method (9–11). This increased assurance that the appropriate levels of organisms were delivered into the assay. For enumeration, a 1-mL aliquot from each dilution tube in the MPN series was directly inoculated into 100 mL of the appropriate culture medium and incubated for 14 days at the recommended temperature (Table I). Positive growth was determined by a visual assessment of turbidity.
Quantitation by MPN
To create the MPN series, a suspension of each microorganism was serially diluted, as described previously, to yield suspensions containing approximately 3.0, 0.3, and 0.03 cfu/mL. From each dilution tube, a series of 1-mL aliquots was aseptically transferred into each of five FIFU assemblies, and similarly into each of five culture vessels, thus creating two 5-tube MPN series (see http://www.cfsan.fda.gov/∼ebam/bam-a2.html) (12). The concentration of microorganisms in one MPN series (FIFU) was determined using the ScanRDI® method, while estimates of the microorganism concentrations in the other series were determined using the reference sterility test method.
Statistical Methods
Test Method and Early Stopping Rule:
A paired t-test was used to test the hypothesis that the concentration of microorganisms detected by ScanRDI® is less than or equal to 70% of the concentration of microorganisms detected and enumerated by the conventional sterility test method (i.e., the ([ScanRDI® mean log10(MPN)] − [Conventional mean log10(MPN)]) > (log10(0.7))) is not inferior to that of the compendial sterility test method) (13). The experimental designed allowed up to 25 pairs of MPN determinations to be collected for each organism. A confidence limit of 95.25% is required to control for an overall type I error using this number of samples. Consequently, a test of the hypothesis was performed by comparing a lower, one-sided 95.25% confidence limit on the paired difference (log-transformed [log10] bacterial concentration determined from the MPN series using ScanRDI® minus the log-transformed concentration of bacteria determined using the reference sterility test method) to the index of non-inferiority (−0.1549 = log10(0.7)).
Fewer than 25 pairs of samples for each organism were collected if there was at least a 95% probability of rejecting the hypothesis using fewer samples; however, the minimum sample size was 10. An early stopping rule was established using the test statistic zn and critical values determined according to Jennison and Turnbull (14 [see p 207, eq 10.3]). The test statistic was calculated according to
where ̄d is the difference in the means between the log-transformed (log10) bacterial concentration determined from the MPN series using ScanRDI® minus the log-transformed concentration of bacteria determined using the reference sterility test method, n is the number of samples, and sd is the standard deviation of the paired differences.
Determination of Limits of Detection:
The limit of detection was the dilution level at which there is 97.5% confidence that there was less than 10% chance of detecting organisms in a sample. The 97.5 % confidence level stems from the upper limit of a two-sided 95% confidence interval for the probability of detecting an organism. The limit of detection was determined from repeated measures logistic regression using three dilutions (as above), five tubes per dilution, and the number of tubes indicating the presence of microorganisms. The predictor variable was the log10 of the dilution. Four thousand two-sided 95% confidence limits on the predicted probability of response were computed, and the log value producing the largest upper confidence limit less than or equal to 0.1 (10%) was selected as the limit of detection. The data analysis was generated using SAS/STAT software, Version 9.1 of the SAS System for Windows (15).
Robustness Evaluation:
Experiments were replicated in three locations (Alcon Research and Development Lab in Fort Worth, Texas; Alcon Quality Assurance Lab in Fort Worth, Texas; and Alcon Quality Assurance Lab in Kaysersberg, France; 120 tests per site) with three teams of analysts. An analysis of variance was used to estimate site-to-site variability. The model is given by
where Yl(ijk) is a measurement of log MPN, μ is an intercept, Ti is the fixed treatment effect, γj is a random effect for sites, δk(j) is a random effect for treatments within sites, and ϵl(ijk) is a random effect for the individual measurements of log10 (MPN). The random effect for sites, γj, is assumed to be normally distributed with a mean of zero and a variance for sites of σγ2, which is tested to be zero. If the site-to-site variance is not significantly greater than zero, then the statistical inference may be considered robust to variation among sites.
Results
The results presented in this report indicate that the ScanRDI® method is numerically superior and statistically non-inferior to the reference sterility test method with respect to the limits of detection for all of organisms evaluated (Table II). The differences between mean MPN values were always positive, indicating that the ScanRDI® method was consistently better at detecting viable organisms at exceedingly low concentrations (Figure 1). However, the confidence interval around the mean difference for Clostridium sporogenes includes zero, which indicates that both methods are at least equivalent in their ability to detect this organism. Since the lower one-sided 95.25% confidence limit on the paired difference for each organism was greater than −0.1549, the acceptance criteria were satisfied and the non-inferiority null hypothesis of inferiority was rejected. Additionally, the robustness of the ScanRDI® method was assessed using analysis of variance to estimate the site-to-site variability and determine if it was significantly different from zero. The site-to-site variance component in this analysis measures the method's consistency among laboratories. The results of this analysis are presented in Table III and indicate that the ScanRDI® sterility test method remains superior or equivalent to the reference sterility method in the determination of mean MPN after adjusting for variation among sites. Moreover, the site-to-site variability in MPN measurement is not significantly different from zero for any organism tested in the limit of detection study.
Paired T-Tests of Non-Inferiority of the ScanRDI® and Compendial Sterility Test
Difference between mean concentrations of bacteria detected by ScanRDI® and the compendial methods for each of eight bacteria. Data points falling within the shaded region would indicate that the ScanRDI® method was inferior to the compendial sterility test method for detecting contaminants. Error bars represent the two 1-sided 95% confidence intervals.
Analysis of Variance for Robustness Evaluation
Because the study was designed to estimate MPN, the data could not support a comparison of accuracy, precision, and specificity between methods, nor could a false-positive rate be established. However, the data indicate that the overall variability (as range in percent positives) of the reference sterility test method appeared to be greater than that of the ScanRDI® method when microbes were inoculated into an MPN series at a cell concentration of 3 cells/mL−1. The overall variability of the reference method appeared to decrease when cells were present at lower concentrations (0.03 cells/mL−1). However, this may be misleading because the reference method rarely recovered bacteria at low cell abundance (high frequency of zero values) but cells were more often recovered by the ScanRDI® (Figure 2, percent positives greater than zero). For example, when all eight organisms were considered, the percentage of positive MPN tubes containing 0.03 bacteria/mL−1 was 2.12% for the reference method but 17.88% for the ScanRDI®. The difference in the ability of the two methods to recover bacteria is clearly revealed in Figure 2. Not only was the ScanRDI® better able to reveal the presence of bacteria when they were very rare, analysis of the data indicate that the limit of detection of the ScanRDI® is about an order of magnitude lower than that of the reference sterility test method (Table IV).
Percent of culture tubes showing the presence of microorganisms at three dilution levels. Open symbols depict data collected using the ScanRDI® method, while closed symbols depict data collected using the reference compendial sterility test method. Values alongside the names of organisms in the legend indicate the total number of tubes tested for a given organism. Horizontal lines (dashed or dotted) depict the overall mean for all organisms at a given dilution.
Comparison of the Limits of Detection (cells/mL−1)
Discussion
One of the challenges for any sterility test is the ability to detect or recover contamination, even when contaminants are exceedingly rare. In this study we established initial conditions in which the concentration of bacteria ranged more than 100-fold, from common (3 cells/mL−1) to rare (0.03 cells/mL−1). The ScanRDI® and the reference sterility test methods were used to indicate presence/absence of growth in a five-tube MPN procedure. Microbes were enumerated using an MPN procedure because the MPN method produces a quantitative measure of the concentration of microorganisms in a sample even when microorganisms are present at very low concentrations. The microorganisms chosen for comparison were those recommended by USP/EP/JP pharmacopeias and supplemented with two additional organisms, E. coli and P. acnes. E. coli was included because it is recommended as a test organism for total viable aerobic count (16), and P. acnes was included because it is a slow-growing, anaerobic to aerotolerant bacterium that is frequently associated with contamination from human flora. As such, the aerobic nature of the ScanRDI® assay method imposes a severe metabolic stress on this organism. It is clear that the limit of detection for P. acnes was not adversely affected by the additional metabolic stress. We thus employed a range in the types of organisms including anaerobes and aerobes, Gram-negatives and Gram-positives, those with fermentative metabolism and those with respiratory metabolism, and prokaryotes and eukaryotes.
In a recent commentary, Moldenhauer and Sutton (1, 17) outlined some of the flaws associated with the standard sterility test. They point out that it is well within the limits of current technology to conduct sterility testing without the necessity of culturing organisms and the attendant problems of culturing techniques (medium type, incubation conditions, detection limits, etc.). The ScanRDI® effectively avoids many of the problems associated with detection of microbes by culture approaches. It links two metrics of cell viability—the action of nonspecific esterases and the presence of an intact-functioning cell-membrane—with the production of a fluorochrome. Cells containing the fluorochrome are easily detected by laser and subsequently detected (and enumerated) by computer. The approach is rapid and sensitive and allows for the detection of a single cell on the surface of a 25-mm membrane (5).
When routinely used, the ScanRDI® requires neither lengthy incubation times nor specific culture media. However, by combining requirements of the reference sterility test (specific organisms, growth in specific media, incubation conditions, and duration) with an MPN enumeration protocol we were able to directly compare the two approaches to sterility testing. The ScanRDI® method for detection of microbes was demonstrated to be statistically non-inferior to the reference sterility test and numerically superior in that it had a likelihood of detecting microbes that was significantly greater at all dilution levels. As such, the ScanRDI® method is appropriate for use as a rapid alternative to the growth-based sterility test method.
Footnotes
- Received July 14, 2009.
- © PDA, Inc. 2010
References
Jump to section
Related Articles
Cited By...
- A Rapid Sterility Method Using Solid Phase Cytometry for Cell-Based Preparations and Culture Media and Buffers
- Rapid Sterility Test Systems in the Pharmaceutical Industry: Applying a Structured Approach to Their Evaluation, Validation and Global Implementation
- Development of next-generation sequencing-based sterility test
- Microbiological Test Data--Assuring Data Integrity
- Survival, Persistence, and Isolation of the Emerging Multidrug-Resistant Pathogenic Yeast Candida auris on a Plastic Health Care Surface
- Evaluation of a Rapid Microbiological Method with a Mixed Culture Biofilm Model
- Validation of the BacT/ALERT(R) 3D System for Rapid Sterility Testing of Biopharmaceutical Samples
- Total and Viable Legionella pneumophila Cells in Hot and Natural Waters as Measured by Immunofluorescence-Based Assays and Solid-Phase Cytometry