Comparison of Different Calculation Approaches for Defining Microbiological Control Levels Based on Historical Data

Abstract

In the present work we compared different calculation approaches for their ability to accurately define microbiological control levels based on historical data. To that end, real microbiological data were used for simulation experiments. The results of our study confirmed that assuming a normal distribution is not appropriate for that purpose. In addition, assumption of a Poisson distribution generally underestimated the control level, and the predictive power for future values was highly insufficient. The non-parametric Excel percentile strongly predicted future values in our simulation experiments (although not as good as some of the parametric models). With the limited amount of data used in the simulations, the calculated control levels for the upper percentiles were on average higher and more variable compared to the parametric models. This was due to the fact that the largest observed value was generally defined as the control level. Accordingly, the Excel percentile is less robust towards outliers and requires more data to accurately define control levels as compared to parametric models. The negative binomial as well as the zero-inflated negative binomial distribution, both parametric models, had good predictive power for future values. Nonetheless, on basis of our simulation experiments, we saw no evidence to generally prefer the zero-inflated model over the non-inflated one. Finally, with our data, the gamma distribution on average had at least as good predictive power as the negative binomial distribution and zero-inflated negative binomial distribution for percentiles ≥98%, indicating that it may represent a viable option for calculating microbiological control levels at high percentiles. Presumably, this was based on the fact that the gamma distribution fitted the upper end of the distribution better than other models. Since in general microbiological control levels would be based on the upper percentiles, microbiologists may exclusively rely on the gamma distribution for calculation of their control levels. As the gamma distribution can conveniently be calculated in standard office calculation software, it may represent a superior alternative to the widely used percentile functions or other distribution models.

LAY ABSTRACT: During the manufacturing of pharmaceutical drug products, the counts of microorganisms are monitored in the cleanroom environment, water, the product's raw materials, and the final product. This enables manufacturers to ensure that high numbers of microorganisms that may impair the product's microbiological quality are detected before the product is released to the patient. Microbiological control levels must be set to determine at which number a count is considered too high. Exceeding such levels may require an investigation to determine the root cause explaining why such high numbers of microorganisms occurred, and a set of actions should be performed with the aim of eliminating this root cause. In order to really differentiate higher-than-usual counts, microbiological control levels should be based on historical data. In the present work we analyzed different calculation approaches towards that purpose. We used real microbiological data and performed simulation experiments to determine which statistical method could calculate the most realistic control levels that would provide the best prediction for future routine testing. Better predictions would ensure that only significant contaminations lead to an excursion of the microbiological control level, which would avoid wasting resources by investigating non-issues or normal/controlled conditions.