Transformation in the Pharmaceutical Industry: Transformation-Induced Quality Risks—A Survey

Piloting the Survey (Mixed Method—Qualitative and Quantitative):

The aim of the pilot survey was to (i) assess validity and reliability of the pilot questionnaire, and (ii) assess the robustness of the data analysis method selected for the main survey. The design approach was based on cognitive interviewing using verbal probing technique. Data were collected using the interview notebook. Collected data was analyzed using qualitative description of the emerging themes, quantitative description of the classified observations, and quantitative analysis of the responses to the questionnaire. The expected outcome was to create the main survey questionnaire by improving the pilot questionnaire and confirmation that the selected data analysis method was appropriate for the main survey.

Conducting the Main Survey (Quantitative Method):

The aim of the main survey was to solicit expert opinion on the relationship between the transformation-induced quality risks and regulatory compliance outcomes. Survey design was based on relational, non-experimental, fixed method (4). Data were collected using the main survey questionnaire (questions are listed in Table I). Collected data were analyzed using descriptive statistics and Microsoft Excel functionality.

Piloting the Questionnaire:

The questionnaire was piloted using the cognitive interviewing method (7) by interviewing participants recruited from the researcher's organization. During piloting the questionnaire, the cognitive interviewing method was applied using the verbal probing technique (7). The focus of the verbal probing was the survey questions. A 1 h interview with each participant was performed during which the participant answered the survey question; the interviewer then asked for other specific information relevant to the questions or to the specific answer given. In general, the interviewer probes further into the rationale and basis for the response. The key benefit was to improve the pilot questions and hence the main survey questionnaire by exploring issues relating to participant comprehension and identify structural problems such as erroneous skip patterns (procedures that direct respondents to answer only those items relevant to them) and unclear layout during the interview process.

The pilot interview notebook was used to collect the observations. The notebook contained the questions, the participant's response to the questions, and classification of any comments (see Table II) that the participant made related to the question or the survey procedure. This interview captured two types of data, namely, participant responses to the survey questions and participant comments to the assessment of survey reliability and validity. The main survey did not start until the piloting activity was completed and the ensuing improvements had been implemented.

Reliability and Validity Assessment:

1. I) Reliability of the data collection method: pilot data relating to participant comments were analyzed with the aim of improving the reproducibility of the survey questionnaire. As a comparative exercise the same data were analyzed statistically using Cronbach's alpha (8).

2. II) Validity of the data collection method: the aim of the validity assessment was to improve fitness of the questionnaire for its intended use. This involved assessment of validity with respect to questionnaire content, structure, and participant sampling (external validity). External validity was improved using the purposive sampling method, construct validity was assessed and improved during cognitive interviewing, and content validity was derived from the literature.

3. III) Validity of the data analysis method: the participant response data captured during piloting activity was analyzed to confirm the appropriateness of the selected data analysis method.

4. IV) Qualitative description of the emerging themes: the data from the cognitive interviewing were categorized into themes, which in turn informed the actions needed to improve the questionnaire and the associated survey procedures.

Questionnaire Anonymization:

The questionnaires were anonymized according to the a pseudo-code procedure and a link file containing the participant details and the corresponding anonymized code.

Sampling Frame:

Sampling frame is the source of the eligible population from which the survey sample was drawn. Potential candidates for this study were recruited from the organizations listed in Table III. These organizations were representative of pharmaceutical experts who gather and formulate solutions to challenging regulatory problems and publish their work. Key criteria for selecting the four organizations listed in Table III were (i) focus on pharmaceutical science, technology, and regulations in the context of drug development, approval, and manufacturing; (ii) diverse membership that included industry professionals, industry service providers, regulatory agencies, and academia; (iii) active involvement in regulatory science topics; and (iv) a specific focus on pharmaceutical quality from the perspective of good laboratory, manufacturing, clinical, or pharmacovigillence practice and risk management. Inclusion of consulting professionals from service providers to the pharmaceutical industry was important for enhancing sample diversity because these professionals experience a wide range of industry practices during their service offerings.

Inclusion Criteria for the Main Survey:

Candidates meeting the following criteria were selected for the survey:

• Those who had quality and compliance knowledge in good laboratory, clinical, and/or manufacturing practice and

• Those who had experience with U.S. Food and Drug Administration (FDA) regulations and/or E.U. European Medicines Agency (EMA) regulation and

• Those who had current working knowledge of quality relevant to medicinal products based on pharmaceuticals and/or biologics

Participants were allowed to withdraw from the study at any time.

Definition of Key Terms:

Interval: Grouping of participant scores in the Likert scale:

Don't Know 0 to 100 (the participant does not know enough to respond but the potential answer could be within the full range of the scale)

Binary: Yes/No 100/0

Midpoint: Midpoint value of each interval

PRC: Participant response count per interval per question

Mean: Average score for each question

Dataset: Refers to participant scores for questions in the questionnaire that use Likert scale for measurement (questions 10 to 30, except 28)

Dataset Mean: Average score for the dataset

PDF: Participant response distribution (probability density function)

Correlation: Strength of relationship between two variables

Covariance: Measure how the independent and dependent variables change together

Excel: The following Excel functions or menu options were used during the data analysis: AVERAGE, CORREL, COUNTIF, Data>Sort>, MEDIAN, NORMDIST, STDEV, SUM, VAR, COVAR (refer to 2003 Microsoft Excel help for detailed definition: http://office.microsoft.com/en-us/excel-help).

Calculation Procedure for Descriptive Statistics:

The data captured within the pilot and main questionnaires were converted from frequency centric score to a standardized score for analysis.

• PRC = COUNTIF ([[[Score, “=x”]^{1 to Q}]^{1 to I}]^{1 to P}) … where Q = number of questions in the dataset, I = number of Intervals, and P = number of participants

• Standardized Score = ([[[PRC * Midpoint]^{1 to Q}]^{1 to I}]^{1 to P})

The central tendency of the participant scores both at the individual question level and for the whole dataset was calculated as follows:

• Mean = SUM ([[Standardized Scores]^{1 to Q}]^{1 to I})/SUM ([[PRC]^{1 to Q}]^{1 to I})

• Dataset Mean = AVERAGE ([Mean]^{1 to Q})

The standard deviation (SD) was calculated in order to test if the dataset was normally distributed. Data points above and below the dataset mean (± 1, 2, 3 SDs) were computed. The corresponding probability density function for each of the seven data points was calculated.

• Variance = VAR ([[Standardized Score]^{1 to Q}]^{1 to P})

• SD = STDEV ([[Standardized Score]^{1 to Q}]^{1 to P})

• Data Point m = Mean + (n * Standard Deviation) where m = 1 to 4 and n = 0 to 3

• Data Point m = Mean − (n * Standard Deviation) where m = 5 to 7 and n = 1 to 3

• [PDF]^{1 to 7} = NORMDIST ([Data Point]^{1 to 7}, dataset Mean, dataset SD, False). False is chosen because the data is discrete. For continuous data, TRUE is used.

The reliability of the data collection method (internal consistency of the pilot survey questionnaire) was tested using Cronbach's Alpha (8). A pair-wised covariance matrix was set up with both rows and columns containing the survey questions. Covariance of each question to itself (self-covariance) and with other questions was calculated.

• Covariance = COVAR ([[Normalized Score]^{1 to Q}]^{1 to P})

• Self-covariance = SUM ([self Covariance]^Q,Q)

• All Covariance = SUM ([[Covariance]^{1 to Q}]^{1 to Q})

• Cronbach's α = (Q/(1 − Q)) * (1 − (self-covariance/all covariance)), for all datasets

The impact on the reliability coefficient was determined by recalculating the Cronbach's alpha for the dataset without including Covariance of each question at a time. The intent was to determine absence of which question had the biggest positive or negative impact in reliability of the pilot survey questionnaire.

The sum of participant comments on the questions mapped against each transformation trigger was considered as the primary indicator of participants' level of interest on a given trigger and hence its perceived importance. This survey variable was therefore used in determining strength of correlation between the survey results and the corresponding theoretical and operational evidence (2, 3).

The correlation between the pharmaceutical transformation triggers (2) and the main survey questions was determined as follows:

• Mapped main survey questions to pharmaceutical transformation triggers*

• Found sum of participant comments to mapped questions corresponding to each trigger

• Tabulated the operational evidence strength corresponding to each trigger*

• Tabulated the theoretical evidence strength (i.e., transformation trigger ranks*), operational evidence strength, and expert opinion strength (i.e., sum of participant comments for mapped questions) in a correlation matrix

• Found the correlation between the expert opinion strength and each of the other arrays within the matrix. In this case correlation rather than covariance is used because the measurement scale for the variables is in different units.

• Correlation_{(trigger/survey)} = CORREL ([expert opinion strength]^{1 to T}, [X]^{1 to T}), where T is the number of transformation triggers = 14 and X = arrays of “theoretical evidence strength” or “operational evidence strength”

*The above procedure steps with the asterisks indicate that these steps were established during an earlier part of the research (2, 3).

The covariance between the transformation-induced quality risks and the regulatory compliance outcomes for the main survey was determined as follows:

• Calculated individual mean of main survey questions for the transformation-induced quality risks (independent variables)

• Calculated individual mean of survey questions for the regulatory compliance outcomes (dependent variables: i.e., pre-market evaluation, marketing approval, and post-market surveillance)

• Computed covariance between the independent and dependent variables

• Covariance = COVAR(( [Quality Risks]^{1 to I}, [Y]^{1 to D}). …where I and D are the number of main survey questions relating to independent and dependent variables = 4 and Y = “pre-market evaluation” or “marketing approval” or “post-market surveillance”

Summary of Participant Remarks:

Quality risk on the one hand and overly prescriptive standard promulgation and an exaggeration of risk on the other were identified as opposing opinions and additional drivers of industry transformation. One participant stated that “open innovation trends currently practiced in the industry include virtual organizations, contract manufacturing, and professional consortiums”. Other areas of risk identified by participants include (i) duration of research partnerships versus duration of the product development lifecycle, (ii) supply chain management in its broadest context, (iii) management understanding of quality rather than just compliance, and (iv) product adulteration and drug counterfeiting.

Summary of Participant Remarks:

Concerning the impact of open innovation on external partner/alliance selection, two participants suggested that companies were already downsizing and outsourcing various activities and that more scrutiny in partner selection was needed. One participant thought that “legal framework for open innovation should probably be established” but was “not sure if it would happen”, and another participant had an opposing view, stating that “legal framework is pretty well developed already”. Regarding the prevalence of biological/biotechnology products, one participant believed that this was already part of project and product portfolios of pharmaceutical companies and this is a trend that will accelerate. In contrast, another participant suggested that biological/biotechnology products tended to be highly expensive targeted drugs and current trends in health care management may not embrace these products as a first line of therapy. On multidisciplinary knowledge/skills, one participant stated that “what is required from a compliance point of view are more individuals with medical experience making judgments regarding patient care and less involvement of bureaucrats to encourage peer review and ensure attention to fundamental scientific principals”. On regulatory approach to compliance and innovation, one participant suggested that “compliance is all too often a self-fulfilling prophecy in which a perceived problem is blown out of proportion to risk. Overbearing regulation for years has been used to control respective markets leading to difficulty in introducing newer technologies particularly those that rely on unique drug delivery systems, are customized to a specific patient, or are multifaceted in one way or another. Overbearing regulation would result in a loss of innovation, the swallowing up of smaller, entrepreneurial organizations, drug shortages of some medicines, and perhaps even the curtailment of generics”. Another participant focused more on the legal aspect, stating that “concern about adverse publicity and legal issues (potential class action lawsuits) are leading to conservative decision making in portfolio management and regulatory review”. Another participant summarized the regulatory approach as “the bar for safe and effective is increasing”. One participant stated that “regulators have no incentive in making anything easier for anyone” and was “not optimistic that the regulatory apparatus which has evolved in Europe and the USA in particular can support an innovative environment”. Concerning regulatory initiatives, one participant opined that “these initiatives may work if regulators provide flexibility in regulatory filings”. A couple of the participants expressed a more skeptical tone, opining that regulation never leads to innovation or innovation drive, regulation should focus on one thing and one thing only, making sure that there is a supply of safe medicine for everyone and that new ideas gain market access at the appropriate pace. Such initiatives to date have tended to be largely political and seem to lack the strategic partnership needed with all sectors of the pharmaceutical industry (i.e., Big Pharma as well as Generics).

Summary of Participant Remarks:

On GxP due diligence for external partners/alliances, one participant commented that “all the phases of the product lifecycle were at risk as long as the focus remains on regulatory compliance rather than real compliance”. The participant defined real compliance as “developing and manufacturing safe and effective medicines from a strong ethical base focusing on combating unethical and criminal elements of the health care product supply chain”. With respect to product transfer, one participant stated that “potential GxP compliance risks will depend a lot on quality of product transfer planning and execution”. Another participant stated that “the GxP compliance was a moving target which often has absolutely nothing to do with product safety”. One participant proposed “inadequate knowledge management and process characterization continue to be major roadblocks in early commercialization”. One comment focused on contractual process expressing a great deal of concern during the technology transfer process from one organization to another and that the “contract giver discovers it takes much more time and resource than they had imagined”. Another comment attributed inadequate knowledge of dosage form processes to “poor inspections outcomes stating that there is a direct relationship between industry implementation of innovative technology and more regulatory scrutiny”. Concerning multidisciplinary regulatory knowledge/skills, one participant stated that “the compliance risks are high after the initial product approval, firms' compliance and quality systems trend to deteriorate during the commercial manufacturing”. Regarding complexity of the biological/biotechnology product characterization, a couple of the participants opined that the process is the product and in some cases they are more difficult to characterize chemically or biochemically, but in some cases they aren't, further explaining that “complex biologics have been used safely for decades; there is no reason to be hung up on complexity”. One comment focused on regulators' understanding of contamination control, stating that the “recent drive to manufacture even non-sterile drugs in classified clean rooms as a complete waste of money and regulatory effort”. Another comment attributed the majority of regulatory product recalls to “poor understanding of product performance long term, especially with product component interactions”.

With respect to externalization of GxP data management, one comment stated that “some aspects of data collection and storage has been in place for decades and compliance problems have been comparatively rare”, and another comment focused more on use of social media in the industry, implying that “it may present a larger quality and compliance risk than internal data management systems”. Concerning technology validation, one comment addressed the new technology adoption in the industry, saying that the “pharmaceutical and biopharmaceutical industries have lagged behind other technological industries in the adaptation of modern information and process management technologies”. The commenter attributed this lag largely to “misplaced regulatory concerns and unfortunate regulatory requirements”. Another comment focused on computer validation approaches, noting that they “have not changed significantly in 20 years and therefore the current methodologies are unprepared for use of new ‘cloud based' computing”.