Abstract
The Pharmaceutical Regulatory Science Team (PRST), a research team based at the Dublin Technological University (TU Dublin) in Ireland, recently conducted a Quality Risk Management (QRM) survey and a face-to-face focus group workshop to assess the level of formality of QRM roles in the biopharmaceutical sector. This was carried out as part of a research study, which identified the need for the development of QRM role-based competencies as fundamental to realizing QRM's benefits. The research study followed a hybrid Delphi research methodology composed of: (1) Survey 1, (2) focus group workshop, (3) Survey 2, and (4) competency model development. This paper presents the results of Survey 1 and the focus group workshop. Survey 1 explored the need for QRM role-based competencies and the subsequent face-to-face focus group workshop built on this to propose initial standard QRM roles, with a view to confirming these and developing associated competencies. This paper presents the findings from Survey 1 and the focus group workshop. The results of the follow-up research activities will be presented in a subsequent paper.
LAY ABSTRACT: The publication of the ICH Q9 Quality Risk Management (QRM) guideline in 2005 has greatly impacted the biopharmaceutical sector. Fourteen years after Q9, the benefits of QRM are yet to be realized. The biopharmaceutical sector still struggles with the implementation of Q9 principles to effectively assess and manage product quality risks as a surrogate for patient safety.
This paper looks at the need for standard QRM roles and the associated competencies for those roles.
1. Introduction
In 2005, the International Conference on Harmonisation's (ICH's) Guideline Q9 (1), aimed at providing the pharmaceutical industry with guidance toward a science- and risk-based approach to quality management, was published.
The purpose of ICH Q9 is to offer a systematic approach to quality risk management (QRM), providing guidance on the principles and tools of QRM that can enable more effective and consistent risk-based decisions across the product life cycle. ICH Q9 states that QRM activities are typically, but not necessarily, undertaken by interdisciplinary teams dedicated to the QRM process. It recommends that teams for specific QRM activities include expertise from the technical areas involved, as well as individuals who are knowledgeable of the QRM process.
Another ICH guideline, ICH Q10 (2), outlined the key requirements of a pharmaceutical quality system (PQS) in which QRM together with knowledge management (KM) were identified as enablers.
ICH Q10, represented in Figure 1, depicts the pharmaceutical product life cycle in four stages and identifies the key elements, together with the two enablers (KM and QRM).
Diagram of the ICH Q10 Pharmaceutical Quality System Model.
The four stages—Pharmaceutical Development, Technology Transfer, Commercial Manufacturing and Product Discontinuation—in the ICH Q10 diagram have commonly become known as the “end-to-end product life cycle”, and for the purpose of this article will be referred to as such.
At the start of this research study, a review of QRM literature indicated that application of formal QRM methodologies across the good manufacturing practice (GMP) environment is inconsistent. Friedman (3) in 2015 suggested that this is due to variations in the interpretation of the high-level conceptual document (ICH Q9) resulting in its projected benefits not being realized.
This PRST research team is interested in understanding and remediating the knowledge gap and the lack of clarity regarding their roles and responsibilities of those involved in QRM, especially when it comes to risk control and decision-making.
In addition, the research team aims to develop a QRM competency model for individuals involved in the end-to-end product life cycle by initially answering the following questions:
What are the roles and responsibilities of those involved in the end-to-end product life cycle necessary to achieve QRM effectiveness?
What are the competencies associated with each of the individual responsibilities?
A hybrid Delphi research methodology as described in Section 3.1.1.1 was selected to explore these questions, and the results of the research are presented in this paper and a follow-up paper.
The research team initially focused on understanding the scope of specific “QRM” roles in organizations by way of a survey of 22 experienced QRM practitioners located across the globe.
This was followed by a face-to-face focus group workshop, where the need for standard QRM roles in the biopharmaceutical sector was identified and discussed. In the course of this workshop, it became apparent that for QRM to be truly effective, it must be the responsibility of not only dedicated QRM practitioners but also all the individuals involved in the end-to-end product life cycle. Indeed, one could aspire to have QRM responsibilities codified in the job descriptions of all of these individuals.
2. Background to the Research
Waldron, in her thesis titled “Managing risk to the patient: Recoding Quality Risk Management for the pharmaceutical and biopharmaceutical industries” (4), discussed how 13 years after ICH Q9, a review of FDA warning letters to the pharmaceutical industry through the lens of QRM revealed an increasing trend of citations by the regulators against the QRM programs and practices themselves.
Waldron's body of work focused on organizational QRM maturity and did not explore in detail how individuals can contribute to QRM effectiveness. Waldron identified the need for role-based competencies in QRM and acknowledged that the traditional training model currently being employed in the industry is inadequate for building such competency and developing a mature state in the management of patient risk.
In addition, Waldron recognized that not all QRM practitioners require the same level of training and that competencies should be based on the practitioner's level of involvement in QRM. Although all employees should be aware of the QRM principles and practices, Waldron proposed that employees in specific roles should have customized training based on their level of interaction with QRM. This is also supported by Greene and Calnan, who identified that, although training is critical to QRM, only the basic concepts to execute a task are taught (5).
Throughout the course of this research, the thinking has evolved as it became apparent that QRM is the responsibility of all individuals involved in the end-to-end product life cycle.
This concept is further supported by O'Donnell. During a seminar held at the Dublin Institute of Technology in October 2018 titled “An Audience with International Regulators in the Manufacture of Medicines—Quality Risk Management & Knowledge Management”, O'Donnell (6) discussed the challenges in the areas of risk-reduction measurement and QRM competency development and also explored the benefits of demonstrating the effectiveness of the QRM activities that are undertaken by companies in the biopharmaceutical sector. Several examples of serious quality defects and recalls in 2018 were presented at the seminar as shown in Table I.
Examples of Quality Defects and Recalls in 2018a
O'Donnell then highlighted the fact that most of the quality defect investigations performed yearly by the Irish regulatory body, the Health Products Regulatory Authority (HPRA), had three common factors:
The defective batches in question were manufactured using qualified equipment.
They were manufactured by trained and qualified staff.
They were manufactured using processes that had been validated earlier.
The continuous occurrence of serious quality defects in the biopharmaceutical sector is a clear indication of the lack of QRM within the good manufacturing environments. O'Donnell also reflected on GMP inspectional findings and observations that relate to QRM (6), such as:
High levels of uncontrolled subjectivity in risk assessments.
Risk assessments that do not reflect the process that is being risk assessed.
The misapplication of QRM in Deviation Investigations and in decision-making.
O'Donnell stated that “the benefits of QRM are probably not being realized for patients—at least not to the extent envisaged by ICH Q9” (5).
Looking toward 2020, O'Donnell presented three strategic recommendations worth considering in the effort to improve how QRM principles are applied within the GMP environment, one of which was to develop a Certification Program for QRM which aligns directly with this research study.
3. Research Methodology
3.1 Research Design
According to the Enterprise Risk Management Academy (ERMA)1 (7), people are competent when they can apply their knowledge and skills to successfully complete their work activities in a range of situations and environments to the standard of performance expected in the position that they occupy. Training and knowledge assessments aim to make sure that the individuals participating in the training and the assessment have the competence to take their role to the expected standard set by their management.
Thus, it is logical that to determine whether those involved in QRM are competent, there needs to be established standards that they can be judged against, and this research seeks to develop such standards in consultation with QRM experts. The research method selected to do this is the Delphi Method.
3.1.1 Delphi Research Methodology:
The Delphi research method is commonly used in academic research as a method to reach consensus among experts; it has an advantage over traditional group meeting scenarios, in that it does not allow one person's influence to sway the group opinion (8) and thus eliminates the potential for bias. It is a social research technique that seeks to obtain a reliable group opinion from a set of experts who can provide valuable aid in solving a complex problem.
Hybrid Delphi, also known as modified Delphi, is a specialized type of Delphi that differs in its structure and objectives from other techniques that are based on the judgment of experts, and it is especially used within the fields of life and health sciences (9).
According to Landeta (10), a Hybrid Delphi considers the needs of the researcher as well as the needs of experts to improve the effectiveness of the preceding techniques in achieving the scientific and social objectives of the study. In general, members of the Delphi study expert focus group are busy executives in their organizations, and this method is suitable for collecting and combining the knowledge of experts who wish to collaborate in an activity that for them is secondary (10). This very accurately describes the QRM practitioners selected for this study.
An overview of the theoretical and methodological frame for the Hybrid Delphi research used by the researcher is shown in Figure 2. It depicts the overall research model starting with the researcher's QRM prior experience and publications, an extensive literature review of QRM publications, as well as a review of competency models from different sectors. The model also shows the progression of the research starting with a Delphi 1 survey, a focus group workshop, a Delphi 2 survey, after which the researcher will publish the QRM competency model.
This paper focuses on the Delphi 1 design survey and the face-to-face focus group workshop as highlighted in Figure 2.
Hybrid Delphi Process.
3.1.2 Delphi 1 Survey Structure:
The survey, which consisted of 10 questions, was designed to gain insights into the formality of QRM roles in the respondents' organizations. It was developed in consultation with academics and an HPRA2 GMP Inspector. It must be noted that the Delphi 1 survey was designed at a time in the research study before it was proposed that QRM was the responsibility of all individuals involved in the end-to-end product life cycle. The focus of the Delphi 1 survey was specifically on QRM practitioners' roles and QRM programs and the language and questions reflected this. The term “formal” was describing specific QRM practitioner roles, whose sole responsibility involved QRM. The questions are presented in Table II.
Delphi Survey 1 Questionnaire
3.1.3 Delphi 1 Survey Sample Group Selection and Results:
The researcher believed that deep inquiry into the opinions of key QRM thought leaders was necessary to drive the research forward. The researcher solicited the opinions of QRM Program Leads from various companies in the biopharmaceutical sector, as well as well-known authors who have published on the subject of QRM. The survey was sent to 30 respondents who worked in the sector, including five GMP inspectors.
The researcher received responses from a total of 22 respondents, including 4 GMP inspectors and 18 QRM practitioners.
Question 1: Do you have formal QRM roles within your organization? If yes, please outline.
Question 1 was designed to gain an understanding of how explicitly defined the QRM roles are at each of the respondents' organizations. The results are shown in Figure 3, and can be summarized as follows:
59% stated that they do have formal QRM roles, while 4% stated that they have formal roles in some areas but not in other areas.
23% of the respondents stated that they do not have formal QRM roles within their organizations.
14% did not respond to this question.
Percentage of responses to Questions 1 of the Delphi 1 Survey Questionnaire.
In addition to indicating whether they have formal QRM roles within their organizations, the respondents were also asked to provide the actual titles of these roles. Roles that occurred in more than one organization are listed below in the order of highest to lowest.
QRM Facilitator
Subject Matter Expert
QRM Risk Lead
Systems and Process Owner
Decision Maker
Quality Participant
QRM Site Lead
QRM Program Manager
Question 2: Are you now or have you ever been involved in implementing a QRM program within an organization?
Question 2 was designed to get a snapshot of the respondents' level of QRM experience. The results are shown in Figure 4 and can be summarized as follows:
91% of the respondents have been involved in implementing a QRM program within an organization.
9% have not been involved, as of the date of response.
Percentage of responses to Questions 2 of the Delphi 1 Survey Questionnaire.
As the participation of experts is a fundamental premise of the Delphi study, the fact that 9% of the respondents indicated that they have not been involved in implementing a QRM program within their organization was unexpected. The Delphi 2 survey that will be presented in the next paper broadened the number of QRM experts to address this issue. However, it important to note that the 9% was likely composed of academics and regulators who are not in a position to implement QRM programs.
Question 3: In your opinion, to what degree is QRM embedded within the day-to-day operations of your organization?
Question 3 was intended to explore the degree of integration of QRM into the quality systems as intended in ICH Q10, Pharmaceutical Quality Systems (10). However, in hindsight and especially considering the current paradigm that proposes that QRM is the responsibility of all of the individuals in the end-to-end product lifecycle, all that can be concluded from the question is the respondents' opinions on the degree that QRM is embedded in their organization. The results are shown in Figure 5 and can be summarized as follows:
50% of the respondents stated that they believed that QRM is fully embedded in their day-to-day operations.
27% of the respondents stated that they believed that QRM is partially embedded in their day-to-day operations.
9% of the respondents stated that they believe that there is variability in the level of QRM use in their organization.
9% of the respondent did not answer this question.
5% of the respondents stated that they believed that QRM is informally embedded in their day-to-day operations.
Percentage of responses to Questions 3 of the Delphi 1 Survey Questionnaire.
One of the respondents stated that there should be no specific QRM roles and responsibilities as QRM is a behavior that one works to acquire. This response stimulated the thought process that led to the hypothesis that QRM is the responsibility of all individuals in the end-to-end product life cycle.
Question 4: In your opinion, what are the key elements that contribute to an effective QRM program?
Question 4 was designed to gain an understanding of the opinion of the respondents about the key elements of an effective QRM program. It was an open question that allowed the respondents to provide input on what specifically they believed were key elements of an effective QRM program. Upon reviewing the responses, the researcher divided the 22 key elements shown in Figure 6 into three categories: (1) Senior Management/Leadership, (2) Program and tools, and (3) QRM training.
Key Elements that contribute to an effective QRM Program.
The responses identified many key elements of a QRM program; however, the focus of the research has evolved to explore at the level of individual responsibilities rather than at the program level, although it is still useful to reflect on these responses when designing individual competencies.
Question 5: Have you defined any specific QRM roles and responsibilities within your organization? If yes, please outline.
Question 5 was intended to seek clarity on the types of roles. However, the respondents addressed this in their answers to Question 1. Thus, the results of Question 5 will not be further discussed.
Questions 6 and 7 explored skills associated with dedicated QRM practitioners. The results must be viewed through this lens.
Question 6: Have you defined the associated skills required for each role involved in QRM? If yes, please outline.
Question 6 was designed to gain an understanding of whether the skills required for those involved in QRM were defined. The results are shown in Figure 7 and can be summarized as follows:
52% of the respondents stated that they do not have defined skills for those involved in QRM.
24% of the respondents said that they have defined skills but only for formal roles.
19% did not answer the question.
5% of the respondents stated that they are still in the process of defining formal skills for each QRM role.
Percentage of responses to Questions 6 of the Delphi 1 Survey Questionnaire.
The researcher noted that those who responded that they have defined skills for formal roles were referring to technical QRM training only. The majority stated that they are in the process of assessing soft skills or behavioral requirements for the formal QRM roles.
Question 7: Do you have specific role-based curricula for those involved in QRM?
Question 7 was designed to gain an understanding of the existing specific role-based curricula for those involved in QRM. The results are shown in Figure 8 and can be summarized as follows:
50% of the respondents stated that they do not have QRM-specific role-based curricula.
36% of the respondents stated that they do have QRM-specific role-based curricula.
14% of the respondents answered with “not applicable”.
Percentage of responses to Questions 7 of the Delphi 1 Survey Questionnaire.
The results of Questions 6 and 7 suggest that if skills are defined for those who are involved in QRM, they tend to be only defined for those formal QRM roles for which there are already role-based curricula.
Questions 8: In your opinion, what other role/roles do you think are necessary to ensure an effective QRM program?
Question 8 was designed at the time to gain an understanding of QRM roles that do not yet exist in the respondents' organizations but that they believe are needed to ensure an effective QRM program. At this point it must be acknowledged that the focus of the Delphi 1 survey was on dedicated QRM roles and the QRM program. The inclusion of question 8 points toward an early but not yet fully developed hypothesis that QRM is the responsibility of all those individuals involved in the end-to-end product lifecycle.
The results are summarized in Table III.
The information in this table was valuable for determining the roles identified later in Table V.
Question 9: In your opinion, which behavioral traits do you consider important for those involved in QRM activities?
Question 9 was designed to gain an understanding of behavioral traits needed for those involved in QRM activities. The results are summarized in Table IV.
Behavioral Traits Identified by Delphi 1 Survey Respondents as Important for Those Involved in QRM Activities
Question 10: Do you believe that identifying specific QRM roles and competencies is essential to sustaining an effective QRM program?
Question 10 focused on specific QRM roles and the QRM program. While the focus of this study has evolved into looking at individual competencies, the results are worth reflecting on. See Figure 9. In response to Question 10:
68% of the respondents believe that identifying specific QRM roles and competencies is essential to an effective QRM program.
14% of the respondents do not think identifying QRM roles and competencies is essential to an effective QRM program.
14% of the respondents replied “depends”.
4% did not answer this question.
Percentage of responses to Questions 10 of the Delphi 1 Survey Questionnaire.
The lack of consensus on the need for specific QRM roles and competencies in order to have an effective QRM program contributed to the refocusing of the research from the QRM roles and the program to the individual competencies.
3.2 Focus Group Workshop:
The workshop goal was to gather expert opinion with a view to characterizing the competencies needed to further advance QRM by first defining the roles needed for a successful and effective QRM program. Then, based on the defined roles, a competency mapping exercise was carried out to solicit opinions on the key competencies needed for those involved in QRM based on their role in the organization.
3.2.1 Focus Group Participants:
Fourteen participants attended the face-to-face workshop: two GMP inspectors, three academics, and nine QRM practitioners.
3.2.2 Focus Group Output:
The workshop was facilitated by a facilitator who opened with a discussion about the importance of the research and the identification of QRM competencies. Envisaging the workshop to be a series of philosophical dialogues, the facilitator provided Steiner's definition: “Conversations with industry and regulatory experts to elicit important descriptions and narratives of people's experiences… [that] can also be employed as conversational ways of producing knowledge that has been justified discursively in a conversation” (11).
At the start of the workshop, the researcher shared the results of the Delphi 1 survey with the attendees in preparation for the discussion.
3.2.2.1 Challenges in Embedding Effective QRM:
The first discussion was designed to identify challenges that currently exist in effectively embedding QRM in day-to-day operations. Several challenges were identified, and the researcher has categorized them under the headings from Question 4 of Survey 1, with the addition of a new category, culture, which came to light in the philosophical dialog.
Senior Management/Leadership
Lack of senior management sponsorship and their failure to understand the complexity of the QRM process.
QRM Programs and Tools
There is no clear QRM effectiveness check. In the biopharmaceutical sector, there is a tendency to want to measure everything but there are challenges when trying to develop quantifiable metrics and key performance indicators and to identify the cost of failure to act.
The absence of technology is often seen as a roadblock to successful QRM implementation. The perception is that if the industry moves away from using Excel to perform risk assessment, then the risk assessments will take less time and their quality will improve, as if QRM success is tied directly to one using or not using a less sophisticated technological tool.
Training
QRM is often viewed as a “one off” activity as there is a lack of understanding of ICH Q9 in the light of ICH Q8 (12) and ICH Q10 (2).
QRM is thought of as an additional “task” instead of an enabler of Quality Systems. Often, there is a tendency to focus blindly on complying with the regulations without understanding the true nature of risk and without understanding how QRM can be applied effectively.
Risk is often a gray area, where scientists in the biopharmaceutical sector are not comfortable operating.
There is a lack of appropriate education, experience, and competencies in the QRM roles that currently exist in the represented sectors.
There is a lack of knowledge management (KM) transfer and linkage between QRM and KM.
Culture
A culture of proactive identification of risks does not exist; leadership often recognizes and rewards fire-fighting.
There is a cultural fear association with identifying risks
QRM is not seen as creating value. This is often the result of companies experiencing ineffective and onerous QRM applications with little value realized.
There is reluctance to having an internal level of transparency associated with potential risks, which often manifests as reluctance to perform risk assessment/identify risks, as if once risks are identified and written down they become a reality that needs to be acted upon.
Lack of regulatory inspection of QRM is often a problem in some quality cultures. There is a perception that if regulators will not be inspecting the topic that there is no incentive to commit to it.
The types and number of challenges identified suggest that the sector is still struggling with embedding effective QRM.
3.2.2.2 Exploration of Individual QRM Roles:
The next part of the focus group was a philosophical dialog that explored individual QRM roles. After a long discussion at the workshop and a detailed analysis of the transcript by the researcher in consultation with the research team, seven standard, individual QRM practitioner roles were identified and are presented in Table V. In light of the hypothesis that all individuals involved in the end-to-end product lifecycle have responsibility for QRM, all individuals are deemed to be QRM practitioners.
Standard Individual QRM Roles
4. Conclusion
The majority of the respondents agreed that clear roles and responsibilities for those involved in QRM are still needed.
Based on the results of the survey, this researcher strongly believes that characterizing the competencies needed to further advance QRM must begin with defining the roles needed for a successful and effective QRM program. Then, based on the defined roles, a competency mapping exercise should be conducted to identify which key competencies are needed for those involved in QRM based on their role in the organization.
The first part of the Delphi 2 survey, which is currently in progress, is validating and confirming these roles and identifying their associated competencies. The final competency model will reflect the feedback on these roles and suggest appropriate competencies. In addition, a standard taxonomy is being developed and will be published in a subsequent paper.
Conflict of Interest Declaration
The authors declare that they have no competing interests.
Acknowledgments
The authors thank Dr. Kevin O'Donnell for his invaluable input and Dr. Kelly Waldron for reviewing this paper.
Footnotes
↵1. ERMA is established as a global learning center for professionals in Enterprise Risk Management. ERMA provides a comprehensive set of Enterprise Risk Management courses and certifications, and is facilitating collaboration efforts of ERM professionals around the world.
↵2. Formerly known as Irish Medicines Board (IMB), became the HPRA in July 2014HPRA: is a state agency that puts the health of people and animals at the core of everything they do. They use scientific and clinical expertise to review and monitor health products available in Ireland or exported abroad.
- © PDA, Inc. 2019
