Hybrid Delphi: A methodology to facilitate contribution from experts in professional contexts
Research highlights
► The Hybrid Delphi is a valid technique for use in professional contexts. ► It draws the best from Focus Groups, the Nominal Group Technique and the Delphi method. ► It satisfies the needs of both the experts and the researchers taking part. ► It encourages creativity and reasoned assessment in experts' contributions
Introduction
What is the best way of extracting and processing the information possessed by a set of professionals concerning a problem or phenomenon? Since Helmer and Rescher [1], [2] laid the foundations for the scientific use of expert opinion in the areas of decision and prediction, different techniques have been developed over the years aimed at responding to the initial question, in different contexts and with different objectives. All these techniques have, in general, sought to contribute improvements in one or some of the fields of action originally delimited by Helmer [3]: 1- improvement in the selection of the most suitable information sources for each concrete case (relying on the appropriate experts), 2- assisting in the effective development of experts' activities, facilitating transmission of the information required, and 3- development of methodologies of action that make the processing of that information possible, achieving outcomes of greater quality.
This methodological development is still of relevance, because knowledge advances and because needs change with time. In this regard, one sphere of great scientific and social interest is the obtaining of ranked lists of problems, practices, solutions… regarding a specific phenomenon using the tacit knowledge of professional experts who are undergoing problems, carrying out practices or applying solutions in their respective contexts of action. By professionals we refer to experts who voluntarily collaborate in a forecasting or group decision activity, freely contributing their knowledge, which is normally a product of their usual working activity. From this category we exclude, therefore, students, scholarship holders and other “captive experts”. For this purpose different group techniques may be employed, some of which already have a broad academic history of application. However, those of us who apply them feel that there is much more knowledge that we have been unable to access. A good deal of the possible inefficiency stems from the fact that it is difficult to find an active route to the professionals who possess the knowledge we are eager for and hard to get them to make that knowledge explicit: it is not easy to locate them, they tend to have little time at their disposal, it is complicated to bring them together in face-to-face sessions, they have needs and interests different from those of researchers, they tend to be hard to pin down in their answers, they may have fears, personal characteristics and assumed roles that limit their responses, and so forth. Consequently, the right methodologies need to be developed for these kinds of experts and research objects.
With this view in mind, we make a methodology proposal in this work that is a hybrid of three techniques that are widely acknowledged and have a long track record: directed face-to-face discussion groups or Focus Groups, the Delphi Method and the Nominal Group Technique. Each of the three has its own characteristics that are highly suitable for obtaining the required results from this kind of expert, but they also have their disadvantages. A careful combination of these characteristics can reduce their limitations whilst retaining what they contribute. In the article we set out the characteristics and limitations of the Focus Group, the Delphi and the NGT, the purpose, configuration and development of the proposed hybrid methodology, which we have called Hybrid Delphi, and the practical application of this new methodology in three different professional contexts.
Section snippets
Focus Group
Focus Groups are carefully planned discussions or interviews, designed to obtain information within a defined area of interest, within a permissive and undirected atmosphere [4], [5], [6], [7], [8]. They fall within the category of structured face-to-face discussion groups [9], which means, therefore, that a variable group of experts (anyone who can, a priori, contribute input relevant for the study [10]) is brought together within the same physical space in order to interact with one another
Delphi method
The Delphi method [25], [26], [27], [28], [29], [30], is a social research technique which seeks to obtain a reliable group opinion from a set of experts. This is a method of structuring communication between a group of individuals who can provide valuable aid for solving a complex problem.
It has been used since the sixties in academic and business spheres and has been employed principally as a technique for planning and consensus in uncertainty situations in which it is not possible to use
Nominal Group Technique (NGT)
The NGT is a structured method for capturing and aggregating opinions emerging from a group of experts who physically coincide in terms of place and time [40], [41], [42]. In this regard, it might be deemed to be a particular case of FG. The NGT was conceived after Delphi, the purpose being to overcome some of its limitations, making use of most of its virtues, and orienting it towards creativity (generation and ordering of ideas). In a nutshell, the process runs as follows: 1- the study leader
Devising a hybrid technique: Hybrid Delphi
The three techniques set out earlier exhibit virtues and deficiencies when employed with groups of professionals (Table 1). The table indicates the potential of each technique for tackling certain problems that increase when used with professionals.
It can be observed that all the problems recorded in dealing with professional experts are resolved by at least one of these approaches, so a hybrid presentation of these three techniques could, in principle, contribute to eliminating or mitigating
Applications of the Hybrid Delphi
Later, by way of example and in an initial testing of our methodological proposal, we present three real applications of the Hybrid Delphi, which we carried out with experts in professional contexts. In the three cases the experts who took part did so voluntarily, contributing knowledge to an academic initiative from their professional activity. They had to fit this collaboration into their tight work schedules, with no compensation other than the information and knowledge that they might
Assessment of Hybrid Delphi applications
In order to assess the validity of the methodology and the confidence and satisfaction of the participants in the Hybrid Delphi Applications executed, we sent a questionnaire to all the experts who took part in the two stages of each of the three applications presented (Appendix A), asking them to assess from 0 to 10 the degree of interest and relevance to the study objective of each of the stages of the methodology applied: (1) presentation of the objectives of the study and of its theoretical
Discussion and conclusions
Hybrid Delphi is a technique that differs in its structure and objectives from other techniques based on the judgement of experts.
It is characterised by the fact that it offers its results as a consequence of the application of a methodology that structures its three basic factors (FG, NGT and Delphi) in a particular way. The order established is fixed and reasoned. Modification of it would alter the result. Not only is it chronological, but it also seeks to feed and channel a flow of
Acknowledgements
Our thanks go to our colleagues who participated with us in the three cases presented: to Jon Hoyos and Andrés Araujo, in the study on Continuous Management Training, Jon Sánchez, in the work on Family Business, and Laura Alonso, in the Assessment of application strategies for an evaluation model of competencies for community nurses, as well as to all the experts who disinterestedly took part in these Delphi exercises.
We would, of course, also like to thank the referees and editors for their
Jon Landeta is Director of the Institute of Applied Business Economics of the University of the Basque Country. He has carried out several Delphi studies, both academic and professional, and has published the book El Método Delphi. Una técnica de previsión para la incertidumbre (The Delphi Method. A forecasting technique for uncertainty), published in 1999 by Ariel, Barcelona, in addition to numerous articles on Delphi and human resources in journals as Technological Forecasting and Social
References (89)
- et al.
Epistemology, social technology, and expert judgment: Olaf Helmer's contribution to futures research
Futures
(2009) - et al.
The focus group method: insights from focus group interviews on sexual health with adolescents
Soc. Sci. Med.
(2005) - et al.
Clinical education and training: using the nominal group technique in research radiographers to identify factors affecting quality and capacity
Radiography
(2006) An examination of group process in judgmental forecasting
Int. J. Forecast.
(1989)The design of a Policy Dephi
Technol. Forecast. Soc. Change
(1970)- et al.
The Delphi technique as a forecasting tool: issues and analysis
Int. J. Forecast.
(1999) Technological forecasting: a criticism of Delphi techniques
Long Range Plan.
(1969)Delphi methodology: a review and critique
Int. J. Urban Syst.
(1979)- et al.
A critical review of the Delphi technique as a research methodology for nursing
Int. J. Nurs. Stud.
(2001) Current validity of the Delphi method in social sciences
Technol. Forecast. Soc. Change
(2006)
Integrating nominal group technique and joint application development for improved systems requirements determination
Inform. Manag. Amster.
Knowledge elicitation for validation of a neonatal ventilation expert system utilizing modified Delphi and focus group techniques
Int. J. Hum. Comput. Stud.
Optimal selection of location for Taiwanese hospitals to ensure a competitive advantage by using the analytic hierarchy process and sensitivity analysis
Building and Environment
Development of ovarian cancer surgery quality indicators using a modified Delphi approach
Gynecol. Oncol.
A comparative analysis of critical issues facing canadian information systems personnel: a national and global perspective
Inform Manage-Amster
Sustainability indicators for managing community tourism
Tourist Manage
Consenso clínico sobre laprofilaxis de la infeccion por virus respiratorio sincitial y el uso del palivizumab en cardiologıa pediátrica
An Pediatr (Barc)
Delphi a reevaluation of research and theory
Technol Forecast Soc
On the epistemology of inexact sciences
Manag. Sci.
Social Technology
The group depth interview
J. Mark.
Focus Groups—a Practical Guide for Applied Research
The Focused Interview: a Manual of Problems and Procedures
The use of focus group methodology—with selected examples from sexual health research
J. Adv. Nurs.
Breaking down the barriers: using focus groups to research small and medium-sized enterprises
Int. Small Bus. J.
Long-range Forecasting: from Crystal Ball to Computer
The Delphi method: substance, content, a critique, and annotated bibliography
Socio-Econ. Plan. Sci.
The effectiveness of Nominal, Delphi, and interacting group decision making processes
Acad. Manag. J
Quality and acceptance of evaluative task: the effects of four group decision-making formats
Group Organ. Stud.
The impact of decision-making methodology on job evaluation outcomes. A look at three consensus approaches
Group Organ. Manag.
Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task
Int. J. Forecast.
A Social Psychology of Group Processes for Decision-making
Groupthink
Effects of different conditions of acceptance upon conformity to group norms
J. Abnorm. Soc. Psychol.
Integrating group judgment in subjective forecast
More than the sum of the parts: research methods in group interviewing
Br. Educ. Res. J.
Exploration in group interviews: an evaluation of a reflexive and partisan approach
Br. Educ. Res. J.
Focus Groups
Soci. Res.
An experimental application of the Delphi Method to the use of experts
Manag. Sci.
Report on a Long-range Forecasting Study
An examination of the factors contributing to Delphi accuracy
J. Forecast.
El método Delphi
Delphi Assessment: Expert Opinion, Forecasting, and Group Process
Cited by (140)
Advanced fibrosis associated with non-alcoholic steatohepatitis (NASH) in Spain: results of a Delphi study
2024, Gastroenterologia y HepatologiaDefining successful climate services for adaptation with experts
2024, Environmental Science and PolicyIncorporating experts’ judgment into machine learning models
2023, Expert Systems with ApplicationsA Delphi study of business models for cycling urban mobility platforms
2022, Research in Transportation Business and Management
Jon Landeta is Director of the Institute of Applied Business Economics of the University of the Basque Country. He has carried out several Delphi studies, both academic and professional, and has published the book El Método Delphi. Una técnica de previsión para la incertidumbre (The Delphi Method. A forecasting technique for uncertainty), published in 1999 by Ariel, Barcelona, in addition to numerous articles on Delphi and human resources in journals as Technological Forecasting and Social Change, International Journal of Forecasting, International Journal of Human Resources Management, The European Journal of Innovation Management or Journal of Intellectual Capital.
Jon Barrutia (Ph.D degree at the University of the Basque Country, in 1988) is a Senior Research Fellow of the Institute of Applied Business Economics of the University of the Basque Country, and Professor and Director of the Department of Finance Economics II of the University of Basque Country. He was previously Deputy Minister for Universities and Research of the Basque Government. His research focuses on the following topics: Public Policy, Innovation, Governance and Human Resource Management. He has published en International Journal of Human Resources Management, International Journal of Forecasting and Nova Science Publishers
Aitziber Lertxundi is PhD from the University of the Basque Country (2008) and Master in Finance. She is Professor in the Department of Finance Economics II in this university. Author of several research articles and other publications, her current research interests include: High Performance Work Systems; Human Resource Management in multinational enterprises and cultural approach in this last field.