Hybrid Delphi: A methodology to facilitate contribution from experts in professional contexts

https://doi.org/10.1016/j.techfore.2011.03.009Get rights and content

Abstract

This work presents a new methodology based on three well-known qualitative techniques (Focus Group, Nominal Group Technique and Delphi method), with a view to harmonising their potentialities and reducing their limitations, through application in real contexts with experts who are professionals in their respective activities.

The main contribution of this methodology is its joint consideration of the needs of the investigators and also of experts who act in professional contexts, in order to improve the effectiveness of preceding techniques in achieving the scientific and social objectives of the study.

We have tested this methodological approach in three real cases, with experts holding different responsibilities in different companies and public organisations, and the results secured are highly satisfactory, due to both the quantity and quality of the proposals obtained, and the satisfaction exhibited by the experts taking part, with regard to the research methodology.

Research highlights

► The Hybrid Delphi is a valid technique for use in professional contexts. ► It draws the best from Focus Groups, the Nominal Group Technique and the Delphi method. ► It satisfies the needs of both the experts and the researchers taking part. ► It encourages creativity and reasoned assessment in experts' contributions

Introduction

What is the best way of extracting and processing the information possessed by a set of professionals concerning a problem or phenomenon? Since Helmer and Rescher [1], [2] laid the foundations for the scientific use of expert opinion in the areas of decision and prediction, different techniques have been developed over the years aimed at responding to the initial question, in different contexts and with different objectives. All these techniques have, in general, sought to contribute improvements in one or some of the fields of action originally delimited by Helmer [3]: 1- improvement in the selection of the most suitable information sources for each concrete case (relying on the appropriate experts), 2- assisting in the effective development of experts' activities, facilitating transmission of the information required, and 3- development of methodologies of action that make the processing of that information possible, achieving outcomes of greater quality.

This methodological development is still of relevance, because knowledge advances and because needs change with time. In this regard, one sphere of great scientific and social interest is the obtaining of ranked lists of problems, practices, solutions… regarding a specific phenomenon using the tacit knowledge of professional experts who are undergoing problems, carrying out practices or applying solutions in their respective contexts of action. By professionals we refer to experts who voluntarily collaborate in a forecasting or group decision activity, freely contributing their knowledge, which is normally a product of their usual working activity. From this category we exclude, therefore, students, scholarship holders and other “captive experts”. For this purpose different group techniques may be employed, some of which already have a broad academic history of application. However, those of us who apply them feel that there is much more knowledge that we have been unable to access. A good deal of the possible inefficiency stems from the fact that it is difficult to find an active route to the professionals who possess the knowledge we are eager for and hard to get them to make that knowledge explicit: it is not easy to locate them, they tend to have little time at their disposal, it is complicated to bring them together in face-to-face sessions, they have needs and interests different from those of researchers, they tend to be hard to pin down in their answers, they may have fears, personal characteristics and assumed roles that limit their responses, and so forth. Consequently, the right methodologies need to be developed for these kinds of experts and research objects.

With this view in mind, we make a methodology proposal in this work that is a hybrid of three techniques that are widely acknowledged and have a long track record: directed face-to-face discussion groups or Focus Groups, the Delphi Method and the Nominal Group Technique. Each of the three has its own characteristics that are highly suitable for obtaining the required results from this kind of expert, but they also have their disadvantages. A careful combination of these characteristics can reduce their limitations whilst retaining what they contribute. In the article we set out the characteristics and limitations of the Focus Group, the Delphi and the NGT, the purpose, configuration and development of the proposed hybrid methodology, which we have called Hybrid Delphi, and the practical application of this new methodology in three different professional contexts.

Section snippets

Focus Group

Focus Groups are carefully planned discussions or interviews, designed to obtain information within a defined area of interest, within a permissive and undirected atmosphere [4], [5], [6], [7], [8]. They fall within the category of structured face-to-face discussion groups [9], which means, therefore, that a variable group of experts (anyone who can, a priori, contribute input relevant for the study [10]) is brought together within the same physical space in order to interact with one another

Delphi method

The Delphi method [25], [26], [27], [28], [29], [30], is a social research technique which seeks to obtain a reliable group opinion from a set of experts. This is a method of structuring communication between a group of individuals who can provide valuable aid for solving a complex problem.

It has been used since the sixties in academic and business spheres and has been employed principally as a technique for planning and consensus in uncertainty situations in which it is not possible to use

Nominal Group Technique (NGT)

The NGT is a structured method for capturing and aggregating opinions emerging from a group of experts who physically coincide in terms of place and time [40], [41], [42]. In this regard, it might be deemed to be a particular case of FG. The NGT was conceived after Delphi, the purpose being to overcome some of its limitations, making use of most of its virtues, and orienting it towards creativity (generation and ordering of ideas). In a nutshell, the process runs as follows: 1- the study leader

Devising a hybrid technique: Hybrid Delphi

The three techniques set out earlier exhibit virtues and deficiencies when employed with groups of professionals (Table 1). The table indicates the potential of each technique for tackling certain problems that increase when used with professionals.

It can be observed that all the problems recorded in dealing with professional experts are resolved by at least one of these approaches, so a hybrid presentation of these three techniques could, in principle, contribute to eliminating or mitigating

Applications of the Hybrid Delphi

Later, by way of example and in an initial testing of our methodological proposal, we present three real applications of the Hybrid Delphi, which we carried out with experts in professional contexts. In the three cases the experts who took part did so voluntarily, contributing knowledge to an academic initiative from their professional activity. They had to fit this collaboration into their tight work schedules, with no compensation other than the information and knowledge that they might

Assessment of Hybrid Delphi applications

In order to assess the validity of the methodology and the confidence and satisfaction of the participants in the Hybrid Delphi Applications executed, we sent a questionnaire to all the experts who took part in the two stages of each of the three applications presented (Appendix A), asking them to assess from 0 to 10 the degree of interest and relevance to the study objective of each of the stages of the methodology applied: (1) presentation of the objectives of the study and of its theoretical

Discussion and conclusions

Hybrid Delphi is a technique that differs in its structure and objectives from other techniques based on the judgement of experts.

It is characterised by the fact that it offers its results as a consequence of the application of a methodology that structures its three basic factors (FG, NGT and Delphi) in a particular way. The order established is fixed and reasoned. Modification of it would alter the result. Not only is it chronological, but it also seeks to feed and channel a flow of

Acknowledgements

Our thanks go to our colleagues who participated with us in the three cases presented: to Jon Hoyos and Andrés Araujo, in the study on Continuous Management Training, Jon Sánchez, in the work on Family Business, and Laura Alonso, in the Assessment of application strategies for an evaluation model of competencies for community nurses, as well as to all the experts who disinterestedly took part in these Delphi exercises.

We would, of course, also like to thank the referees and editors for their

Jon Landeta is Director of the Institute of Applied Business Economics of the University of the Basque Country. He has carried out several Delphi studies, both academic and professional, and has published the book El Método Delphi. Una técnica de previsión para la incertidumbre (The Delphi Method. A forecasting technique for uncertainty), published in 1999 by Ariel, Barcelona, in addition to numerous articles on Delphi and human resources in journals as Technological Forecasting and Social

References (89)

  • E.W. Duggan et al.

    Integrating nominal group technique and joint application development for improved systems requirements determination

    Inform. Manag. Amster.

    (2004)
  • K. Tan et al.

    Knowledge elicitation for validation of a neonatal ventilation expert system utilizing modified Delphi and focus group techniques

    Int. J. Hum. Comput. Stud.

    (2010)
  • C. Wu et al.

    Optimal selection of location for Taiwanese hospitals to ensure a competitive advantage by using the analytic hierarchy process and sensitivity analysis

    Building and Environment

    (2007)
  • A.R. Gagliardi et al.

    Development of ovarian cancer surgery quality indicators using a modified Delphi approach

    Gynecol. Oncol.

    (2005)
  • S.C. Hayne et al.

    A comparative analysis of critical issues facing canadian information systems personnel: a national and global perspective

    Inform Manage-Amster

    (2000)
  • H.C. Choi et al.

    Sustainability indicators for managing community tourism

    Tourist Manage

    (2006)
  • C. Medrano Lopez et al.

    Consenso clínico sobre laprofilaxis de la infeccion por virus respiratorio sincitial y el uso del palivizumab en cardiologıa pediátrica

    An Pediatr (Barc)

    (2010)
  • G. Rowe et al.

    Delphi a reevaluation of research and theory

    Technol Forecast Soc

    (1991)
  • H. Helmer et al.

    On the epistemology of inexact sciences

    Manag. Sci.

    (1959)
  • O. Helmer

    Social Technology

  • A.E. Goldman

    The group depth interview

    J. Mark.

    (July, 1962)
  • R.A. Krueger

    Focus Groups—a Practical Guide for Applied Research

    (1994)
  • R.K. Merton et al.

    The Focused Interview: a Manual of Problems and Procedures

    (1990)
  • N. Robinson

    The use of focus group methodology—with selected examples from sexual health research

    J. Adv. Nurs.

    (1999)
  • R. Blackburn

    Breaking down the barriers: using focus groups to research small and medium-sized enterprises

    Int. Small Bus. J.

    (2000)
  • J.S. Armstrong

    Long-range Forecasting: from Crystal Ball to Computer

    (1978)
  • J. Pill

    The Delphi method: substance, content, a critique, and annotated bibliography

    Socio-Econ. Plan. Sci.

    (1971)
  • A.H. Van de Ven et al.

    The effectiveness of Nominal, Delphi, and interacting group decision making processes

    Acad. Manag. J

    (1974)
  • R.C. Erffmeyer et al.

    Quality and acceptance of evaluative task: the effects of four group decision-making formats

    Group Organ. Stud.

    (1984)
  • J.S. Hornsby et al.

    The impact of decision-making methodology on job evaluation outcomes. A look at three consensus approaches

    Group Organ. Manag.

    (1994)
  • A. Graefe et al.

    Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task

    Int. J. Forecast.

    (2010)
  • B.E. Collins et al.

    A Social Psychology of Group Processes for Decision-making

    (1964)
  • I.L. Janis

    Groupthink

    (1982)
  • J.E. Dittes et al.

    Effects of different conditions of acceptance upon conformity to group norms

    J. Abnorm. Soc. Psychol.

    (1956)
  • A. Lock

    Integrating group judgment in subjective forecast

  • M. Watts et al.

    More than the sum of the parts: research methods in group interviewing

    Br. Educ. Res. J.

    (1987)
  • M. Descombe

    Exploration in group interviews: an evaluation of a reflexive and partisan approach

    Br. Educ. Res. J.

    (1995)
  • A. Gibbs

    Focus Groups

    Soci. Res.

    (1997)
  • N. Dalkey et al.

    An experimental application of the Delphi Method to the use of experts

    Manag. Sci.

    (1963)
  • T.J. Gordon et al.

    Report on a Long-range Forecasting Study

    (1964)
  • R.J. Parentee et al.

    An examination of the factors contributing to Delphi accuracy

    J. Forecast.

    (1984)
  • J. Landeta

    El método Delphi

    (1999)
  • H. Sackman

    Delphi Assessment: Expert Opinion, Forecasting, and Group Process

    (1974)
  • Cited by (140)

    • A Delphi study of business models for cycling urban mobility platforms

      2022, Research in Transportation Business and Management
    View all citing articles on Scopus

    Jon Landeta is Director of the Institute of Applied Business Economics of the University of the Basque Country. He has carried out several Delphi studies, both academic and professional, and has published the book El Método Delphi. Una técnica de previsión para la incertidumbre (The Delphi Method. A forecasting technique for uncertainty), published in 1999 by Ariel, Barcelona, in addition to numerous articles on Delphi and human resources in journals as Technological Forecasting and Social Change, International Journal of Forecasting, International Journal of Human Resources Management, The European Journal of Innovation Management or Journal of Intellectual Capital.

    Jon Barrutia (Ph.D degree at the University of the Basque Country, in 1988) is a Senior Research Fellow of the Institute of Applied Business Economics of the University of the Basque Country, and Professor and Director of the Department of Finance Economics II of the University of Basque Country. He was previously Deputy Minister for Universities and Research of the Basque Government. His research focuses on the following topics: Public Policy, Innovation, Governance and Human Resource Management. He has published en International Journal of Human Resources Management, International Journal of Forecasting and Nova Science Publishers

    Aitziber Lertxundi is PhD from the University of the Basque Country (2008) and Master in Finance. She is Professor in the Department of Finance Economics II in this university. Author of several research articles and other publications, her current research interests include: High Performance Work Systems; Human Resource Management in multinational enterprises and cultural approach in this last field.

    View full text