The use of Delphi methodology in agrifood policy development: Some lessons learned

https://doi.org/10.1016/j.techfore.2011.05.005Get rights and content

Abstract

Developing policy in the agrifood area is an inexact process, usually relying upon effective integration of opinions from multiple experts from different disciplines, organisational types, and regions/countries. Delphi would appear to have the potential to overcome some of the typical limitations related to soliciting expert opinion and identifying consensus on future activities or options, particularly where relevant experts are dispersed geographically, and international consensus is required, as is the case in this domain. Three case studies, focused on the application of Delphi to emerging policy needs in international or European agrifood policy, are presented here to exemplify the utility of the technique. A number of practical recommendations are drawn from these case studies that may be applicable to other major policy making arenas. Among these recommendations are; that an exploratory workshop to refine round one Delphi questions is essential; that the implementation of “cascade” methodology (utilizing the personal contacts of researchers or members of existing policy networks) appears to increase response rates in subsequent Delphi rounds; and that the policy issue under discussion should be particularly relevant to stakeholders in order to increase participation rates. Further research might usefully focus on developing ways to incorporate measurements of uncertainty associated with stakeholder judgement into quantitative responses, and on establishing how best to utilise such information in feedback in subsequent Delphi rounds. Ensuring how best to inform policy uptake of the outputs of Delphi merits further research in particular.

Research highlights

►An exploratory workshop to refine round one Delphi questions is essential. ►Using “cascade” methodology to recruit participant recruitment increases response rates in subsequent Delphi rounds. ►Policy issues under discussion should be particularly relevant to stakeholders in order to increase participation rates. ►Future research is needed to assess how to inform policy uptake, and measure policy impact of Delphi outputs.

Introduction

Established, emerging, and indeed re-emerging food-borne risks, in particular diseases of production animals and plants, represent serious threats globally. Failure to effectively deal with existing and emerging hazards in the agrifood context may have potentially negative impacts on human and animal health and the environment, in both developed and developing countries [1], [2], [3]. The socio-economic functioning of (different groups within) populations may also be adversely affected [4], [5]. Direct economic costs may be incurred through impacts associated with health care and time lost from employment, or from veterinary treatments and other mitigating actions, as well as from losses incurred by farmers and industry as a consequence of production stoppages and food and product recalls [6]. Indirect costs may be reflected through the loss of consumer confidence in particular types of food product or specific brands [7], or in industry, tourism, and policy-related governance structures [8]. At the same time, the development of agrifood policy is a continuously changing area [9] in particular with respect to increased globalisation of the food supply. This has resulted in more rapid dissemination of (emerging and re-emerging) food-borne risks [10] and rapid spread of animal diseases [11], [12]. Furthermore, changing human demographic trends, such as the ageing population and increased population density in urban environments, have created population groups with increased vulnerabilities to the health impact of food risks or zoonoses [13], [14]. As a consequence, these issues demand greater policy consideration.

Appropriate agrifood policy, for example relating to successful plant and animal disease control, depends on effective and timely risk detection, identification, prevention, and mitigation, which may require harmonisation of existing knowledge and methodologies to ensure transparent and proactive assessment of risks in the global food production chain. An internationally harmonised risk governance and concerted international effort is needed to develop a strategy for how these measures are to be achieved.

Policy development related to emerging food and animal health risks has made extensive use of the traditional and well-established expert scientific advisory committee model. At a national level, for example, during a crisis situation, such expert advisory bodies may be supplemented by, and report to, national crisis committees, to facilitate more rapid policy responses and action based on national need (e.g. foot and mouth disease in the UK in 2001). This can, however, result in scientific experts being pressed to consider other aspects outside of their fields of expertise, including social and political dimensions that they may not be qualified to address [15] This may lead to perceived or actual spurious scientific legitimation for policy decisions [16], [17]. At the international level (EU, FAO, OIE, WHO), a similar approach, involving internationally constituted expert advisory groups, is frequently adopted. However, other complications arise here as the committees may find themselves under pressure from differing, and even conflicting individual national policy responses [18]. This makes the involvement of supranational, or even global, panels necessary for food safety risk assessment and management. International scientific advisory bodies also have to operate within an established system of decision-making and governance, which may affect the independence and impact of their recommendations and advice [19]. This may result in policy makers having some control over ‘independent’ scientific committees [20]. All these issues raise the question of how to feasibly and validly consult diverse and significant constituencies in order to assess their policy priorities. It would appear that an expert consultation method that is able to deal with a broad range of different experts, that can be applied on a global scale, enabling anonymous responses, and that can deal with (initial) dissensus among experts from different geographical backgrounds is needed.

One such method that would appear to have potential utility in the agrifood domain for identifying policy issues and indicating levels of consensus is the Delphi technique. This approach, as classically construed, involves iterated questionnaires being presented anonymously to experts, with controlled feedback between rounds, and the equal weighting of final round responses to produce a group judgement [21]. Variations of the method exist, in terms of the number of rounds used, whether or not the first round is structured (quantitative) or unstructured (qualitative), whether the process takes place using paper-and-pencil questionnaires or ‘online’, whether the process is synchronous or asynchronous, and so on, as is the case with an online, real time Delphi [e.g. [22], [23]]. Delphi has also been combined with other processes to produce multi-stage or hybrid methods (e.g. see [24]).The aims of the approach may also vary, that is, it may be conducted in order to gain expert consensus or, importantly, identify dissensus where this exists (e.g. the Policy Delphi. See Ref. [25]).

Regardless of these variations, however, the emphasis of the method is on maintaining anonymity within the group regarding individual responses in order to ameliorate the potential social and political interactions that often take places within groups and which can prove counterproductive for identifying acceptable problem solutions. This issue may be of particular relevance to the agrifood domain, where personal, professional and even national sensitivities might act as barriers to effective problem solving. Delphi's distributive nature would also appear especially apt for this domain, where the highly dispersed nature of busy experts appears to militate against the traditional stakeholder committee approach that is frequently used, requiring as it does the gathering together of all relevant stakeholders and experts at the same time and in the same place [26]. Another constraint to ‘traditional’ approaches may be linguistic, as simultaneous translation into multiple languages is likely to be difficult and expensive, if feasible at all. Survey methodology, which solicits answers to key questions of interest, is ideally suited to identifying consensus or disagreement, and surveys can be relatively easily translated, and, through internet application, attract participants with a large geographical dispersion. However, the survey method is limited as it does not allow for any possibility of interaction between participants, or resolution of initial dissensus. Whilst some disagreement may be fundamental, the lack of resolution may emphasise minor differences in opinion, which may be particularly problematic in the policy arena [28]. Delphi involves iterated questionnaires and feedback, and there is an opportunity for problem resolution. In this way Delphi would appear superior to a straightforward survey approach. On the other hand, despite these hypothetical benefits, there are other issues that might interfere with the “ideal” conduct of a Delphi, primarily linked to the nature of relevant participants. For many agrifood policy issues, relevant experts are generally a) highly dispersed (indeed, for some issues there may be a specific requirement for there to be representation from different and specific nations or regions), which may include developing countries in particular, since these may be the origins of novel risks, or suffer disproportionate adverse consequences should the risk occur, and b) from many diverse disciplines, who are unlikely to be members of the same policy network. These factors have implications for how one selects relevant experts (different nations, disciplines, and networks), and conducts the process (in terms of language and presentation of information). These issues will be further discussed later.

Despite of these hypothetical benefits, however, Delphi has not been frequently employed in the agrifood policy area. In this paper, three recent Delphi applications (in which the authors were involved as researchers1) are discussed, in order to demonstrate the approach's utility in this novel area, and highlight particular difficulties that may require innovations or particular variations in the method. The three case studies focus on eliciting international expert opinion regarding agrifood policy focused on food safety governance, emerging food risk identification, and the prevention of emerging infectious diseases of production animals. The results of these studies have been published elsewhere. The aim of the current paper is to extract pragmatic methodological lessons from these cases, in order to inform the conduct of such large scale Delphi exercises in the future. Note that it was felt inappropriate for experts of the seniority of those involved in the three studies to be offered “incentives” following their inclusion as participants, other than being offered copies of the reports once these had been finalised. This issue is therefore not discussed further in the current paper.

Section snippets

The Delphi method applied to the development of agrifood policy—three case studies

To study the applicability of Delphi for policy consensus in the case of global or European food policy issues, the three recent Delphi exercises used as cases were analysed with respect to their utility and effectiveness as a policy tool. These Delphi exercises focused on policy issues in the agrifood sector. The first focused on food risk governance, the second on emerging food risk identification, and the third on emerging infectious animal diseases. To assess the benefits of the Delphi

Discussion

The case studies confirm the utility of internet approaches to Delphi in allowing for consulting large, geographically dispersed, expert communities. In practice, the application to a broad and dispersed expert community (the Safe Foods Delphi) and a large geographically dispersed expert community (the GoGlobal Delphi) was successful in raising opinions, identifying agreements and pre-existing dissensus and consensus between different (groups of) participants. However, these Delphis were less

Conclusions

For broad, international Delphis an exploratory workshop to refine round 1 Delphi questions is important to ensure the best, most comprehensive, and most focused issues for inclusion in a largely quantitative survey (necessary to overcome possible limitations of international participation). This may involve few key stakeholders or experts in the policy area under consideration.

The use of forms of “cascade” methodology, utilizing the personal contacts of researchers or members of existing

Acknowledgements

This research was funded by the European Commission projects Safe Foods (Food-CT-2004-506446) GO-GLOBAL (contract number FOOD-CT-2006-043053), and INNOVA-P2 (contract number 217665), and ERA-NET EMIDA (project number 219235). The views expressed here are entirely those of the authors.

Lynn J. Frewer is Professor of Food and Society at the Centre for Rural Economy, Newcastle University in the UK. Lynn is interested in all aspects of food and society, including the developments of Stakeholder analysis and operationalising scientific foresight regarding research agenda setting, policy and governance in the area of emerging food technologies, and food and agricultural risks. Other interests include consumer behavior in response to food choices, and related methodological

References (43)

  • M.T.A. Wentholt et al.

    Effective identification and management of emerging food risks: results of an international Delphi survey

    Food Control

    (2010)
  • G. Wright et al.

    Group-based judgmental forecasting: an integration of extant knowledge and the development of priorities for a new research agenda

    Int. J. Forecast.

    (2011)
  • FAO/WHO

    Food safety risk analysis—a guide for national food safety authorities

    Food and Nutrition paper 87 World Health Organization and Food and Agricultural Organization of United Nations, Rome

    (2006)
  • R. Le Heron

    Creating food futures: reflections on food governance issues in New Zealand's agrifood sector

    J. Rural. Stud.

    (2003)
  • F.E. El-Gazzar et al.

    Foodborne disease: investigative procedures and economic assessment

    J. Environ. Health

    (1992)
  • J.R. Houghton et al.

    Consumer perceptions of the effectiveness of food risk management practices: a cross-cultural study

    Health Risk Soc.

    (2008)
  • M. Figuié et al.

    Avian influenza in Vietnam: chicken-hearted consumers?

    Risk Anal.

    (2008)
  • Office of Science and Technology—UK, Foresight—detection and identification of infectious diseases. UK Government report, Office of Science and Technology—DTI

  • M.E.J. Woolhouse et al.

    Host range and emerging and reemerging pathogens

  • T.A. Crowl et al.

    The spread of invasive species and infectious disease as drivers of ecosystem change

    Front. Ecol. Environ.

    (2008)
  • Cited by (0)

    Lynn J. Frewer is Professor of Food and Society at the Centre for Rural Economy, Newcastle University in the UK. Lynn is interested in all aspects of food and society, including the developments of Stakeholder analysis and operationalising scientific foresight regarding research agenda setting, policy and governance in the area of emerging food technologies, and food and agricultural risks. Other interests include consumer behavior in response to food choices, and related methodological development.

    Arnout R. H. Fischer is assistant professor with the Marketing and Consumer Behaviour Group at Wageningen University, the Netherlands. Arnout aims to understand food related consumer behaviour for future trends and technologies through combining social psychological approaches with forecasting methodology. Arnout publishes in the area of consumer behaviour and stakeholder analysis in relation to food safety and food technologies in academic books and peer reviewed journals.

    Meike T.A. Wentholt is a researcher in the Marketing and Consumer Behaviour Group at Wageningen University in the Netherlands. Meike is currently conducting PhD research into effective stakeholder involvement in agrifood governance and policy development. Meike's research includes applying and assessing methodology for effective stakeholder inputs into policy and research agenda setting, in particular using Delphi Methodology.

    Hans J.P. Marvin is senior scientist at RIKILT-Institute of Food Safety. His main research interest is to understand the way (new) food safety problems may occur and to develop methods and procedures to predict their development. In addition, his research interests include assessment and management of the safety of food and feed products produced with emerging technologies (such as nanotechnology).

    B.W. (Wim) Ooms, has a degree in veterinary medicine and is a senior scientific officer of Veterinary Public Health for the Office for Risk Assessment of the Dutch Food and Consumer Product Safety Authority (VWA). Before this, Wim was a specialist meat hygienist. After which he acquired experience in governmental research coordination in food safety on behalf of the Ministry of Agriculture, Nature and Food Quality (now the Ministry of Economic Affairs, Agriculture and Innovation) in the Netherlands before joining the VWA. For five years Wim has been committed to objectives such as the early detection and identification of emerging risks and emerging (zoonotic) diseases.

    David Coles coordinates collaborations between national European research programmes on poverty related diseases in sub-Saharan Africa. David is a member of the Wageningen University MCB group and Director of Enhance International Limited. Previously he worked for the European Commission, developing the EU ethics review. In the UK, David implemented guidelines on the use of scientific advice and developed a code of practice for scientific advisory committees for the Office of Science and Technology. At the Department of Health, David was involved in the development and implementation of DH policy on medical ethics, research ethics committees, risk analysis and Government guidelines on risk communication.

    Gene Rowe is an independent research consultant (Gene Rowe Evaluations). He is also a visiting scientist at the Institute of Food Research, Norwich, UK (where he was formerly Head of Consumer Science) and at Wageningen University, The Netherlands. Gene's early research focused on group judgemental forecasting, and on Delphi in particular, although he has latterly branched out to conduct research into risk perception, risk communication, and public engagement processes (and the evaluation for these).

    View full text