The use of Delphi methodology in agrifood policy development: Some lessons learned
Research highlights
►An exploratory workshop to refine round one Delphi questions is essential. ►Using “cascade” methodology to recruit participant recruitment increases response rates in subsequent Delphi rounds. ►Policy issues under discussion should be particularly relevant to stakeholders in order to increase participation rates. ►Future research is needed to assess how to inform policy uptake, and measure policy impact of Delphi outputs.
Introduction
Established, emerging, and indeed re-emerging food-borne risks, in particular diseases of production animals and plants, represent serious threats globally. Failure to effectively deal with existing and emerging hazards in the agrifood context may have potentially negative impacts on human and animal health and the environment, in both developed and developing countries [1], [2], [3]. The socio-economic functioning of (different groups within) populations may also be adversely affected [4], [5]. Direct economic costs may be incurred through impacts associated with health care and time lost from employment, or from veterinary treatments and other mitigating actions, as well as from losses incurred by farmers and industry as a consequence of production stoppages and food and product recalls [6]. Indirect costs may be reflected through the loss of consumer confidence in particular types of food product or specific brands [7], or in industry, tourism, and policy-related governance structures [8]. At the same time, the development of agrifood policy is a continuously changing area [9] in particular with respect to increased globalisation of the food supply. This has resulted in more rapid dissemination of (emerging and re-emerging) food-borne risks [10] and rapid spread of animal diseases [11], [12]. Furthermore, changing human demographic trends, such as the ageing population and increased population density in urban environments, have created population groups with increased vulnerabilities to the health impact of food risks or zoonoses [13], [14]. As a consequence, these issues demand greater policy consideration.
Appropriate agrifood policy, for example relating to successful plant and animal disease control, depends on effective and timely risk detection, identification, prevention, and mitigation, which may require harmonisation of existing knowledge and methodologies to ensure transparent and proactive assessment of risks in the global food production chain. An internationally harmonised risk governance and concerted international effort is needed to develop a strategy for how these measures are to be achieved.
Policy development related to emerging food and animal health risks has made extensive use of the traditional and well-established expert scientific advisory committee model. At a national level, for example, during a crisis situation, such expert advisory bodies may be supplemented by, and report to, national crisis committees, to facilitate more rapid policy responses and action based on national need (e.g. foot and mouth disease in the UK in 2001). This can, however, result in scientific experts being pressed to consider other aspects outside of their fields of expertise, including social and political dimensions that they may not be qualified to address [15] This may lead to perceived or actual spurious scientific legitimation for policy decisions [16], [17]. At the international level (EU, FAO, OIE, WHO), a similar approach, involving internationally constituted expert advisory groups, is frequently adopted. However, other complications arise here as the committees may find themselves under pressure from differing, and even conflicting individual national policy responses [18]. This makes the involvement of supranational, or even global, panels necessary for food safety risk assessment and management. International scientific advisory bodies also have to operate within an established system of decision-making and governance, which may affect the independence and impact of their recommendations and advice [19]. This may result in policy makers having some control over ‘independent’ scientific committees [20]. All these issues raise the question of how to feasibly and validly consult diverse and significant constituencies in order to assess their policy priorities. It would appear that an expert consultation method that is able to deal with a broad range of different experts, that can be applied on a global scale, enabling anonymous responses, and that can deal with (initial) dissensus among experts from different geographical backgrounds is needed.
One such method that would appear to have potential utility in the agrifood domain for identifying policy issues and indicating levels of consensus is the Delphi technique. This approach, as classically construed, involves iterated questionnaires being presented anonymously to experts, with controlled feedback between rounds, and the equal weighting of final round responses to produce a group judgement [21]. Variations of the method exist, in terms of the number of rounds used, whether or not the first round is structured (quantitative) or unstructured (qualitative), whether the process takes place using paper-and-pencil questionnaires or ‘online’, whether the process is synchronous or asynchronous, and so on, as is the case with an online, real time Delphi [e.g. [22], [23]]. Delphi has also been combined with other processes to produce multi-stage or hybrid methods (e.g. see [24]).The aims of the approach may also vary, that is, it may be conducted in order to gain expert consensus or, importantly, identify dissensus where this exists (e.g. the Policy Delphi. See Ref. [25]).
Regardless of these variations, however, the emphasis of the method is on maintaining anonymity within the group regarding individual responses in order to ameliorate the potential social and political interactions that often take places within groups and which can prove counterproductive for identifying acceptable problem solutions. This issue may be of particular relevance to the agrifood domain, where personal, professional and even national sensitivities might act as barriers to effective problem solving. Delphi's distributive nature would also appear especially apt for this domain, where the highly dispersed nature of busy experts appears to militate against the traditional stakeholder committee approach that is frequently used, requiring as it does the gathering together of all relevant stakeholders and experts at the same time and in the same place [26]. Another constraint to ‘traditional’ approaches may be linguistic, as simultaneous translation into multiple languages is likely to be difficult and expensive, if feasible at all. Survey methodology, which solicits answers to key questions of interest, is ideally suited to identifying consensus or disagreement, and surveys can be relatively easily translated, and, through internet application, attract participants with a large geographical dispersion. However, the survey method is limited as it does not allow for any possibility of interaction between participants, or resolution of initial dissensus. Whilst some disagreement may be fundamental, the lack of resolution may emphasise minor differences in opinion, which may be particularly problematic in the policy arena [28]. Delphi involves iterated questionnaires and feedback, and there is an opportunity for problem resolution. In this way Delphi would appear superior to a straightforward survey approach. On the other hand, despite these hypothetical benefits, there are other issues that might interfere with the “ideal” conduct of a Delphi, primarily linked to the nature of relevant participants. For many agrifood policy issues, relevant experts are generally a) highly dispersed (indeed, for some issues there may be a specific requirement for there to be representation from different and specific nations or regions), which may include developing countries in particular, since these may be the origins of novel risks, or suffer disproportionate adverse consequences should the risk occur, and b) from many diverse disciplines, who are unlikely to be members of the same policy network. These factors have implications for how one selects relevant experts (different nations, disciplines, and networks), and conducts the process (in terms of language and presentation of information). These issues will be further discussed later.
Despite of these hypothetical benefits, however, Delphi has not been frequently employed in the agrifood policy area. In this paper, three recent Delphi applications (in which the authors were involved as researchers1) are discussed, in order to demonstrate the approach's utility in this novel area, and highlight particular difficulties that may require innovations or particular variations in the method. The three case studies focus on eliciting international expert opinion regarding agrifood policy focused on food safety governance, emerging food risk identification, and the prevention of emerging infectious diseases of production animals. The results of these studies have been published elsewhere. The aim of the current paper is to extract pragmatic methodological lessons from these cases, in order to inform the conduct of such large scale Delphi exercises in the future. Note that it was felt inappropriate for experts of the seniority of those involved in the three studies to be offered “incentives” following their inclusion as participants, other than being offered copies of the reports once these had been finalised. This issue is therefore not discussed further in the current paper.
Section snippets
The Delphi method applied to the development of agrifood policy—three case studies
To study the applicability of Delphi for policy consensus in the case of global or European food policy issues, the three recent Delphi exercises used as cases were analysed with respect to their utility and effectiveness as a policy tool. These Delphi exercises focused on policy issues in the agrifood sector. The first focused on food risk governance, the second on emerging food risk identification, and the third on emerging infectious animal diseases. To assess the benefits of the Delphi
Discussion
The case studies confirm the utility of internet approaches to Delphi in allowing for consulting large, geographically dispersed, expert communities. In practice, the application to a broad and dispersed expert community (the Safe Foods Delphi) and a large geographically dispersed expert community (the GoGlobal Delphi) was successful in raising opinions, identifying agreements and pre-existing dissensus and consensus between different (groups of) participants. However, these Delphis were less
Conclusions
For broad, international Delphis an exploratory workshop to refine round 1 Delphi questions is important to ensure the best, most comprehensive, and most focused issues for inclusion in a largely quantitative survey (necessary to overcome possible limitations of international participation). This may involve few key stakeholders or experts in the policy area under consideration.
The use of forms of “cascade” methodology, utilizing the personal contacts of researchers or members of existing
Acknowledgements
This research was funded by the European Commission projects Safe Foods (Food-CT-2004-506446) GO-GLOBAL (contract number FOOD-CT-2006-043053), and INNOVA-P2 (contract number 217665), and ERA-NET EMIDA (project number 219235). The views expressed here are entirely those of the authors.
Lynn J. Frewer is Professor of Food and Society at the Centre for Rural Economy, Newcastle University in the UK. Lynn is interested in all aspects of food and society, including the developments of Stakeholder analysis and operationalising scientific foresight regarding research agenda setting, policy and governance in the area of emerging food technologies, and food and agricultural risks. Other interests include consumer behavior in response to food choices, and related methodological
References (43)
- et al.
A working procedure for identifying emerging food safety issues at an early stage: implications for European and international risk management practices
Food Control
(2009) - et al.
Exploring the social impact of food safety issues: including social impact assessment in food safety governance
Food Control
(2010) - et al.
Economic assessment of food safety standards: costs and benefits of alternative approaches
Food Control
(2010) - et al.
A note on modelling consumer reactions to a crisis: the case of mad cow disease
Int. J. Res. Mark.
(2002) - et al.
A qualitative assessment tool for the potential of infectious disease emergence and spread
Prev. Vet. Med.
(2007) - et al.
RT Delphi: an efficient, “round-less” almost real time Delphi method
Technol. Forecast Soc. Change
(2006) - et al.
The Delphi technique: a re-evaluation of research and theory
Technol. Forecast Soc. Change
(1991) The design of a policy Delphi
Technol. Forecast Soc. Change
(1970)- et al.
The views of key stakeholders on an evolving food risk governance framework: results from a Delphi study
Food Policy
(2009) Milk falsified by melamine: another scandal in public health [Lait falsifié par la mélamine: un autre scandale de santé publique]
Ann. Pharm. Fr.
(2009)
Effective identification and management of emerging food risks: results of an international Delphi survey
Food Control
Group-based judgmental forecasting: an integration of extant knowledge and the development of priorities for a new research agenda
Int. J. Forecast.
Food safety risk analysis—a guide for national food safety authorities
Food and Nutrition paper 87 World Health Organization and Food and Agricultural Organization of United Nations, Rome
Creating food futures: reflections on food governance issues in New Zealand's agrifood sector
J. Rural. Stud.
Foodborne disease: investigative procedures and economic assessment
J. Environ. Health
Consumer perceptions of the effectiveness of food risk management practices: a cross-cultural study
Health Risk Soc.
Avian influenza in Vietnam: chicken-hearted consumers?
Risk Anal.
Office of Science and Technology—UK, Foresight—detection and identification of infectious diseases. UK Government report, Office of Science and Technology—DTI
Host range and emerging and reemerging pathogens
The spread of invasive species and infectious disease as drivers of ecosystem change
Front. Ecol. Environ.
Cited by (0)
Lynn J. Frewer is Professor of Food and Society at the Centre for Rural Economy, Newcastle University in the UK. Lynn is interested in all aspects of food and society, including the developments of Stakeholder analysis and operationalising scientific foresight regarding research agenda setting, policy and governance in the area of emerging food technologies, and food and agricultural risks. Other interests include consumer behavior in response to food choices, and related methodological development.
Arnout R. H. Fischer is assistant professor with the Marketing and Consumer Behaviour Group at Wageningen University, the Netherlands. Arnout aims to understand food related consumer behaviour for future trends and technologies through combining social psychological approaches with forecasting methodology. Arnout publishes in the area of consumer behaviour and stakeholder analysis in relation to food safety and food technologies in academic books and peer reviewed journals.
Meike T.A. Wentholt is a researcher in the Marketing and Consumer Behaviour Group at Wageningen University in the Netherlands. Meike is currently conducting PhD research into effective stakeholder involvement in agrifood governance and policy development. Meike's research includes applying and assessing methodology for effective stakeholder inputs into policy and research agenda setting, in particular using Delphi Methodology.
Hans J.P. Marvin is senior scientist at RIKILT-Institute of Food Safety. His main research interest is to understand the way (new) food safety problems may occur and to develop methods and procedures to predict their development. In addition, his research interests include assessment and management of the safety of food and feed products produced with emerging technologies (such as nanotechnology).
B.W. (Wim) Ooms, has a degree in veterinary medicine and is a senior scientific officer of Veterinary Public Health for the Office for Risk Assessment of the Dutch Food and Consumer Product Safety Authority (VWA). Before this, Wim was a specialist meat hygienist. After which he acquired experience in governmental research coordination in food safety on behalf of the Ministry of Agriculture, Nature and Food Quality (now the Ministry of Economic Affairs, Agriculture and Innovation) in the Netherlands before joining the VWA. For five years Wim has been committed to objectives such as the early detection and identification of emerging risks and emerging (zoonotic) diseases.
David Coles coordinates collaborations between national European research programmes on poverty related diseases in sub-Saharan Africa. David is a member of the Wageningen University MCB group and Director of Enhance International Limited. Previously he worked for the European Commission, developing the EU ethics review. In the UK, David implemented guidelines on the use of scientific advice and developed a code of practice for scientific advisory committees for the Office of Science and Technology. At the Department of Health, David was involved in the development and implementation of DH policy on medical ethics, research ethics committees, risk analysis and Government guidelines on risk communication.
Gene Rowe is an independent research consultant (Gene Rowe Evaluations). He is also a visiting scientist at the Institute of Food Research, Norwich, UK (where he was formerly Head of Consumer Science) and at Wageningen University, The Netherlands. Gene's early research focused on group judgemental forecasting, and on Delphi in particular, although he has latterly branched out to conduct research into risk perception, risk communication, and public engagement processes (and the evaluation for these).