Introduction - Research Explorer | The University of ...



Title: “I was trying to do the maths”: exploring the impact of risk communication in discrete choice experimentsAuthors: Caroline Vass, PhDa*, Dan Rigby, PhDb; Katherine Payne, PhDaaManchester Centre for Health Economics, The University of Manchester, Oxford Road, Manchester M13 9PL, UKbDepartment of Economics, The University of Manchester, Oxford Road, Manchester M13 9PL, UK*Corresponding author: Caroline VassTelephone: +44 (0) 161 306 7970Email: caroline.vass@manchester.ac.ukTarget journal: The PatientWord count: 4,016/4,000 words for Original Research No. of tables: 1No. of figures: 1Appendices: 3Keywords: qualitative research; risk; discrete choice experiment; stated preference; breast screeningEthics: Ethical approval for this study was granted by The University of Manchester’s Research Ethics Committee. All participants provided consent. Funding: Preparation of this manuscript was made possible by a grant awarded by The Swedish Foundation for Humanities and Social Sciences (Riksbankens Jubileumsfond) for a project entitled ‘Mind the Risk’. Caroline Vass was in receipt of a National Institute for Health Research (NIHR) School for Primary Care Research (SPCR) PhD Studentship between October 2011 and 2014. The views expressed in this article are those of the authors and not of the funding bodies.Running head: Understanding the interpretation of risk in discrete choice experimentsConflict of Interest: Caroline Vass, Dan Rigby and Katherine Payne declare that they have no conflicts of interest.Acknowledgements: DR and KP conceptualised the research question, helped the develop the interview schedule, read transcripts, provided feedback on the analysis of the data, and contributed to the writing of the manuscript. CV arranged and conducted all interviews, completed the analysis of the qualitative data, and prepared the first draft of the manuscript.The authors are grateful for feedback received at the Society for Medical Decision Making’s 36th and 37th annual meetings. We are also grateful to experts Professor Gareth Evans and Professor Tony Howell for their clinical input on framing the choice question for a hypothetical breast screening programme in the UK. We are grateful to Dr Michelle Harvie for her assistance in specifying the relevant background questions and Ms Paula Stavrinos for her feedback on the introductory materials and video used in the survey. We are also grateful to Professor Stephen Campbell of the Centre for Primary Care at The University of Manchester for providing his thoughts and comments on the interview schedule and providing feedback as the themes developed. ORCIDCaroline Vass0000-0002-6385-2812Dan Rigby0000-0003-0820-5740Katherine Payne0000-0002-3938-4350 Abstract: [249/250 words] Background: Risk is increasingly used as an attribute in discrete choice experiments (DCEs). However, risk and probabilities are complex concepts that can be open to misinterpretation, potentially undermining the robustness of DCEs as a valuation method. This study aimed to understand how respondents made benefit-risk trade-offs in a DCE and if these were affected by the communication of the risk attributes.Methods: Female members of the public were recruited via local advertisements to participate in think-aloud interviews when completing a DCE eliciting their preferences for a hypothetical breast screening programme described by three attributes: probability of detecting a cancer; risk of unnecessary follow-up; and cost of screening. Women were randomised to receive risk information as either: 1) percentages or 2) percentages and icon arrays. Interviews were digitally recorded then transcribed to generate qualitative data for thematic analysis. Results: Nineteen women completed the interviews (icon arrays n=9; percentages n=10). Analysis revealed four key themes where women made references to: 1) the nature of the task; 2) their feelings; 3) their experiences, for instance making analogies to similar risks; and 4) economic phenomena such as opportunity costs and discounting. Conclusion: Most women completed the DCE in line with economic theory; however, violations were identified. Women appeared to visualise risk whether they received icon arrays or percentages only. Providing clear instructions and graphics to aid interpretation of risk and qualitative piloting to verify understanding is recommended. Further investigation is required to determine if the process of verbalising thoughts changes the behaviour of respondents.Key points:This study used cognitive interviews to explore how respondents traded-off risk attributes in a discrete choice experiment.The communication format of the risk attributes did not substantially alter choice making behaviour; however, visualisation of risk was supported by icon arrays.Although most respondents appeared to complete the experiment in line with economic theory, there were some violations of the underpinning axioms such as non-compensatory preferences.IntroductionDiscrete choice experiments (DCEs) are a survey-based method used to quantify preferences for characteristics (termed ‘attributes’) of different healthcare interventions, goods or services ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1136/qhc.0100055..", "ISBN" : "0963-8172 (Print)\\r0963-8172 (Linking)", "ISSN" : "1475-3898", "PMID" : "11533440", "abstract" : "This paper considers the application of discrete choice experiments for eliciting preferences in the delivery of health care. Drawing upon the results from a recently completed systematic review, the paper summarises the application of this technique in health care. It then presents a case study applying the technique to rheumatology outpatient clinics. 200 patients were questioned about the importance of six attributes: staff seen (junior doctor or specialist nurse); time in waiting area; continuity of contact with same staff; provision of a phone-in/advice service; length of consultation; and change in pain levels. The systematic review indicated that discrete choice experiments have been applied to a wide number of areas and a number of methodological issues have been addressed. Consistent with this literature, the case study found evidence of both rationality and theoretical validity of responses. The approach was used to establish the relative importance of different attributes, how individuals trade between these attributes, and overall benefit scores for different clinic configurations. The value of attributes was estimated in terms of time, and this was converted to a monetary measure using the value of waiting time for public transport. Discrete choice experiments represent a potentially useful instrument for eliciting preferences. Future methodological work should explore issues related to the experimental design of the study, methods of data collection and analysis, and satisfaction with the economic axioms of the instrument. Collaborative work with psychologists and qualitative researchers will prove useful in this research agenda.", "author" : [ { "dropping-particle" : "", "family" : "Ryan", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Bate", "given" : "a", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Eastmond", "given" : "C J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ludbrook", "given" : "a", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Quality in Health Care", "id" : "ITEM-1", "issue" : "Suppl I", "issued" : { "date-parts" : [ [ "2001" ] ] }, "page" : "i55-i60", "title" : "Use of discrete choice experiments to elicit preferences.", "type" : "article-journal", "volume" : "10" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[1]", "plainTextFormattedCitation" : "[1]", "previouslyFormattedCitation" : "[1]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[1]. In a DCE, the respondents are asked to state their preferred option in a series of hypothetical scenarios. From the choices made, it is possible to estimate how the survey respondent balanced (traded-off) between the different attributes when making choices. DCEs are underpinned by two key economic theories: Random Utility Theory (RUT) and Lancaster’s attribute-based theory of consumption ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Lancaster", "given" : "Kelvin J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "The Journal of Political Economy", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "1966" ] ] }, "page" : "132-157", "title" : "A new approach to consumer theory", "type" : "article-journal", "volume" : "74" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "McFadden", "given" : "Daniel", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Frontiers in Econometrics", "editor" : [ { "dropping-particle" : "", "family" : "Zarembka", "given" : "P", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "id" : "ITEM-2", "issued" : { "date-parts" : [ [ "1974" ] ] }, "page" : "105-142", "publisher" : "Academic Press INC", "publisher-place" : "New York", "title" : "Conditional logit analysis of qualitative choice behaviour", "type" : "chapter" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[2,3]", "plainTextFormattedCitation" : "[2,3]", "previouslyFormattedCitation" : "[2,3]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[2,3]. These theories inform the framework by which the choice data are analysed: that respondents are most likely to choose the option from each choice-set, described by the attribute-levels, which provides them with the most satisfaction or ‘utility’ ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/S0969-6997(01)00031-X", "ISSN" : "09696997", "author" : [ { "dropping-particle" : "", "family" : "Hensher", "given" : "D", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of Air Transport Management", "id" : "ITEM-1", "issue" : "6", "issued" : { "date-parts" : [ [ "2001", "11" ] ] }, "page" : "373-379", "title" : "An exploratory analysis of the effect of numbers of choice sets in designed choice experiments: an airline choice application", "type" : "article-journal", "volume" : "7" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "ISBN" : "0301-4797", "abstract" : "The Lake Champlain Basin in Vermont and New York, USA and Quebec, Canada includes a large lake and watershed with complex management issues. A transboundary comprehensive management plan prepared for the lake includes 11 goals across many issue areas. We developed a choice experiment to examine public preferences for alternative Lake Champlain management scenarios across these issue areas. Five ecosystem attributes (water clarity-algae blooms, public beach closures, land use change, fish consumption advisories and the spread of water chestnut, an invasive plant) were varied across three levels and arrayed into paired comparisons following an orthogonal fractional factorial design. Two thousand questionnaires were distributed to basin residents, each including nine paired comparisons that required trading off two, three or four attributes. Completed surveys yielded 6541 responses which were analyzed using binary logistic regression. The results showed that although water clarity and beach closures were important, safe fish consumption was the strongest predictor of choice. Land use pattern and water chestnut distribution were weaker but also significant predictors, with respondents preferring less land development and preservation of the agricultural landscape. Current management efforts in the Lake Champlain Basin are heavily weighted toward improving water clarity by reducing phosphorus pollution. Our results suggest that safe fish consumption warrants additional management attention. Because choice experiments provide information that is much richer than the simple categorical judgments more commonly used in surveys, they can provide managers with information about tradeoffs that could be used to enhance public support and maximize the social benefits of an ecosystem management program.", "author" : [ { "dropping-particle" : "", "family" : "Smyth", "given" : "Robyn L", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Watzin", "given" : "Mary C", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Manning", "given" : "Robert E", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of environmental management", "id" : "ITEM-2", "issue" : "1", "issued" : { "date-parts" : [ [ "2009" ] ] }, "page" : "615-623", "publisher" : "Rubenstein School of Environment and Natural Resources, Aiken Center, University of Vermont, Burlington, VT 05045, USA.", "publisher-place" : "England", "title" : "Investigating public preferences for managing Lake Champlain using a choice experiment.", "type" : "article-journal", "volume" : "90" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[4,5]", "plainTextFormattedCitation" : "[4,5]", "previouslyFormattedCitation" : "[4,5]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[4,5].As a ubiquitous component of healthcare delivery, risk is a commonly occurring attribute in DCEs eliciting individuals’ preferences for health-related goods or services ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s40271-014-0048-1", "ISBN" : "4027101400481", "ISSN" : "1178-1653", "PMID" : "24566923", "abstract" : "BACKGROUND: Discrete choice experiments (DCEs) are used to elicit preferences of current and future patients and healthcare professionals about how they value different aspects of healthcare. Risk is an integral part of most healthcare decisions. Despite the use of risk attributes in DCEs consistently being highlighted as an area for further research, current methods of incorporating risk attributes in DCEs have not been reviewed explicitly.\\n\\nOBJECTIVES: This study aimed to systematically identify published healthcare DCEs that incorporated a risk attribute, summarise and appraise methods used to present and analyse risk attributes, and recommend best practice regarding including, analysing and transparently reporting the methodology supporting risk attributes in future DCEs.\\n\\nDATA SOURCES: The Web of Science, MEDLINE, EMBASE, PsycINFO and Econlit databases were searched on 18 April 2013 for DCEs that included a risk attribute published since 1995, and on 23 April 2013 to identify studies assessing risk communication in the general (non-DCE) health literature.\\n\\nSTUDY ELIGIBILITY CRITERIA: Healthcare-related DCEs with a risk attribute mentioned or suggested in the title/abstract were obtained and retained in the final review if a risk attribute meeting our definition was included.\\n\\nSTUDY APPRAISAL AND SYNTHESIS METHODS: Extracted data were tabulated and critically appraised to summarise the quality of reporting, and the format, presentation and interpretation of the risk attribute were summarised.\\n\\nRESULTS: This review identified 117 healthcare DCEs that incorporated at least one risk attribute. Whilst there was some evidence of good practice incorporated into the presentation of risk attributes, little evidence was found that developing methods and recommendations from other disciplines about effective methods and validation of risk communication were systematically applied to DCEs. In general, the reviewed DCE studies did not thoroughly report the methodology supporting the explanation of risk in training materials, the impact of framing risk, or exploring the validity of risk communication.\\n\\nLIMITATIONS: The primary limitation of this review was that the methods underlying presentation, format and analysis of risk attributes could only be appraised to the extent that they were reported.\\n\\nCONCLUSIONS: Improvements in reporting and transparency of risk presentation from conception to the analysis of DCEs are needed. To define best practice\u2026", "author" : [ { "dropping-particle" : "", "family" : "Harrison", "given" : "Mark", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rigby", "given" : "Dan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Flynn", "given" : "Terry", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Louviere", "given" : "JJ Jordan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "The patient", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2014", "1" ] ] }, "page" : "151-70", "title" : "Risk as an attribute in discrete choice experiments: A systematic review of the literature", "type" : "article-journal", "volume" : "7" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[6]", "plainTextFormattedCitation" : "[6]", "previouslyFormattedCitation" : "[6]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[6]. There is also a rising interest in using DCEs as a means of providing information about preferences for the perceived balance between the benefits (desirable outcomes) and the likelihood (risk) of harms (unwanted outcomes) ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s40273-017-0518-0", "ISSN" : "11792027", "abstract" : "\u00a9 2017, The Author(s). There is emerging interest in the use of discrete choice experiments as a means of quantifying the perceived balance between benefits and risks (quantitative benefit-risk assessment) of new healthcare interventions, such as medicines, under assessment by regulatory agencies. For stated preference data on benefit-risk assessment to be used in regulatory decision making, the methods to generate these data must be valid, reliable and capable of producing meaningful estimates understood by decision makers. Some reporting guidelines exist for discrete choice experiments, and for related methods such as conjoint analysis. However, existing guidelines focus on reporting standards, are general in focus and do not consider the requirements for using discrete choice experiments specifically for quantifying benefit-risk assessments in the context of regulatory decision making. This opinion piece outlines the current state of play in using discrete choice experiments for benefit-risk assessment and proposes key areas needing to be addressed to demonstrate that discrete choice experiments are an appropriate and valid stated preference elicitation method in this context. Methodological research is required to establish: how robust the results of discrete choice experiments are to formats and methods of risk communication; how information in the discrete choice experiment can be presented effectually to respondents; whose preferences should be elicited; the correct underlying utility function and analytical model; the impact of heterogeneity in preferences; and the generalisability of the results. We believe these methodological issues should be addressed, alongside developing a \u2018reference case\u2019, before agencies can safely and confidently use discrete choice experiments for quantitative benefit-risk assessment in the context of regulatory decision making for new medicines and healthcare products.", "author" : [ { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "PharmacoEconomics", "id" : "ITEM-1", "issue" : "9", "issued" : { "date-parts" : [ [ "2017" ] ] }, "page" : "1-21", "title" : "Using Discrete Choice Experiments to Inform the Benefit \u00ad Risk Assessment of Medicines : Are We Ready Yet ?", "type" : "article-journal", "volume" : "35" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1007/s40258-013-0028-y", "ISSN" : "1179-1896", "PMID" : "23637054", "abstract" : "Decisions regarding the development, regulation, sale, and utilization of pharmaceutical and medical interventions require an evaluation of the balance between benefits and risks. Such evaluations are subject to two fundamental challenges-measuring the clinical effectiveness and harms associated with the treatment, and determining the relative importance of these different types of outcomes. In some ways, determining the willingness to accept treatment-related risks in exchange for treatment benefits is the greater challenge because it involves the individual subjective judgments of many decision makers, and these decision makers may draw different conclusions about the optimal balance between benefits and risks. In response to increasing demand for benefit-risk evaluations, researchers have applied a variety of existing welfare-theoretic preference methods for quantifying the tradeoffs decision makers are willing to accept among expected clinical benefits and risks. The methods used to elicit benefit-risk preferences have evolved from different theoretical backgrounds. To provide some structure to the literature that accommodates the range of approaches, we begin by describing a welfare-theoretic conceptual framework underlying the measurement of benefit-risk preferences in pharmaceutical and medical treatment decisions. We then review the major benefit-risk preference-elicitation methods in the empirical literature and provide a brief overview of the studies using each of these methods. The benefit-risk preference methods described in this overview fall into two broad categories: direct-elicitation methods and conjoint analysis. Rating scales (6 studies), threshold techniques (9 studies), and standard gamble (2 studies) are examples of direct elicitation methods. Conjoint analysis studies are categorized by the question format used in the study, including ranking (1 study), graded pairs (1 study), and discrete choice (21 studies). The number of studies reviewed here demonstrates that this body of research already is substantial, and it appears that the number of benefit-risk preference studies in the literature will continue to increase. In addition, benefit-risk preference-elicitation methods have been applied to a variety of healthcare decisions and medical interventions, including pharmaceuticals, medical devices, surgical and medical procedures, and diagnostics, as well as resource-allocation decisions such as facility placement. While preference-\u2026", "author" : [ { "dropping-particle" : "", "family" : "Hauber", "given" : "AB", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Fairchild", "given" : "Angelyn O", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Johnson", "given" : "FR", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Applied health economics and health policy", "id" : "ITEM-2", "issue" : "4", "issued" : { "date-parts" : [ [ "2013", "8" ] ] }, "page" : "319-29", "title" : "Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature.", "type" : "article-journal", "volume" : "11" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[7,8]", "plainTextFormattedCitation" : "[7,8]", "previouslyFormattedCitation" : "[7,8]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[7,8]. The quantification of benefit-risk trade-offs are being considered by national agencies to inform the regulation of new healthcare goods and services ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s00464-014-4044-2", "ISBN" : "0930-2794", "ISSN" : "14322218", "PMID" : "25552232", "abstract" : "BACKGROUND: Patients have a unique role in deciding what treatments should be available for them and regulatory agencies should take their preferences into account when making treatment approval decisions. This is the first study designed to obtain quantitative patient-preference evidence to inform regulatory approval decisions by the Food and Drug Administration Center for Devices and Radiological Health. METHODS: Five-hundred and forty United States adults with body mass index (BMI) \u226530 kg/m(2) evaluated tradeoffs among effectiveness, safety, and other attributes of weight-loss devices in a scientific survey. Discrete-choice experiments were used to quantify the importance of safety, effectiveness, and other attributes of weight-loss devices to obese respondents. A tool based on these measures is being used to inform benefit-risk assessments for premarket approval of medical devices. RESULTS: Respondent choices yielded preference scores indicating their relative value for attributes of weight-loss devices in this study. We developed a tool to estimate the minimum weight loss acceptable by a patient to receive a device with a given risk profile and the maximum mortality risk tolerable in exchange for a given weight loss. For example, to accept a device with 0.01 % mortality risk, a risk tolerant patient will require about 10 % total body weight loss lasting 5 years. CONCLUSIONS: Patient preference evidence was used make regulatory decision making more patient-centered. In addition, we captured the heterogeneity of patient preferences allowing market approval of effective devices for risk tolerant patients. CDRH is using the study tool to define minimum clinical effectiveness to evaluate new weight-loss devices. The methods presented can be applied to a wide variety of medical products. This study supports the ongoing development of a guidance document on incorporating patient preferences into medical-device premarket approval decisions.", "author" : [ { "dropping-particle" : "", "family" : "Ho", "given" : "Martin P.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Gonzalez", "given" : "Juan Marcos", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lerner", "given" : "Herbert P.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Neuland", "given" : "Carolyn Y.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Whang", "given" : "Joyce M.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "McMurry-Heath", "given" : "Michelle", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Brett Hauber", "given" : "a.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Irony", "given" : "Telba", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Surgical Endoscopy and Other Interventional Techniques", "id" : "ITEM-1", "issue" : "10", "issued" : { "date-parts" : [ [ "2015" ] ] }, "page" : "2984-2993", "title" : "Incorporating patient-preference evidence into regulatory decision making", "type" : "article-journal", "volume" : "29" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1016/j.jval.2016.06.004", "ISSN" : "10983015", "author" : [ { "dropping-particle" : "", "family" : "Reed", "given" : "Shelby D.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lavezzari", "given" : "Gabriela", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-2", "issue" : "6", "issued" : { "date-parts" : [ [ "2016" ] ] }, "page" : "727-729", "title" : "International Experiences in Quantitative Benefit-Risk Analysis to Support Regulatory Decisions", "type" : "article", "volume" : "19" }, "uris" : [ "" ] }, { "id" : "ITEM-3", "itemData" : { "DOI" : "10.1016/j.jval.2016.04.006", "ISSN" : "15244733", "abstract" : "Regulatory decisions are often based on multiple clinical end points, but the perspectives used to judge the relative importance of those end points are predominantly those of expert decision makers rather than of the patient. However, there is a growing awareness that active patient and public participation can improve decision making, increase acceptance of decisions, and improve adherence to treatments. The assessment of risk versus benefit requires not only information on clinical outcomes but also value judgments about which outcomes are important and whether the potential benefits outweigh the harms. There are a number of mechanisms for capturing the input of patients, and regulatory bodies within the European Union are participating in several initiatives. These can include patients directly participating in the regulatory decision-making process or using information derived from patients in empirical studies as part of the evidence considered. One promising method that is being explored is the elicitation of ???patient preferences.??? Preferences, in this context, refer to the individual's evaluation of health outcomes and can be understood as statements regarding the relative desirability of a range of treatment options, treatment characteristics, and health states. Several methods for preference measurement have been proposed, and pilot studies have been undertaken to use patient preference information in regulatory decision making. This article describes how preferences are currently being considered in the benefit-risk assessment context, and shows how different methods of preference elicitation are used to support decision making within the European context.", "author" : [ { "dropping-particle" : "", "family" : "Muhlbacher", "given" : "Axel C.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Juhnke", "given" : "Christin", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Beyer", "given" : "Andrea R.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Garner", "given" : "Sarah", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-3", "issue" : "6", "issued" : { "date-parts" : [ [ "2016" ] ] }, "page" : "734-740", "title" : "Patient-Focused Benefit-Risk Analysis to Inform Regulatory Decisions: The European Union Perspective", "type" : "article-journal", "volume" : "19" }, "uris" : [ "" ] }, { "id" : "ITEM-4", "itemData" : { "DOI" : "10.1016/j.jval.2016.04.008", "ISSN" : "15244733", "abstract" : "Demands for greater transparency in US regulatory assessments of benefits and risks, together with growing interest in engaging patients in Food and Drug Administration regulatory decision making, have resulted in several recent regulatory developments. Although Food and Drug Administration's Center for Drug Evaluation and Research (CDER) and Center for Devices and Radiological Health (CDRH) have established patient-engagement initiatives, CDRH has proposed guidelines for considering quantitative data on patients??? benefit-risk perspectives, while CDER has focused on a more qualitative approach. We summarize two significant studies that were developed in collaboration and consultation with CDER and CDRH. CDER encouraged a patient advocacy group to propose draft guidance on engaging patient and caregiver stakeholders in regulatory decision making for Duchenne muscular dystrophy. CDRH sponsored a discrete-choice experiment case study to quantify obese respondents??? perspectives on ???meaningful benefits.??? CDRH and CDER issued draft guidance in May and June 2015, respectively, on including patient-preference data in regulatory submissions. Both organizations face challenges. CDER is working on integrating qualitative data into existing evidence-based review processes and is exploring options for therapeutic areas not included on a priority list. CDRH has adopted an approach that requires patient-preference data to satisfy standards of valid scientific evidence. Although that strategy could facilitate integrating patient perspectives directly with clinical data on benefits and harms, generating such data requires building capacity.", "author" : [ { "dropping-particle" : "", "family" : "Johnson", "given" : "F. Reed", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Zhou", "given" : "Mo", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-4", "issue" : "6", "issued" : { "date-parts" : [ [ "2016" ] ] }, "page" : "741-745", "title" : "Patient Preferences in Regulatory Benefit-Risk Assessments: A US Perspective", "type" : "article-journal", "volume" : "19" }, "uris" : [ "" ] }, { "id" : "ITEM-5", "itemData" : { "DOI" : "10.1007/s40273-017-0518-0", "ISSN" : "11792027", "abstract" : "\u00a9 2017, The Author(s). There is emerging interest in the use of discrete choice experiments as a means of quantifying the perceived balance between benefits and risks (quantitative benefit-risk assessment) of new healthcare interventions, such as medicines, under assessment by regulatory agencies. For stated preference data on benefit-risk assessment to be used in regulatory decision making, the methods to generate these data must be valid, reliable and capable of producing meaningful estimates understood by decision makers. Some reporting guidelines exist for discrete choice experiments, and for related methods such as conjoint analysis. However, existing guidelines focus on reporting standards, are general in focus and do not consider the requirements for using discrete choice experiments specifically for quantifying benefit-risk assessments in the context of regulatory decision making. This opinion piece outlines the current state of play in using discrete choice experiments for benefit-risk assessment and proposes key areas needing to be addressed to demonstrate that discrete choice experiments are an appropriate and valid stated preference elicitation method in this context. Methodological research is required to establish: how robust the results of discrete choice experiments are to formats and methods of risk communication; how information in the discrete choice experiment can be presented effectually to respondents; whose preferences should be elicited; the correct underlying utility function and analytical model; the impact of heterogeneity in preferences; and the generalisability of the results. We believe these methodological issues should be addressed, alongside developing a \u2018reference case\u2019, before agencies can safely and confidently use discrete choice experiments for quantitative benefit-risk assessment in the context of regulatory decision making for new medicines and healthcare products.", "author" : [ { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "PharmacoEconomics", "id" : "ITEM-5", "issue" : "9", "issued" : { "date-parts" : [ [ "2017" ] ] }, "page" : "1-21", "title" : "Using Discrete Choice Experiments to Inform the Benefit \u00ad Risk Assessment of Medicines : Are We Ready Yet ?", "type" : "article-journal", "volume" : "35" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[7,9\u201312]", "plainTextFormattedCitation" : "[7,9\u201312]", "previouslyFormattedCitation" : "[7,9\u201312]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[7,9–12]. In 2016, the Food and Drug Administration (FDA) released draft guidance on the methods to quantify preferences for benefit-risk trade-offs to make regulatory decisions about medical devices ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "FDA", "given" : "", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "U.S. Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2016" ] ] }, "title" : "Patient Preference Information Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applicationsm and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling", "type" : "article-journal", "volume" : "FDA-2015-D" }, "uris" : [ "", "" ] } ], "mendeley" : { "formattedCitation" : "[13]", "plainTextFormattedCitation" : "[13]", "previouslyFormattedCitation" : "[13]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[13]. Decision-makers working in a regulatory capacity must be confident that the methods used to quantify preferences are robust and reliable ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s40273-017-0518-0", "ISSN" : "11792027", "abstract" : "\u00a9 2017, The Author(s). There is emerging interest in the use of discrete choice experiments as a means of quantifying the perceived balance between benefits and risks (quantitative benefit-risk assessment) of new healthcare interventions, such as medicines, under assessment by regulatory agencies. For stated preference data on benefit-risk assessment to be used in regulatory decision making, the methods to generate these data must be valid, reliable and capable of producing meaningful estimates understood by decision makers. Some reporting guidelines exist for discrete choice experiments, and for related methods such as conjoint analysis. However, existing guidelines focus on reporting standards, are general in focus and do not consider the requirements for using discrete choice experiments specifically for quantifying benefit-risk assessments in the context of regulatory decision making. This opinion piece outlines the current state of play in using discrete choice experiments for benefit-risk assessment and proposes key areas needing to be addressed to demonstrate that discrete choice experiments are an appropriate and valid stated preference elicitation method in this context. Methodological research is required to establish: how robust the results of discrete choice experiments are to formats and methods of risk communication; how information in the discrete choice experiment can be presented effectually to respondents; whose preferences should be elicited; the correct underlying utility function and analytical model; the impact of heterogeneity in preferences; and the generalisability of the results. We believe these methodological issues should be addressed, alongside developing a \u2018reference case\u2019, before agencies can safely and confidently use discrete choice experiments for quantitative benefit-risk assessment in the context of regulatory decision making for new medicines and healthcare products.", "author" : [ { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "PharmacoEconomics", "id" : "ITEM-1", "issue" : "9", "issued" : { "date-parts" : [ [ "2017" ] ] }, "page" : "1-21", "title" : "Using Discrete Choice Experiments to Inform the Benefit \u00ad Risk Assessment of Medicines : Are We Ready Yet ?", "type" : "article-journal", "volume" : "35" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[7]", "plainTextFormattedCitation" : "[7]", "previouslyFormattedCitation" : "[7]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[7]. A key part of generating reliable values and inferences from DCEs is communicating the attributes and levels effectively to respondents ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1002/hec", "author" : [ { "dropping-particle" : "", "family" : "Coast", "given" : "Joanna", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Al-Janabi", "given" : "Hareth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Sutton", "given" : "Eileen", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Horrocks", "given" : "Susan A", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Vosper", "given" : "Jane", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Swancutt", "given" : "Dawn R", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Flynn", "given" : "Terry", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-1", "issue" : "6", "issued" : { "date-parts" : [ [ "2012" ] ] }, "page" : "730-741", "title" : "Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations.", "type" : "article-journal", "volume" : "21" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[14]", "plainTextFormattedCitation" : "[14]", "previouslyFormattedCitation" : "[14]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[14]. Risk is a complex and multifaceted concept which can make communicating probabilistic information challenging ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "ISSN" : "08955646", "abstract" : "Efficient investments in health protection require valid estimates of the public's willingness to forgo consumption for diminished probabilities of death, injury, and disease. Stated valuations of risk reduction are not valid measures of economic preference if the valuations are insensitive to probability variation. This article reviews the existing literature on CV studies of reductions in health risk and finds that most studies are poorly designed to assess the sensitivity of stated valuations to changes in risk - magnitude. Replication of a recent study published in this journal by Johannesson et al. 1997 demonstrates how serious the problem of insensitivity can be, even for a study that reports plausible results. New empirical results are presented from telephone surveys designed to provide internal and external tests of how WTP responds to size of risk reduction. The effect of variations in instrument design on estimated sensitivity to magnitude is examined. Overall, estimated WTP for risk reduction is inadequately sensitive to the difference in probability, that is, the magnitude of the difference in WTP for different reductions in risk is typically smaller than suggested by standard economic theory. Additional research to improve methods for communicating changes in risk is needed, and future studies of stated WTP to reduce risk should include rigorous validity checks. PUBLICATION ABSTRACT", "author" : [ { "dropping-particle" : "", "family" : "Hammitt", "given" : "James K", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Graham", "given" : "John D", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of Risk and Uncertainty", "id" : "ITEM-1", "issue" : "1", "issued" : { "date-parts" : [ [ "1999" ] ] }, "note" : "\n From Duplicate 1 ( \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n \n\n \n\n \n\n \n \n\n \n \n\n \n \n Willingness to Pay for Health Protection : Inadequate Sensitivity to Probability ?\n \n \n \n\n \n \n\n \n \n\n \n\n \n\n \n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n - Hammitt, James K; Graham, John D )\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \nPaper on the classification of risk \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n From Duplicate 2 ( \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n \n\n \n\n \n\n \n \n\n \n \n\n \n \n Willingness to Pay for Health Protection: Inadequate Sensitivity to Probability?\n \n \n \n\n \n \n\n \n \n\n \n\n \n\n \n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n - Hammitt, James K; Graham, John D )\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n \n\n \n\n \n\n ", "page" : "33-62", "title" : "Willingness to pay for health protection: inadequate sensitivity to probability?", "type" : "article-journal", "volume" : "18" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1177/0272989X07307271", "ISSN" : "0272-989X", "PMID" : "17873259", "abstract" : "Perception of health risk can affect medical decisions and health behavior change. Yet the concept of risk is a difficult one for the public to grasp. Whether perceptions of risk affect decisions and behaviors often relies on how messages of risk magnitudes (i.e., likelihood) are conveyed. Based on expert opinion, this article offers, when possible, best practices for conveying magnitude of health risks using numeric, verbal, and visual formats. This expert opinion is based on existing empirical evidence, review of papers and books, and consultations with experts in risk communication. This article also discusses formats to use pertaining to unique risk communication challenges (e.g., conveying small-probability events, interactions). Several recommendations are suggested for enhancing precision in perception of risk by presenting risk magnitudes numerically and visually. Overall, there are little data to suggest best practices for verbal communication of risk magnitudes. Across the 3 formats, few overall recommendations could be suggested because of 1) lack of consistency in testing formats using the same outcomes in the domain of interest, 2) lack of critical tests using randomized controlled studies pitting formats against one another, and 3) lack of theoretical progress detailing and testing mechanisms why one format should be more efficacious in a specific context to affect risk magnitudes than others. Areas of future research are provided that it is hoped will help illuminate future best practices.", "author" : [ { "dropping-particle" : "", "family" : "Lipkus", "given" : "I", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical Decision Making", "id" : "ITEM-2", "issue" : "5", "issued" : { "date-parts" : [ [ "2007" ] ] }, "page" : "696-713", "title" : "Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations.", "type" : "article-journal", "volume" : "27" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[15,16]", "plainTextFormattedCitation" : "[15,16]", "previouslyFormattedCitation" : "[15,16]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[15,16]. There is a substantial evidence base to suggest many people find ‘risk’ a difficult concept to understand, regardless of demographic or education ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "ISSN" : "0272-989X", "PMID" : "11206945", "abstract" : "Numeracy, how facile people are with basic probability and mathematical concepts, is associated with how people perceive health risks. Performance on simple numeracy problems has been poor among populations with little as well as more formal education. Here, we examine how highly educated participants performed on a general and an expanded numeracy scale. The latter was designed within the context of health risks.", "author" : [ { "dropping-particle" : "", "family" : "Lipkus", "given" : "I", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Samsa", "given" : "G", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rimer", "given" : "Barbara", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical decision making", "id" : "ITEM-1", "issue" : "1", "issued" : { "date-parts" : [ [ "2001" ] ] }, "page" : "37-44", "title" : "General performance on a numeracy scale among highly educated samples.", "type" : "article-journal", "volume" : "21" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Gigerenzer", "given" : "Gerd", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Gaissmaier", "given" : "W", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Kurz-Milcke", "given" : "Elke", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Schwartz", "given" : "LM", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Woloshin", "given" : "S", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Psychological science in the public interest", "id" : "ITEM-2", "issue" : "2", "issued" : { "date-parts" : [ [ "2007" ] ] }, "page" : "53-96", "title" : "Helping doctors and patients make sense of health statistics", "type" : "article-journal", "volume" : "8" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[17,18]", "plainTextFormattedCitation" : "[17,18]", "previouslyFormattedCitation" : "[17,18]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[17,18]. In a DCE, differences in the magnitude of risk (in levels) must be understood and then considered alongside the other attributes. It has been found that DCE respondents may have not have understood the risk attributes as presented ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1111/j.1524-4733.2008.00451.x", "ISBN" : "1524-4733", "ISSN" : "1524-4733", "PMID" : "18783389", "abstract" : "OBJECTIVE: To examine women's preferences for characteristics of chlamydia screening. Chlamydia trachomatis is the most common curable sexually transmitted disease. To design effective screening programs, it is important to fully capture the benefits of screening to patients. Thus, the value of experience factors must be considered alongside health outcomes., METHODS: A self-complete discrete choice experiment questionnaire was administered to women attending a family planning clinic. Chlamydia screening was described by five characteristics: location of screening; type of screening test; cost of screening test; risk of developing pelvic inflammatory disease if chlamydia is untreated; and support provided when receiving results., RESULTS: One hundred twenty-six women completed the questionnaire. Respondents valued characteristics of the care experience. Screening was valued at 15 pound; less invasive screening tests increase willingness to pay by 7 pound, and more invasive tests reduce willingness to pay by 3.50 pound. The most preferred screening location was the family planning clinic, valued at 5 pound. The support of a trained health-care professional when receiving results was valued at 4 pound. Respondents under 25 years and those in a casual relationship were less likely to be screened., CONCLUSIONS: Women valued experience factors in the provision of chlamydia screening. To correctly value these screening programs and to predict uptake, cost-effectiveness studies should take such values into account. Failure to do this may result in incorrect policy recommendations.", "author" : [ { "dropping-particle" : "", "family" : "Watson", "given" : "V", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ryan", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Watson", "given" : "Emma", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-1", "issue" : "4", "issued" : { "date-parts" : [ [ "2009", "6" ] ] }, "note" : "From Duplicate 1 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of chlamydia screening: an application to women attending the family planning clinic\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, V; Ryan, M; Watson, E )\nAnd Duplicate 2 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, Verity; Ryan, M; Watson, Emma )\nAnd Duplicate 3 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, V; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nFrom Duplicate 4 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, V; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nFrom Duplicate 1 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, Verity; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nFrom Duplicate 2 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, Verity; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\u00a0\u00a0\u00a0\u00a0\u00a0 (50) \u00a0\u00a0 Watson V, Ryan M, Watson E. Valuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic. Value in Health 12(4):621-3, 2009 Jun. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nQualitative Methods\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThis study provides no detail about qualitative methods employed.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nAttributes were chosen &quot;Based on a literature review, policy variations, and advice from the family planning clinic\u2019s doctor, five screening attributes were identified.&quot; (p. 621)\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe DCEs were administered as a survey on arrival to clinic. Qualitative methods could have contributed to the authors' understanding of non-responses, whether the attributes were relevant and whether the respondents understood the task/attributes/levels. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThey even conclude that &quot;Future work should explore if other attributes are important to this group&quot; (p. 623) and &quot;the insignificance of the risk of pelvic inflammatory disease may reflect the difficulties that respondents had in understanding this attribute&quot; (p.623)", "page" : "621-623", "publisher" : "Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, UK. v.watson@abdn.ac.uk", "publisher-place" : "United States", "title" : "Valuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.", "type" : "article-journal", "volume" : "12" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[19]", "plainTextFormattedCitation" : "[19]", "previouslyFormattedCitation" : "[19]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[19] and presentation of risk in DCEs has not aligned with best practices in the risk communication literature ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s40271-014-0048-1", "ISBN" : "4027101400481", "ISSN" : "1178-1653", "PMID" : "24566923", "abstract" : "BACKGROUND: Discrete choice experiments (DCEs) are used to elicit preferences of current and future patients and healthcare professionals about how they value different aspects of healthcare. Risk is an integral part of most healthcare decisions. Despite the use of risk attributes in DCEs consistently being highlighted as an area for further research, current methods of incorporating risk attributes in DCEs have not been reviewed explicitly.\\n\\nOBJECTIVES: This study aimed to systematically identify published healthcare DCEs that incorporated a risk attribute, summarise and appraise methods used to present and analyse risk attributes, and recommend best practice regarding including, analysing and transparently reporting the methodology supporting risk attributes in future DCEs.\\n\\nDATA SOURCES: The Web of Science, MEDLINE, EMBASE, PsycINFO and Econlit databases were searched on 18 April 2013 for DCEs that included a risk attribute published since 1995, and on 23 April 2013 to identify studies assessing risk communication in the general (non-DCE) health literature.\\n\\nSTUDY ELIGIBILITY CRITERIA: Healthcare-related DCEs with a risk attribute mentioned or suggested in the title/abstract were obtained and retained in the final review if a risk attribute meeting our definition was included.\\n\\nSTUDY APPRAISAL AND SYNTHESIS METHODS: Extracted data were tabulated and critically appraised to summarise the quality of reporting, and the format, presentation and interpretation of the risk attribute were summarised.\\n\\nRESULTS: This review identified 117 healthcare DCEs that incorporated at least one risk attribute. Whilst there was some evidence of good practice incorporated into the presentation of risk attributes, little evidence was found that developing methods and recommendations from other disciplines about effective methods and validation of risk communication were systematically applied to DCEs. In general, the reviewed DCE studies did not thoroughly report the methodology supporting the explanation of risk in training materials, the impact of framing risk, or exploring the validity of risk communication.\\n\\nLIMITATIONS: The primary limitation of this review was that the methods underlying presentation, format and analysis of risk attributes could only be appraised to the extent that they were reported.\\n\\nCONCLUSIONS: Improvements in reporting and transparency of risk presentation from conception to the analysis of DCEs are needed. To define best practice\u2026", "author" : [ { "dropping-particle" : "", "family" : "Harrison", "given" : "Mark", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rigby", "given" : "Dan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Flynn", "given" : "Terry", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Louviere", "given" : "JJ Jordan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "The patient", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2014", "1" ] ] }, "page" : "151-70", "title" : "Risk as an attribute in discrete choice experiments: A systematic review of the literature", "type" : "article-journal", "volume" : "7" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[6]", "plainTextFormattedCitation" : "[6]", "previouslyFormattedCitation" : "[6]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[6].If attributes are not understood by respondents, they may fail to trade-off appropriately or adopt simplifying heuristics to complete the choice sets. Non-trading behaviour may violate the key preference axioms underpinning DCEs as a preference elicitation method ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1002/hec", "abstract" : "Stated preference methods assume respondents' preferences are consistent with utility theory, but many empirical studies report evidence of preferences that violate utility theory. This evidence is often derived from quantitative tests that occur naturally within, or are added to, stated preference tasks. In this study, we use qualitative methods to explore three axioms of utility theory: completeness, monotonicity, and continuity. We take a novel approach, adopting a 'think aloud' technique to identify violations of the axioms of utility theory and to consider how well the quantitative tests incorporated within a discrete choice experiment are able to detect these. Results indicate that quantitative tests classify respondents as being 'irrational' when qualitative statements would indicate they are 'rational'. In particular, 'non-monotonic' responses can often be explained by respondents inferring additional information beyond what is presented in the task, and individuals who appear to adopt non-compensatory decision-making strategies do so because they rate particular attributes very highly (they are not attempting to simplify the task). The results also provide evidence of 'cost-based responses': respondents assumed tests with higher costs would be of higher quality. The value of including in-depth qualitative validation techniques in the development of stated preference tasks is shown. Copyright 2008 John Wiley & Sons, Ltd", "author" : [ { "dropping-particle" : "", "family" : "Ryan", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Watson", "given" : "V", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Entwistle", "given" : "Vikki", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2009" ] ] }, "page" : "321-336", "title" : "Rationalising the 'irrational': a think aloud study of a discrete choice experiment responses", "type" : "article-journal", "volume" : "18" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[20]", "plainTextFormattedCitation" : "[20]", "previouslyFormattedCitation" : "[20]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[20]. If the attribute is ignored completely, then ‘attribute non-attendance’ occurs, violating the assumption of continuity in preferences ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1002/hec", "author" : [ { "dropping-particle" : "", "family" : "Lagarde", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-1", "issue" : "5", "issued" : { "date-parts" : [ [ "2012" ] ] }, "page" : "554-567", "title" : "Investigating attribute non-attendance and its consequences in choice experiments with latent class models", "type" : "article-journal", "volume" : "22" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[21]", "plainTextFormattedCitation" : "[21]", "previouslyFormattedCitation" : "[21]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[21]. This can be identified with quantitative methods ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1002/hec", "author" : [ { "dropping-particle" : "", "family" : "Lagarde", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-1", "issue" : "5", "issued" : { "date-parts" : [ [ "2012" ] ] }, "page" : "554-567", "title" : "Investigating attribute non-attendance and its consequences in choice experiments with latent class models", "type" : "article-journal", "volume" : "22" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[21]", "plainTextFormattedCitation" : "[21]", "previouslyFormattedCitation" : "[21]" }, "properties" : { "noteIndex" : 4 }, "schema" : "" }[21]; however, whether this is a preference or a heuristic is unknown to the analyst. Qualitative research methods, such as cognitive interviews, can be used to discover more about individuals’ thoughts and reasoning ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1111/j.1365-2702.2010.03578.x", "ISBN" : "0962-1067", "ISSN" : "09621067", "PMID" : "21689181", "abstract" : "AIM: The aim of this study was to explore the thinking strategies and clinical reasoning processes registered nurses use during simulated care planning for malnutrition and pressure ulcers in nursing home care. BACKGROUND: Clinical reasoning is an essential component of nursing practice. Registered nurses' thinking strategies and clinical reasoning have received limited attention in nursing science. Further research is needed to understand registered nurses' clinical reasoning, especially for prevention of malnutrition and pressure ulcers as they are important quality indicators of resident care in nursing homes. DESIGN: A qualitative explorative design was used with a think-aloud interview technique. METHODS: The transcribed verbalisations were analysed with qualitative deductive content analysis. Data were collected during six months in 2007-2008 from 30 registered nurses at nine nursing homes in Norway. RESULTS: The registered nurses used a variety of thinking strategies, but there were differences in the frequency of use of the different strategies. The three most commonly used thinking strategies were 'making choices', 'forming relationships' and 'drawing conclusions'. None of the nurses performed a structured risk assessment of malnutrition or pressure ulcers. Registered nurses started with assessing data from the scenarios, but after a short and elementary assessment they moved directly to planning. CONCLUSION: Many different thinking strategies were used in registered nurses' clinical reasoning for prevention of malnutrition and pressure ulcers. The thinking strategy 'making choices' was most commonly used and registered nurses' main focus in their reasoning was on planning nursing interventions. RELEVANCE TO CLINICAL PRACTICE: This study showed that most of the registered nurses go directly to planning when reasoning clinically about residents in nursing homes. A lack of systematic risk assessments was identified. The insight gained from this study can be used to recommend improvements in tools designed for nursing homes to support the registered nurses.", "author" : [ { "dropping-particle" : "", "family" : "Fossum", "given" : "Mariann", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Alexander", "given" : "Gregory L", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "G\u00f6ransson", "given" : "Katarina E", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ehnfors", "given" : "Margareta", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ehrenberg", "given" : "Anna", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of Clinical Nursing", "id" : "ITEM-1", "issue" : "17-18", "issued" : { "date-parts" : [ [ "2011", "9" ] ] }, "page" : "2425-2435", "publisher" : "Wiley/Blackwell (10.1111)", "title" : "Registered nurses' thinking strategies on malnutrition and pressure ulcers in nursing homes: A scenario-based think-aloud study", "type" : "article-journal", "volume" : "20" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1016/S1499-4046(06)60200-5", "ISBN" : "1499-4046(Print)", "ISSN" : "14994046", "PMID" : "12773287", "abstract" : "The purpose of this report is to describe the development and implementation of the think aloud method in relation to fruit and vegetable purchasing behaviors of low-income African American mothers. Women (n = 70) were audio-taped as they thought aloud while selecting fruits and vegetables during a routine shopping trip. Audiotapes were transcribed, text was coded, and coded text was sorted using a database software program. Data were analyzed using content analysis procedures. The method was found to be useful in its ability to provide verbalization data for the majority of the women in the sample that reflected a typical shopping experience, were not excessively affected by the presence of the investigator, and captured information processing in relation to salient factors that influenced food purchasing decisions. Because a few women indicated that the method itself may have influenced behavior, future research is needed to test the reactivity of the think aloud method and its relationship to final choice of products.", "author" : [ { "dropping-particle" : "", "family" : "Reicks", "given" : "Marla", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Smith", "given" : "Chery", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Henry", "given" : "Helen", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Reimer", "given" : "Kathy", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Atwell", "given" : "Janine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Thomas", "given" : "Ruth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of Nutrition Education and Behavior", "id" : "ITEM-2", "issue" : "3", "issued" : { "date-parts" : [ [ "2003" ] ] }, "page" : "154-160", "title" : "Use of the think aloud method to examine fruit and vegetable purchasing behaviors among low-income African American women", "type" : "article-journal", "volume" : "35" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[22,23]", "plainTextFormattedCitation" : "[22,23]", "previouslyFormattedCitation" : "[22,23]" }, "properties" : { "noteIndex" : 4 }, "schema" : "" }[22,23] and, in the case of DCEs, pilot the survey or understand how respondents complete choice tasks ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1177/0272989X16683934", "ISSN" : "0272-989X", "author" : [ { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rigby", "given" : "Dan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical Decision Making", "id" : "ITEM-1", "issue" : "3", "issued" : { "date-parts" : [ [ "2017" ] ] }, "page" : "298-313", "title" : "The Role of Qualitative Research Methods in Discrete Choice Experiments: A Systematic Review and Survey of Authors", "type" : "article-journal", "volume" : "37" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1016/j.socscimed.2008.05.015", "ISSN" : "0277-9536", "PMID" : "18572295", "abstract" : "This paper reports the first application of the capabilities approach to the development and valuation of an instrument for use in the economic evaluation of health and social care interventions. The ICECAP index of capability for older people focuses on quality of life rather than health or other influences on quality of life, and is intended to be used in decision making across health and social care in the UK. The measure draws on previous qualitative work in which five conceptual attributes were developed: attachment, security, role, enjoyment and control. This paper details the innovative use within health economics of further iterative qualitative work in the UK among 19 informants to refine lay terminology for each of the attributes and levels of attributes used in the eventual index. For the first time within quality of life measurement for economic evaluation, a best-worst scaling exercise has been used to estimate general population values (albeit for the population of those aged 65+ years) for the levels of attributes, with values anchored at one for full capability and zero for no capability. Death was assumed to be a state in which there is no capability. The values obtained indicate that attachment is the attribute with greatest impact but all attributes contribute to the total estimation of capability. Values that were estimated are feasible for use in practical applications of the index to measure the impact of health and social care interventions.", "author" : [ { "dropping-particle" : "", "family" : "Coast", "given" : "Joanna", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Flynn", "given" : "Terry", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Natarajan", "given" : "Lucy", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Sproston", "given" : "Kerry", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lewis", "given" : "Jane", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Louviere", "given" : "JJ", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Peters", "given" : "Tim", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Social science & medicine", "id" : "ITEM-2", "issue" : "5", "issued" : { "date-parts" : [ [ "2008", "9" ] ] }, "page" : "874-82", "title" : "Valuing the ICECAP capability index for older people.", "type" : "article-journal", "volume" : "67" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[24,25]", "plainTextFormattedCitation" : "[24,25]", "previouslyFormattedCitation" : "[24,25]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[24,25]. This can reveal whether their behaviour aligns with a priori expectations or whether it results in violation of the underlying utility theories. Existing studies have illustrated the potential power of qualitative research methods to reveal influences on choice or decision strategies that may be difficult, or impossible, to measure in traditional quantitative surveys alone ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1002/hec", "abstract" : "Stated preference methods assume respondents' preferences are consistent with utility theory, but many empirical studies report evidence of preferences that violate utility theory. This evidence is often derived from quantitative tests that occur naturally within, or are added to, stated preference tasks. In this study, we use qualitative methods to explore three axioms of utility theory: completeness, monotonicity, and continuity. We take a novel approach, adopting a 'think aloud' technique to identify violations of the axioms of utility theory and to consider how well the quantitative tests incorporated within a discrete choice experiment are able to detect these. Results indicate that quantitative tests classify respondents as being 'irrational' when qualitative statements would indicate they are 'rational'. In particular, 'non-monotonic' responses can often be explained by respondents inferring additional information beyond what is presented in the task, and individuals who appear to adopt non-compensatory decision-making strategies do so because they rate particular attributes very highly (they are not attempting to simplify the task). The results also provide evidence of 'cost-based responses': respondents assumed tests with higher costs would be of higher quality. The value of including in-depth qualitative validation techniques in the development of stated preference tasks is shown. Copyright 2008 John Wiley & Sons, Ltd", "author" : [ { "dropping-particle" : "", "family" : "Ryan", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Watson", "given" : "V", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Entwistle", "given" : "Vikki", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2009" ] ] }, "page" : "321-336", "title" : "Rationalising the 'irrational': a think aloud study of a discrete choice experiment responses", "type" : "article-journal", "volume" : "18" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "PMID" : "17478438", "abstract" : "BACKGROUND: Delivering effective health care within limited budgets requires an understanding of patient priorities. Discrete choice experiments (DCEs) provide patients with choices, where each choice differs in terms of certain attributes (such as waiting times, quality of care). Although this technique has significant potential in examining priorities, its use raises practical and conceptual issues. This paper describes the development of a DCE evaluating patient priorities in primary care. METHODS: Twenty patients completed a DCE using a 'think aloud' protocol, where they verbalized their thinking while making choices. The analysis examined their decision-making processes. RESULTS: There was evidence that patients reinterpreted some attributes, and related some to others outside the task. The cost attribute was interpreted in a variety of ways, dominating some patients' decision-making, being seen as irrelevant by others and being interpreted appropriately by some. The deree to which patients exhibited trading in line with theoretical assumptions also varied. Some choices in the hypothetical task were restricted by their previous experience, but more frequently patients tested the boundaries of the task in ways which directly reflected the primary care context. CONCLUSION: Patient interpretation of the discrete choice task was varied and some went beyond the formal boundaries of the task to make their choices. This highlights the importance of piloting attributes, providing clear instructions about the task and developing models of patient decision-making so that responses can be interpreted correctly.", "author" : [ { "dropping-particle" : "", "family" : "Cheraghi-Sohi", "given" : "S", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Bower", "given" : "P", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Mead", "given" : "Nicola", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "McDonald", "given" : "Ruth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Whalley", "given" : "Diane", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Roland", "given" : "Martin", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Family Practice", "id" : "ITEM-2", "issue" : "3", "issued" : { "date-parts" : [ [ "2007" ] ] }, "page" : "276-282", "title" : "Making sense of patient priorities: applying discrete choice methods in primary care using 'think aloud' technique.", "type" : "article-journal", "volume" : "24" }, "uris" : [ "" ] }, { "id" : "ITEM-3", "itemData" : { "ISBN" : "1544-1717", "abstract" : "PURPOSE: The consultation is fundamental to the delivery of primary care, but different ways of organizing consultations may lead to different patient experiences in terms of access, continuity, technical quality of care, and communication. Patients' priorities for these different issues need to be understood, but the optimal methods for assessing priorities are unclear. This study used a discrete choice experiment to assess patients' priorities., METHODS: We surveyed patients from 6 family practices in England. The patients chose between primary care consultations differing in attributes such as ease of access (wait for an appointment), choice (flexibility of appointment times), continuity (physician's knowledge of the patient), technical quality (thoroughness of physical examination), and multiple aspects of patient-centered care (interest in patient's ideas, inquiry about patient's social and emotional well-being, and involvement of patient in decision making). We used probit models to assess the relative priority patients placed on different attributes and to estimate how much they were willing to pay for them., RESULTS: Analyses were based on responses from 1,193 patients (a 53% response rate). Overall, patients were willing to pay the most for a thorough physical examination ($40.87). The next most valued attributes of care were seeing a physician who knew them well ($12.18), seeing a physician with a friendly manner ($8.50), having a reduction in waiting time of 1 day ($7.22), and having flexibility of appointment times ($6.71). Patients placed similar value on the different aspects of patient-centered care ($12.06-$14.82). Responses were influenced by the scenario in which the decision was made (minor physical problem vs urgent physical problem vs ambiguous physical or psychological problem) and by patients' demographic characteristics., CONCLUSIONS: Although patient-centered care is important to patients, they may place higher priority on the technical quality of care and continuity of care. Discrete choice experiments may be a useful method for assessing patients' priorities in health care.", "author" : [ { "dropping-particle" : "", "family" : "Cheraghi-Sohi", "given" : "S", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Hole", "given" : "A R", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Mead", "given" : "Nicola", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "McDonald", "given" : "Ruth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Whalley", "given" : "Diane", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Bower", "given" : "P", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Roland", "given" : "Martin", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Annals of family medicine", "id" : "ITEM-3", "issue" : "2", "issued" : { "date-parts" : [ [ "2008" ] ] }, "note" : "From Duplicate 1 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWhat patients want from primary care consultations: a discrete choice experiment to identify patients\u2019 priorities\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Cheraghi-Sohi, S; Hole, A R; Mead, N; McDonald, R; Whalley, D; Bower, P; Roland, M )\nAnd Duplicate 3 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWhat patients want from primary care consultations: a discrete choice experiment to identify patients' priorities.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Cheraghi-Sohi, S; Hole, Arne Risa; Mead, Nicola; McDonald, Ruth; Whalley, Diane; Bower, P; Roland, Martin )\n\n\n\n\n\n\n\n", "page" : "107-115", "publisher" : "National Primary Care Research and Development Centre (NPCRDC), University of Manchester, Manchester, United Kingdom.", "publisher-place" : "United States", "title" : "What patients want from primary care consultations: a discrete choice experiment to identify patients' priorities.", "type" : "article-journal", "volume" : "6" }, "uris" : [ "" ] }, { "id" : "ITEM-4", "itemData" : { "DOI" : "10.1371/journal.pone.0090635", "ISSN" : "1932-6203", "PMID" : "24759637", "abstract" : "OBJECTIVES: This study provides insights into the validity and acceptability of Discrete Choice Experiment (DCE) and profile-case Best Worst Scaling (BWS) methods for eliciting preferences for health care in a priority-setting context. METHODS: An adult sample (N = 24) undertook a traditional DCE and a BWS choice task as part of a wider survey on Health Technology Assessment decision criteria. A 'think aloud' protocol was applied, whereby participants verbalized their thinking while making choices. Internal validity and acceptability were assessed through a thematic analysis of the decision-making process emerging from the qualitative data and a repeated choice task. RESULTS: A thematic analysis of the decision-making process demonstrated clear evidence of 'trading' between multiple attribute/levels for the DCE, and to a lesser extent for the BWS task. Limited evidence consistent with a sequential decision-making model was observed for the BWS task. For the BWS task, some participants found choosing the worst attribute/level conceptually challenging. A desire to provide a complete ranking from best to worst was observed. The majority (18,75%) of participants indicated a preference for DCE, as they felt this enabled comparison of alternative full profiles. Those preferring BWS were averse to choosing an undesirable characteristic that was part of a 'package', or perceived BWS to be less ethically conflicting or burdensome. In a repeated choice task, more participants were consistent for the DCE (22,92%) than BWS (10,42%) (p = 0.002). CONCLUSIONS: This study supports the validity and acceptability of the traditional DCE format. Findings relating to the application of BWS profile methods are less definitive. Research avenues to further clarify the comparative merits of these preference elicitation methods are identified.", "author" : [ { "dropping-particle" : "", "family" : "Whitty", "given" : "Jennifer", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Walker", "given" : "Ruth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Golenko", "given" : "Xanthe", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ratcliffe", "given" : "Julie", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "PloS one", "id" : "ITEM-4", "issue" : "4", "issued" : { "date-parts" : [ [ "2014", "1" ] ] }, "page" : "e90635", "title" : "A think aloud study comparing the validity and acceptability of discrete choice and best worst scaling methods.", "type" : "article-journal", "volume" : "9" }, "uris" : [ "" ] }, { "id" : "ITEM-5", "itemData" : { "DOI" : "10.1177/0272989X16683934", "ISSN" : "0272-989X", "author" : [ { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rigby", "given" : "Dan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical Decision Making", "id" : "ITEM-5", "issue" : "3", "issued" : { "date-parts" : [ [ "2017" ] ] }, "page" : "298-313", "title" : "The Role of Qualitative Research Methods in Discrete Choice Experiments: A Systematic Review and Survey of Authors", "type" : "article-journal", "volume" : "37" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[20,24,26\u201328]", "plainTextFormattedCitation" : "[20,24,26\u201328]", "previouslyFormattedCitation" : "[20,24,26\u201328]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[20,24,26–28].The primary aim of this study was to use a type of cognitive interview, called ‘think-aloud’ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Ericson", "given" : "KA", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Simon", "given" : "HA", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "1993" ] ] }, "title" : "Protocol analysis: Verbal reports as data (revised edition)", "type" : "book" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[29]", "plainTextFormattedCitation" : "[29]", "previouslyFormattedCitation" : "[29]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[29], to reveal if, and how, respondents bring meaning to the information, particularly risk, presented to them in discrete choice-sets. A secondary aim was to understand how two competing approaches to communicating risk affected respondents’ accounts of their choice making strategies.Methods2.1 DCEThe relative benefits and risks of screening for breast cancer via mammography and the merit of national programmes have been extensively discussed ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1136/bmj.f385", "ISSN" : "1756-1833", "author" : [ { "dropping-particle" : "", "family" : "Baum", "given" : "M.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "BMJ British Medical Journal", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2013", "1", "23" ] ] }, "page" : "f385", "title" : "Harms from breast cancer screening outweigh benefits if death caused by treatment is included", "type" : "article-journal", "volume" : "346" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1136/bmj.f87", "ISSN" : "1756-1833", "author" : [ { "dropping-particle" : "", "family" : "Kirwan", "given" : "C. C.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "BMJ British Medical Journal", "id" : "ITEM-2", "issue" : "jan23 1", "issued" : { "date-parts" : [ [ "2013", "1", "23" ] ] }, "page" : "f87-f87", "title" : "Breast cancer screening: what does the future hold?", "type" : "article-journal", "volume" : "346" }, "uris" : [ "" ] }, { "id" : "ITEM-3", "itemData" : { "DOI" : "10.1001/jama.2014.1398", "ISBN" : "1538-3598 (Electronic)\\r0098-7484 (Linking)", "ISSN" : "1538-3598", "PMID" : "24691608", "abstract" : "IMPORTANCE: Breast cancer is the second leading cause of cancer deaths among US women. Mammography screening may be associated with reduced breast cancer mortality but can also cause harm. Guidelines recommend individualizing screening decisions, particularly for younger women. OBJECTIVES: We reviewed the evidence on the mortality benefit and chief harms of mammography screening and what is known about how to individualize mammography screening decisions, including communicating risks and benefits to patients. EVIDENCE ACQUISITION: We searched MEDLINE from 1960-2014 to describe (1) benefits of mammography, (2) harms of mammography, and (3) individualizing screening decisions and promoting informed decision making. We also manually searched reference lists of key articles retrieved, selected reviews, meta-analyses, and practice recommendations. We rated the level of evidence using the American Heart Association guidelines. RESULTS: Mammography screening is associated with a 19% overall reduction of breast cancer mortality (approximately 15% for women in their 40s and 32% for women in their 60s). For a 40- or 50-year-old woman undergoing 10 years of annual mammograms, the cumulative risk of a false-positive result is about 61%. About 19% of the cancers diagnosed during that 10-year period would not have become clinically apparent without screening (overdiagnosis), although there is uncertainty about this estimate. The net benefit of screening depends greatly on baseline breast cancer risk, which should be incorporated into screening decisions. Decision aids have the potential to help patients integrate information about risks and benefits with their own values and priorities, although they are not yet widely available for use in clinical practice. CONCLUSIONS AND RELEVANCE: To maximize the benefit of mammography screening, decisions should be individualized based on patients' risk profiles and preferences. Risk models and decision aids are useful tools, but more research is needed to optimize these and to further quantify overdiagnosis. Research should also explore other breast cancer screening strategies.", "author" : [ { "dropping-particle" : "", "family" : "Pace", "given" : "Lydia E", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Keating", "given" : "Nancy L", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "JAMA", "id" : "ITEM-3", "issue" : "13", "issued" : { "date-parts" : [ [ "2014" ] ] }, "page" : "1327-35", "title" : "A systematic assessment of benefits and risks to guide breast cancer screening decisions.", "type" : "article-journal", "volume" : "311" }, "uris" : [ "" ] }, { "id" : "ITEM-4", "itemData" : { "DOI" : "10.1136/bmj.h139", "ISSN" : "1756-1833", "PMID" : "25588758", "author" : [ { "dropping-particle" : "", "family" : "Torjesen", "given" : "Ingrid", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "BMJ (Clinical research ed.)", "id" : "ITEM-4", "issue" : "h139", "issued" : { "date-parts" : [ [ "2015" ] ] }, "title" : "How much is too much breast screening?", "type" : "article-journal", "volume" : "350" }, "uris" : [ "" ] }, { "id" : "ITEM-5", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "G\u00f8tzsche", "given" : "PC", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Nielsen", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "The Cochrane Library", "id" : "ITEM-5", "issue" : "4", "issued" : { "date-parts" : [ [ "2009" ] ] }, "title" : "Screening for breast cancer with mammography ( Review )", "type" : "article-journal" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[30\u201334]", "plainTextFormattedCitation" : "[30\u201334]", "previouslyFormattedCitation" : "[30\u201334]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[30–34]. This study used a DCE survey designed to elicit preferences of female members of the public for a national breast screening programme to understand how they balanced the benefits and risks of participation. Three attributes (probability of detecting a cancer, risk of unnecessary follow-up and out-of-pocket screening costs) and their levels were selected through interviews with clinical experts (n=4), female members of the public (n=4), and a patient representative (n=1). Respondents were required to choose between two hypothetical breast screening programmes or no screening in 10 choice-sets generated using a Bayesian D-efficient design, with priors from a quantitative pilot study. An additional choice set, testing for monotonicity in preferences, was also included. Survey respondents were randomised to one of two surveys, presenting risk as either percentages only, as most common in healthcare DCEs ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s40271-014-0048-1", "ISBN" : "4027101400481", "ISSN" : "1178-1653", "PMID" : "24566923", "abstract" : "BACKGROUND: Discrete choice experiments (DCEs) are used to elicit preferences of current and future patients and healthcare professionals about how they value different aspects of healthcare. Risk is an integral part of most healthcare decisions. Despite the use of risk attributes in DCEs consistently being highlighted as an area for further research, current methods of incorporating risk attributes in DCEs have not been reviewed explicitly.\\n\\nOBJECTIVES: This study aimed to systematically identify published healthcare DCEs that incorporated a risk attribute, summarise and appraise methods used to present and analyse risk attributes, and recommend best practice regarding including, analysing and transparently reporting the methodology supporting risk attributes in future DCEs.\\n\\nDATA SOURCES: The Web of Science, MEDLINE, EMBASE, PsycINFO and Econlit databases were searched on 18 April 2013 for DCEs that included a risk attribute published since 1995, and on 23 April 2013 to identify studies assessing risk communication in the general (non-DCE) health literature.\\n\\nSTUDY ELIGIBILITY CRITERIA: Healthcare-related DCEs with a risk attribute mentioned or suggested in the title/abstract were obtained and retained in the final review if a risk attribute meeting our definition was included.\\n\\nSTUDY APPRAISAL AND SYNTHESIS METHODS: Extracted data were tabulated and critically appraised to summarise the quality of reporting, and the format, presentation and interpretation of the risk attribute were summarised.\\n\\nRESULTS: This review identified 117 healthcare DCEs that incorporated at least one risk attribute. Whilst there was some evidence of good practice incorporated into the presentation of risk attributes, little evidence was found that developing methods and recommendations from other disciplines about effective methods and validation of risk communication were systematically applied to DCEs. In general, the reviewed DCE studies did not thoroughly report the methodology supporting the explanation of risk in training materials, the impact of framing risk, or exploring the validity of risk communication.\\n\\nLIMITATIONS: The primary limitation of this review was that the methods underlying presentation, format and analysis of risk attributes could only be appraised to the extent that they were reported.\\n\\nCONCLUSIONS: Improvements in reporting and transparency of risk presentation from conception to the analysis of DCEs are needed. To define best practice\u2026", "author" : [ { "dropping-particle" : "", "family" : "Harrison", "given" : "Mark", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rigby", "given" : "Dan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Flynn", "given" : "Terry", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Louviere", "given" : "JJ Jordan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "The patient", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2014", "1" ] ] }, "page" : "151-70", "title" : "Risk as an attribute in discrete choice experiments: A systematic review of the literature", "type" : "article-journal", "volume" : "7" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[6]", "plainTextFormattedCitation" : "[6]", "previouslyFormattedCitation" : "[6]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[6], or icon arrays and percentages. Appendix A presents an example of the choice set used in the DCE. The icon array images were created by the Risk Science Center, University of Michigan ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "URL" : "", "container-title" : "University of Michigan", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "0" ] ] }, "title" : "Center and Center for Bioethics and Social Sciences in Medicine", "type" : "webpage" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[35]", "plainTextFormattedCitation" : "[35]", "previouslyFormattedCitation" : "[35]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[35]. Details about the design of the DCE are reported elsewhere (see Vass et al.ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/j.jval.2017.07.010", "ISSN" : "15244733", "PMID" : "29477404", "abstract" : "Background: The relative benefits and risks of screening programs for breast cancer have been extensively debated. Objectives: To quantify and investigate heterogeneity in women's preferences for the benefits and risks of a national breast screening program (NBSP) and to understand the effect of risk communication format on these preferences. Methods: An online discrete choice experiment survey was designed to elicit preferences from female members of the public for an NBSP described by three attributes (probability of detecting a cancer, risk of unnecessary follow-up, and out-of-pocket screening costs). Survey respondents were randomized to one of two surveys, presenting risk either as percentages only or as icon arrays and percentages. Respondents were required to choose between two hypothetical NBSPs or no screening in 11 choice sets generated using a Bayesian D-efficient design. The trade-offs women made were analyzed using heteroskedastic conditional logit and scale-adjusted latent class models. Results: A total of 1018 women completed the discrete choice experiment (percentages-only version = 507; icon arrays and percentages version = 511). The results of the heteroskedastic conditional logit model suggested that, on average, women were willing-to-accept 1.72 (confidence interval 1.47\u20131.97) additional unnecessary follow-ups and willing-to-pay \u00a379.17 (confidence interval \u00a366.98\u2013\u00a391.35) for an additional cancer detected per 100 women screened. Latent class analysis indicated substantial heterogeneity in preferences with six latent classes and three scale classes providing the best fit. The risk communication format received was not a predictor of scale class or preference class membership. Conclusions: Most women were willing to trade-off the benefits and risks of screening, but decision makers seeking to improve uptake should consider the disparate needs of women when configuring services.", "author" : [ { "dropping-particle" : "", "family" : "Vass", "given" : "Caroline M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rigby", "given" : "Dan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Payne", "given" : "Katherine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2018" ] ] }, "page" : "219-228", "title" : "Investigating the Heterogeneity in Women's Preferences for Breast Screening: Does the Communication of Risk Matter?", "type" : "article-journal", "volume" : "21" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[36]", "plainTextFormattedCitation" : "[36]", "previouslyFormattedCitation" : "[36]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[36]). 2.2 SampleA sampling strategy was employed to attract a diverse sample ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/0149-7189(87)90056-5", "ISBN" : "0803924313", "ISSN" : "01497189", "PMID" : "338", "abstract" : "Naturalistic Inquiry provides social scientists with a basic but comprehensive rationale for non-positivistic approaches to research. It confronts the basic premise underlying the scientific tradition that all questions can be answered by employing empirical, testable, replicable research techniques. The authors maintain that there are scientific facts that existing paradigms cannot explain, and argue against traditional positivistic inquiry. They suggest an alternative approach supporting the use of the naturalistic paradigm.", "author" : [ { "dropping-particle" : "", "family" : "Lincoln", "given" : "Yvonna S", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Guba", "given" : "Egon G", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Evaluation and Program Planning", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "1985" ] ] }, "publisher" : "Sage Publications", "title" : "Naturalistic inquiry", "type" : "article-journal" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[37]", "plainTextFormattedCitation" : "[37]", "previouslyFormattedCitation" : "[37]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[37]. The sample selection criteria were limited to females, fluent in English and between 18 and 70 years of age (the cut-off for routine screening in England ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/S0140-6736(12)61611-0", "ISSN" : "1474-547X", "PMID" : "23117178", "abstract" : "Whether breast cancer screening does more harm than good has been debated extensively. The main questions are how large the benefit of screening is in terms of reduced breast cancer mortality and how substantial the harm is in terms of overdiagnosis, which is defined as cancers detected at screening that would not have otherwise become clinically apparent in the woman's lifetime. An independent Panel was convened to reach conclusions about the benefits and harms of breast screening on the basis of a review of published work and oral and written evidence presented by experts in the subject. To provide estimates of the level of benefits and harms, the Panel relied mainly on findings from randomised trials of breast cancer screening that compared women invited to screening with controls not invited, but also reviewed evidence from observational studies. The Panel focused on the UK setting, where women aged 50-70 years are invited to screening every 3 years. In this Review, we provide a summary of the full report on the Panel's findings and conclusions. In a meta-analysis of 11 randomised trials, the relative risk of breast cancer mortality for women invited to screening compared with controls was 0\u00b780 (95% CI 0\u00b773-0\u00b789), which is a relative risk reduction of 20%. The Panel considered the internal biases in the trials and whether these trials, which were done a long time ago, were still relevant; they concluded that 20% was still a reasonable estimate of the relative risk reduction. The more reliable and recent observational studies generally produced larger estimates of benefit, but these studies might be biased. The best estimates of overdiagnosis are from three trials in which women in the control group were not invited to be screened at the end of the active trial period. In a meta-analysis, estimates of the excess incidence were 11% (95% CI 9-12) when expressed as a proportion of cancers diagnosed in the invited group in the long term, and 19% (15-23) when expressed as a proportion of the cancers diagnosed during the active screening period. Results from observational studies support the occurrence of overdiagnosis, but estimates of its magnitude are unreliable. The Panel concludes that screening reduces breast cancer mortality but that some overdiagnosis occurs. Since the estimates provided are from studies with many limitations and whose relevance to present-day screening programmes can be questioned, they have substantial uncertainty and should be r\u2026", "author" : [ { "dropping-particle" : "", "family" : "Independent UK Panel on Breast Cancer Screening", "given" : "", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Lancet", "id" : "ITEM-1", "issue" : "9855", "issued" : { "date-parts" : [ [ "2012", "12", "17" ] ] }, "page" : "1778-86", "publisher" : "Elsevier Ltd", "title" : "The benefits and harms of breast cancer screening: an independent review.", "type" : "article-journal", "volume" : "380" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[38]", "plainTextFormattedCitation" : "[38]", "previouslyFormattedCitation" : "[38]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[38]). The advertising strategy involved placing advertisements online and through email distribution lists. Paper advertisements were placed in shops, cafes, bars and public noticeboards around Greater Manchester, a conurbation in the North West of England noticeable for a diverse population with health inequality ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1080/00344893.2016.1165513", "ISSN" : "0034-4893", "PMID" : "27499557", "abstract" : "This article maps the history of health organisation across Greater Manchester (GM), primarily since the Second World War, to show how against a continuing backdrop of health inequalities, services have been driven (and constrained) by the needs and the politics of each period. Defining 'success' as benefits for patients the article identifies examples such as Salford's mental health services (1950s and 1960s), public health in North Manchester (1970s and 1980s), the creation of centres for diabetes, sickle-cell and thalassaemia (1980s) and the formation of the Joint Health Unit in 2002. What this history shows is that over the period the common factors influencing the 'success' of health organisation across GM have been the championing of particular issues by multi-disciplinary groups working across health and social care and stability in structures and personnel.", "author" : [ { "dropping-particle" : "", "family" : "Snow", "given" : "Stephanie J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Representation (McDougall Trust)", "id" : "ITEM-1", "issue" : "4", "issued" : { "date-parts" : [ [ "2015", "10", "2" ] ] }, "page" : "439-452", "publisher" : "Taylor & Francis", "title" : "Health and Greater Manchester in Historical Perspective.", "type" : "article-journal", "volume" : "51" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[39]", "plainTextFormattedCitation" : "[39]", "previouslyFormattedCitation" : "[39]" }, "properties" : { "noteIndex" : 5 }, "schema" : "" }[39].2.3 Ethical approvalApproval for the study was obtained from The University of Manchester’s Research Ethics Committee (Ref: Ref 13178). 2.4 Data collectionA preliminary think-aloud interview schedule was tested in pilot interviews (n=5). CV conducted all interviews which were digitally recorded and then transcribed verbatim. A warm-up question (How many windows do you have in your house?) was used to prepare interviewees to vocalise their thoughts. After the warm-up question, interviewees completed the DCE, thinking aloud whilst selecting their answers on an iPad. In this part of the interview, participants were prompted to keep verbalising their thoughts through encouraging questions (e.g Why did you choose that? What are you thinking?) and did not follow a pre-defined structure. Open-ended de-briefing questions (e.g Would you change any of your answers? Do you have any other thoughts?) were also used to elicit anything else that might explain the interviewees’ choices. The interview transcriptions were supplemented with field notes where appropriate (for example, when an interviewee had pointed to something then notes were used to identify the object – usually a particular level). NVivo? qualitative software for research ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "QSR", "given" : "", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2012" ] ] }, "title" : "NVivo qualitative data analysis software: Version 10.0", "type" : "article" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[40]", "plainTextFormattedCitation" : "[40]", "previouslyFormattedCitation" : "[40]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[40] was used to import the transcripts and also store the audio files in preparation for the analysis. Figure 1 describes the process of data collection.2.5 AnalysisThe data generated from the think-aloud interviews were pooled with the data from the debriefing questions and all were analysed using the same methods. The first stage of the analysis involved listening to interviews and re-reading transcripts to ensure complete immersion in the qualitative data. From the transcripts and notes, the coding and generation of themes began under a loose initial framework ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Gale", "given" : "NK", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Heath", "given" : "G", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Cameron", "given" : "E", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "BMC medical research methodology", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2013" ] ] }, "page" : "117", "title" : "Using the framework method for the analysis of qualitative data in multi-disciplinary health research", "type" : "article-journal", "volume" : "1" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[41]", "plainTextFormattedCitation" : "[41]", "previouslyFormattedCitation" : "[41]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[41]. The initial codes were highlighted on the transcript text and involved a constrained form of open-coding under the framework; the approach was ‘open’ in that all new themes were allowed to develop but ‘constrained’ in that they were restricted to the two distinct, but not mutually exclusive, key concepts: 1) risk and 2) decision strategies. Risk referred to any data relating to the probability attributes, whether this be their thoughts about likelihood, perceptions of risk based on experience, or visualisation of the numbers. Decision strategies related to any accounts of behaviour or heuristics adapted when completing the choice task. One researcher (CV) coded the original transcripts and generated initial themes. The coding process was iterative, with new themes allowed to develop from the initial set. It was also unrestrictive as the same text could be coded multiple times and generate different codes which could possibly overlap. A coding tree was created to enable face-to-face discussions with the research team (comprising three senior academics who each had the transcripts) when the interviews were ongoing. In the initial stages, the tree was a web of loosely related codes with many branches of similar topics and tenuous links. After more interviews and further analysis, the codes were reorganised into a condensed set by moving or attaching related codes and creating sub-codes. The analytical cycle of suggesting, checking and re-checking to expand and then contract codes and themes continued until no new items or ideas were emerging. At this point, it was agreed data saturation had occurred and the interviews ceased. All transcripts were then recoded with the final set of themes. The choice data were analysed using a heteroscedastic conditional logit model ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Hole", "given" : "Arne Risa", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Economics Bulletin", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2006" ] ] }, "page" : "1-14", "title" : "Small-sample properties of tests for heteroscedasticity in the conditional logit model", "type" : "article-journal", "volume" : "3" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[42]", "plainTextFormattedCitation" : "[42]", "previouslyFormattedCitation" : "[42]" }, "properties" : { "noteIndex" : 6 }, "schema" : "" }[42] allowing the scale term to vary by risk communication format received, and with interaction terms to test the effect of format on preferences. ResultsThe final sample comprised nineteen women randomised to complete one of the two risk communication formats, with nine women receiving the icon arrays version and 10 women receiving the percentages only version. The average (mean) time for an interview was 37 minutes and 50 seconds. The sample included a range of ages and professions as described in Table 1. Quantitative results from the heteroscedastic conditional logit model are presented in Table 2. <insert Table 1><insert Table 2>An initial coding tree (shown in Appendix B) included all themes and sub-themes identified in the initial stages of the analysis. A simplified account of what occurred in the collected data is presented in Figure 2. The themes are shown in the circles with linking patterns in behaviour shown by arrows. The results section is split into the four themes of Figure 1 with interpretation of risk described in sections 3.1-3.3.<insert Figure 2>3.1 Task feedbackFor some respondents, the use of risk attributes appeared to induce behaviour indicative of calculating and a desire to ‘work out the answer’: I was trying to do the maths...[paused] Interviewer: Ok, so when you try and ‘do the math’ what do you mean? Well… I think if you actually worked out the maths, I might be slightly out. (Female 3: percentages only) I suppose because I work in engineering I do tend to consider things quite carefully and I'm quite logical in how I make choices... I'm sure that things like this can probably trip me up too. (Female 18: icon arrays)One woman reported flipping the negative risk into a positive one to make the comparison between attributes easier: Errr.. actually..well 80% wouldn't have an unnecessary follow-up... (Female 3: percentages only)It was apparent that for many respondents receiving the PO version, there was an initial alarm over the numbers presented, but some women felt that they could overcome this as the task progressed: I’m thinking, oh God, numbers, percentages! ... For me, I mean, I’ve never actually been that good at maths or anything, and perhaps I’ve got a bit of a number blindness I don’t know, but I feel a bit stressed when I see numbers .. I knew what my options were and I was quite clear in what I wanted to choose, but it was the numbers that I found difficult to work with... (Female 14: percentages only) It’s just personal to me I think, but it was quite a lot of figures and when you sort of look at one and then look at the other, and then it clicked with me and I was fine, but I think I needed perhaps a little bit more time to understand the percentage. (Female 13: percentages only)A few interviewees also found it hard to get around the hypothetical nature of the DCE: You see, that’s why it’s difficult, because you’re not actually presented with it now, as in real life kind of thing, it’s difficult. (Female 16: percentages only) You don't actually know until it happens.. you'll have lots of other things to think about... (Female 3: percentages only)However, when asked whether they would make the same choices in real life, most confirmed that they probably would: Yep, I'm not 100% but I think I'd be quite confident. I'm trying to imagine, if I got a letter, what would I do? I think this is what I would do. So I'm quite confident. (Female 4: icon arrays)Some women were quite clear that they did not consider the option of ‘no screening’: That none-option doesn't even come into it. (Female 3: percentages only) That wouldn’t be an option for me at all…Yes, sorry, I hadn’t even bothered with that one because you know, I would definitely go.. (Female 14: percentages only)For another woman, this opt-out offered a way-out of making a choice at a point of indifference: I think I’m probably in the not sure and I’d probably in this one go no screening for this one. (Female 6: icon arrays)3.2 Feelings and trustMany women expressed that their choices were completely dependent on the advice they received from ‘experts’: If the NHS said oh, we think you should go, then I would be like ok... I would trust them. Only because I don't know anything about it myself... (Female 4: icon arrays) Obviously the NHS has decided for a reason that 50-70 is a good age to go so I think I would go for it… (Female 18: icon arrays)3.3 Experience and exemplificationMany interviewees drew upon their previous experiences to determine their own perceived risk: I suppose, like, cancer is not in my family, either breast cancer or any kind of cancer is not in my family at all, so it’s something that I kind of don’t think about (Female 17: icon arrays) I mean my grandmother died quite young of breast cancer, but I still think my risk is probably average because I think that it's quite a common thing to happen. (Female 18: icon arrays)To aid their interpretation of risk, some women used analogies to compare the magnitude of the different levels: It’s royalties on a book. It’s 20% off something as in discounts, it’s a fairly high percentage compared to most discounts that you’re offered. (Female 13: percentages only)Women in both versions of the DCE often read out-loud the risk attributes as natural frequencies: 1 in 10 people will have unnecessary worry in A, whereas 1 in 20 in B (Female 11: percentages only) I think saying that something happens a fifth of the time that's quite - it's a more meaningful statement to me I would say. (Female 18: icon arrays)However, this was mostly done for the risk of unnecessary follow-up which contained easier levels, rather than detection which contained more complex probabilities (particularly 3% and 14%) which were more challenging to convert into a fraction.There was also evidence that some women were translating the percentages into frequencies to visualise the risk: There’s ten and then you’re just like one of them and the odds of it all, one in 20 or one in five that kind of thing. So I literally imagined just the number of people rather than how much it would work out in terms of hundreds and thousands of people. (Female 6: icon arrays) …because I think in pictures I think. I’m trying to visualise it. And I’m finding that difficult.. I don’t know, I mean for me I’m trying to look at it as though they were people. Little pictures of people there, that’s like if there was ten people and five people… (Female 14: percentages only)One woman who received icons explicitly commented on how it aided her visualisation of risk: It's easy to visualise it where you've got the pictograms here because you can quite clearly see it's one line out of five. (Female 18: icon arrays)3.4 Evidence of economic phenomenaIn their responses, many women expressed behaviours that have resonance with established economic phenomena. One emergent theme was that women seemed to be thinking in terms of ‘efficiency’ of, or the opportunity costs to, the NHS more generally expressing concerns about waste: See this is a waste of a lot of resources. I think I would go for none of these… This is too expensive and the other is too wasteful. (Female 2: percentages only)Although in neoclassical economics it is established that people have a preference for money now rather than in the future ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Frederick", "given" : "Shane", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Loewenstein", "given" : "George", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "O'donoghue", "given" : "T", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of economic literature", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2002" ] ] }, "note" : "From Duplicate 2 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTime discounting and time preference: A critical review\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Frederick, S; Loewenstein, G; O'donoghue, T )\n\n\n\n\n\n\n\n", "page" : "351-401", "title" : "Time discounting and time preference: A critical review", "type" : "article-journal", "volume" : "40" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[44]", "plainTextFormattedCitation" : "[44]", "previouslyFormattedCitation" : "[44]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[44], it is less clear how people treat non-financial benefits such as gains to healthADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Torgerson", "given" : "David J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Raftery", "given" : "James", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "BMJ British Medical Journal", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "1999" ] ] }, "page" : "914-915", "title" : "Discounting", "type" : "article-journal", "volume" : "319" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[45]", "plainTextFormattedCitation" : "[45]", "previouslyFormattedCitation" : "[45]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[45]. In our interviews, women expressed uncertainty surrounding both their future income and future health which could have led to discounting: Obviously you don’t know if you’re going to have ?1,000 just to spend over 20 years. (Female 11: percentages only) You never know how you’re going to be at any age really, do you? (Female 19: icon arrays)But also I think - I know I'm only sort of, like 19 years away from starting such a programme, but it seems like a long way away. Dying doesn't seem to me - that's not something that's going to happen to me anytime soon. (Female 18: icon arrays)Some women’s statements indicated that their choice was probably only reflecting their preferences for early screens, updating their priors about the expected utility from screening given their previous results: I think it would also depend on if I’ve had one or two before and they’ve turned out okay, I think I would be willing to pay less for further ones but I would be willing to pay probably the most for the first few. (Female 6: icon arrays) I would always go, but dependent on what it would say would influence my decision to go back. (Female 4: icon arrays)In some stated preference studies, completeness of preferences is investigated by testing if preferences are stable over sequential tasks ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Czajkowski", "given" : "Miko\u0142aj", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Barczak", "given" : "Anna", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Budzi\u0144ski", "given" : "Wiktor", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Giergiczny", "given" : "Marek", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Hanley", "given" : "N", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "University of St Andrews Discussion Papers in Environmental Economics", "id" : "ITEM-1", "issue" : "6", "issued" : { "date-parts" : [ [ "2014" ] ] }, "page" : "1-24", "title" : "Within- and between- sample tests of preference stability and willingness to pay for forest management", "type" : "article-journal" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[43]", "plainTextFormattedCitation" : "[43]", "previouslyFormattedCitation" : "[43]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[43]. However, some women reported that their preferences became more defined as they completed the experiment: Maybe at the start, I was more focussed on the price, but then actually having gone through a few and weighing it up it’s actually more important to look at the other factors rather than the money. (Female 9: icon arrays)In this DCE, some women expressed sentiments implying they were completing the DCE from a societal perspective: I think that on a humanitarian level, that you’ve got to open it to everybody, and try and save as many people as possible. (Female 13: percentages only)I don’t ever want to have to put 20% of people through any unnecessary follow ups… I suppose part of my influence is also the wider community. (Female 10: percentages only)There was also evidence some were using other’s budget constraints to make their choices:Even though I personally would still pay ?200… I think that for the population… that ?200 might be too much for people of that age maybe. So to encourage more people to go for it I would go for programme B in that case. (Female 8: icon arrays)Some women were reluctant to trade-off the cost attribute and this was particularly common in women of screening age: I don’t think you can really put a cost on your health... (Female 14: percentages only) Money against life it’s no comparison... And I do stress that’s not because I’m well off … generally speaking, to be told with great relief that you haven’t got cancer, you can’t buy that. (Female 13: percentages only)Whereas other women could interpret the cost attribute and made choices in line with the underlying economic theories: It sounds silly because it’s obviously your health and you can’t put a figure or price on it but people will do because that’s the day to day life that we lead. (Female 10: percentages only)However, a number of women also associated the cost of the screening programme with its quality:It's like why wouldn't I maybe pay a bit more to be seen, sort of, in a more beneficial and professional way perhaps? (Female 18: icon arrays)Even the unnecessary treatment I’d rather be on the safe side than think I’ve gone and I’ve got something and it’s not been detected because it’s cheaper. (Female 11: percentages only) DiscussionThe findings of this study suggested most of the women included in the sample were able to trade-off the attributes presented and understand the choice tasks. Although sometimes unwilling to trade-off the cost attribute, this was clearly a preference rather than an expression of cognitive burden or confusion. Some women expressed feelings of confusion when presented with percentages only, and overall the task appeared to be more daunting. Icon arrays appeared to make the prospect of the DCE task more attractive and engaged women in the survey. Similarly, a few respondents also felt the need to make a calculation and produce a ‘correct’ answer when the DCE was presented with percentages; despite the training materials and interviewer explicitly stating that there were no right answers. This should be emphasised to DCE respondents possibly throughout the task.Respondents who received icon arrays expressed support for the use of these graphics with one woman explicitly reporting that their presence improved her ability to trade-off between attributes. As visualisation of risk was common, the addition of an icon array appeared to relieve the cognitive burden of ‘imagining women’ in an already imaginary scenario. This is in line with previous studies which have found icon arrays aid people’s processing of screening information specifically ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Hess", "given" : "Rebecca", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Siegrist", "given" : "Michael", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Judgment and Decision Making", "id" : "ITEM-1", "issue" : "3", "issued" : { "date-parts" : [ [ "2011" ] ] }, "page" : "263-274", "title" : "Risk communication with pictographs: the role of numeracy and graph processing", "type" : "article-journal", "volume" : "6" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[46]", "plainTextFormattedCitation" : "[46]", "previouslyFormattedCitation" : "[46]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[46] and for health risks more generally ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1037/a0014474", "ISSN" : "0278-6133", "PMID" : "19290713", "abstract" : "OBJECTIVE: Icon arrays have been suggested as a potentially promising format for communicating risks to patients-especially those with low numeracy skills-but experimental studies are lacking. This study investigates whether icon arrays increase accuracy of understanding medical risks, and whether they affect perceived seriousness of risks and helpfulness of treatments. DESIGN: Two experiments were conducted on samples of older adults (n = 59, 62 to 77 years of age) and university students (n = 112, 26 to 35 years of age). MAIN OUTCOME MEASURES: Accuracy of understanding risk reduction; perceived seriousness of risks; perceived helpfulness of treatments. RESULTS: Icon arrays increased accuracy of both low- and high-numeracy people, even when transparent numerical representations were used. Risks presented via icon arrays were perceived as less serious than those presented numerically. With larger icon arrays (1,000 instead of 100 icons) risks were perceived more serious, and risk reduction larger. CONCLUSIONS: Icon arrays are a promising way of communicating medical risks to a wide range of patient groups, including older adults with lower numeracy skills.", "author" : [ { "dropping-particle" : "", "family" : "Galesic", "given" : "Mirta", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Garcia-Retamero", "given" : "R.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Gigerenzer", "given" : "Gerd", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Psychology", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2009", "3" ] ] }, "page" : "210-6", "title" : "Using icon arrays to communicate medical risks: overcoming low numeracy.", "type" : "article-journal", "volume" : "28" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1177/0272989X10369000", "ISBN" : "0272-989X", "ISSN" : "0272-989X", "PMID" : "20484088", "abstract" : "Background and Objective. Denominator neglect is the focus on the number of times a target event has happened (e.g., the number of treated and nontreated patients who die) without considering the overall number of opportuni-ties for it to happen (e.g., the overall number of treated and nontreated patients). In 2 studies, we addressed the effect of denominator neglect in problems involving treatment risk reduction where samples of treated and non-treated patients and the relative risk reduction were of different sizes. We also tested whether using icon arrays helps peo-ple take these different sample sizes into account. We espe-cially focused on older adults, who are often more disadvantaged when making decisions about their health. Design. Study 1 was conducted on a laboratory sample using a within-subjects design; study 2 was conducted on a nonstudent sample interviewed through the Web using a between-subjects design. Outcome Measures. Accuracy of understanding risk reduction. Results. Participants often paid too much attention to numerators and insufficient attention to denominators when numerical information about treatment risk reduction was provided. Adding icon arrays to the numerical information, however, drew partici-pants' attention to the denominators and helped them make more accurate assessments of treatment risk reduc-tion. Icon arrays were equally helpful to younger and older adults. Conclusions. Building on previous research show-ing that problems with understanding numerical informa-tion often do not reside in the mind but in the representation of the problem, the results show that icon arrays are an effective method of eliminating denominator neglect.", "author" : [ { "dropping-particle" : "", "family" : "Garcia-Retamero", "given" : "Rocio", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Galesic", "given" : "Mirta", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Gigerenzer", "given" : "Gerd", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical Decision Making", "id" : "ITEM-2", "issued" : { "date-parts" : [ [ "2010" ] ] }, "page" : "672-684", "title" : "Do Icon Arrays Help Reduce Denominator Neglect?", "type" : "article-journal", "volume" : "30" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[47,48]", "plainTextFormattedCitation" : "[47,48]", "previouslyFormattedCitation" : "[47,48]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[47,48]. For some individuals, who visualise risk, icon arrays may have a greater impact on their ability to understand probabilities. However, further research is required to understand if, and how, different icon types, sizes or arrangements affect trade-offs made in a benefit-risk DCE ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1177/0272989X13511706", "ISBN" : "1552-681X (Electronic)\\r0272-989X (Linking)", "ISSN" : "1552-681X", "PMID" : "24246564", "abstract" : "BACKGROUND Research has demonstrated that icon arrays (also called \"pictographs\") are an effective method of communicating risk statistics and appear particularly useful to less numerate and less graphically literate people. Yet research is very limited regarding whether icon type affects how people interpret and remember these graphs. METHODS 1502 people age 35-75 from a demographically diverse online panel completed a cardiovascular risk calculator based on Framingham data using their actual age, weight, and other health data. Participants received their risk estimate in an icon array graphic that used 1 of 6 types of icons: rectangular blocks, filled ovals, smile/frown faces, an outline of a person's head and shoulders, male/female \"restroom\" person icons (gender matched), or actual head-and-shoulder photographs of people of varied races (gender matched). In each icon array, blue icons represented cardiovascular events and gray icons represented those who would not experience an event. We measured perceived risk magnitude, approximate recall, and opinions about the icon arrays, as well as subjective numeracy and an abbreviated measure of graphical literacy. RESULTS Risk recall was significantly higher with more anthropomorphic icons (restroom icons, head outlines, and photos) than with other icon types, and participants rated restroom icons as most preferred. However, while restroom icons resulted in the highest correlations between perceived and actual risk among more numerate/graphically literate participants, they performed no better than other icon types among less numerate/graphically literate participants. CONCLUSIONS Icon type influences both risk perceptions and risk recall, with restroom icons in particular resulting in improved outcomes. However, optimal icon types may depend on numeracy and/or graphical literacy skills.", "author" : [ { "dropping-particle" : "", "family" : "Zikmund-Fisher", "given" : "Brian J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Witteman", "given" : "Holly O", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Dickson", "given" : "Mark", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Fuhrel-Forbis", "given" : "Andrea", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Kahn", "given" : "Valerie C", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Exe", "given" : "Nicole L", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Valerio", "given" : "Melissa", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Holtzman", "given" : "Lisa G", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Scherer", "given" : "Laura D", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Fagerlin", "given" : "Angela", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical decision making", "id" : "ITEM-1", "issue" : "4", "issued" : { "date-parts" : [ [ "2014" ] ] }, "page" : "443-53", "title" : "Blocks, ovals, or people? Icon type affects risk perceptions and recall of pictographs.", "type" : "article-journal", "volume" : "34" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[49]", "plainTextFormattedCitation" : "[49]", "previouslyFormattedCitation" : "[49]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[49].Some women also created analogies to aid their interpretation of the risk attribute. It is established that people find risk a difficult concept to understand partly because it is unfamiliar and point-probabilities are not often obvious in day-to-day life. Literature investigating how analogies can help communicate risk has mixed evidence; generally suggesting they work most effectively in highly-numerate populations ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Galesic", "given" : "Mirta", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Garcia-retamero", "given" : "Rocio", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Applied Cognitive Psychology", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "2013" ] ] }, "page" : "33-42", "title" : "Using analogies to communicate information about health risks", "type" : "article-journal", "volume" : "27" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[50]", "plainTextFormattedCitation" : "[50]", "previouslyFormattedCitation" : "[50]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[50]. Providing some analogies in the training materials might have proved useful to some respondents. There is also evidence in the literature that individuals’ risk experience and perception could be important factors in how risk information is presented and understood. Women in this study drew on such experience and perceptions when trading-off the attributes ‘probability of detecting a cancer’ and ‘risk of unnecessary follow-up’. A review ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/j.ypmed.2003.11.012", "ISSN" : "0091-7435", "PMID" : "15020172", "abstract" : "BACKGROUND: Perceived risk is a principal variable in theoretical models that attempt to predict the adoption of health-protective behaviors. METHODS: This meta-analysis synthesizes findings from 42 studies, identified in PubMed and PsycInfo from 1985 onward. Studies examined demographic and psychological variables as predictors of perceived breast cancer risk and the relationship between perceived risk and breast cancer screening. Statistical relationships, weighted for sample size, were transformed to effect sizes and 95% CIs. RESULTS: Women do not have accurate perceptions of their breast cancer risk (N = 5561, g = 1.10). Overall, they have an optimistic bias about their personal risk (g = 0.99). However, having a positive family history (N = 70660, g = 0.88), recruitment site, and measurement error confounded these results. Perceived risk is weakly influenced by age (N = 38000, g = 0.13) and education (N = 1979, g = 0.16), and is moderately affected by race/culture (N = 2192, g = 0.38) and worry (N = 6090, g = 0.49). There is an association between perceived risk and mammography screening (N = 52766, g = 0.19). It is not clear whether perceived risk influences adherence to breast self-examination. Women who perceived a higher breast cancer risk were more likely to pursue genetic testing or undergo prophylactic mastectomy. CONCLUSION: Perceived breast cancer risk depends on psychological and cognitive variables and influences adherence to mammography screening guidelines.", "author" : [ { "dropping-particle" : "", "family" : "Katapodi", "given" : "Maria C", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lee", "given" : "Kathy", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Facione", "given" : "Noreen C", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Dodd", "given" : "Marylin J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Preventive medicine", "id" : "ITEM-1", "issue" : "4", "issued" : { "date-parts" : [ [ "2004", "4" ] ] }, "page" : "388-402", "title" : "Predictors of perceived breast cancer risk and the relation between perceived risk and breast cancer screening: a meta-analytic review.", "type" : "article-journal", "volume" : "38" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[51]", "plainTextFormattedCitation" : "[51]", "previouslyFormattedCitation" : "[51]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[51] of women’s perceived risk of breast cancer found that women were both inaccurate and conservative in their estimations. Women’s perception of their risk may also be related to their understanding of health generally which can be assessed using health literacy measures ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1177/0272989X15597225", "ISBN" : "1552-681X (Electronic)\r0272-989X (Linking)", "ISSN" : "1552-681X", "PMID" : "26338176", "abstract" : "PURPOSE: The purpose of this study is to examine to what extent health literacy is associated with parental preferences concerning childhood vaccination.\\n\\nMETHODS: A cross-sectional study was conducted among 467 Dutch parents of newborns aged 6 weeks (response rate of 37%). A self-reported questionnaire was used to measure health literacy by means of Chew's Set of Brief Screening Questions, as well as parental preferences for rotavirus vaccination by means of a discrete choice experiment. Five rotavirus-related characteristics were included (i.e., vaccine effectiveness, frequency of severe side effects, location of vaccination, protection duration, and out-of-pocket costs). Panel latent class models were conducted, and health literacy and educational level were added to the class probability model to determine the association between health literacy and study outcomes.\\n\\nRESULTS: Lower educated and lower health literate respondents considered protection duration to be more important and vaccine effectiveness and frequency of severe side effects to be less important compared with higher educated and higher health literate respondents. While all respondents were willing to vaccinate against rotavirus when the vaccine was offered as part of the National Immunization Program, only lower educated and lower health literate parents were willing to vaccinate when the vaccine was offered on the free market.\\n\\nCONCLUSION: Health literacy is associated with parents' preferences for rotavirus vaccination. Whether differences in vaccination decisions are actually due to varying preferences or might be better explained by varying levels of understanding should be further investigated. To contribute to more accurate interpretation of study results, it may be advisable that researchers measure and report health literacy when they study vaccination decision behavior.", "author" : [ { "dropping-particle" : "", "family" : "Veldwijk", "given" : "Jorien", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Heide", "given" : "Iris", "non-dropping-particle" : "van der", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rademakers", "given" : "Jany", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Schuit", "given" : "a Jantine", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Wit", "given" : "G Ardine", "non-dropping-particle" : "de", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Uiters", "given" : "Ellen", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lambooij", "given" : "Mattijs S", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical decision making : an international journal of the Society for Medical Decision Making", "id" : "ITEM-1", "issue" : "8", "issued" : { "date-parts" : [ [ "2015" ] ] }, "page" : "948-58", "title" : "Preferences for Vaccination: Does Health Literacy Make a Difference?", "type" : "article-journal", "volume" : "35" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[52]", "plainTextFormattedCitation" : "[52]", "previouslyFormattedCitation" : "[52]" }, "properties" : { "noteIndex" : 13 }, "schema" : "" }[52]. Future research could consider the effect of an individual’s risk status and their health literacy on their interpretation of risk information and their preferences. A number of key axioms underline the analysis of choice data ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "ISBN" : "0393957357", "author" : [ { "dropping-particle" : "", "family" : "Varian", "given" : "H A L R", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Taxes", "edition" : "3rd", "id" : "ITEM-1", "issued" : { "date-parts" : [ [ "1992" ] ] }, "title" : "Microeconomic Analysis", "type" : "book" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[53]", "plainTextFormattedCitation" : "[53]", "previouslyFormattedCitation" : "[53]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[53], and violation of an axiom may result in biased results. For example, if a respondent exhibits non-compensatory preferences then no amount of one attribute can compensate them for the absence of another, violating the continuity axiom. An example of this occurred when respondents were completely unwilling to make trade-offs using the cost attribute; exhibiting attribute non-attendance. This violation of the continuity axiom could potentially result in upwardly biased willingness-to-pay (WTP) valuations. DCE respondents’ reluctance to trade-off attributes in a health setting have been identified in other studies ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1002/hec", "author" : [ { "dropping-particle" : "", "family" : "Lagarde", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-1", "issue" : "5", "issued" : { "date-parts" : [ [ "2012" ] ] }, "page" : "554-567", "title" : "Investigating attribute non-attendance and its consequences in choice experiments with latent class models", "type" : "article-journal", "volume" : "22" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[21]", "plainTextFormattedCitation" : "[21]", "previouslyFormattedCitation" : "[21]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[21]. Similar behaviour has also been reported in environmental DCEs ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1007/s10640-010-9365-6", "ISSN" : "0924-6460", "author" : [ { "dropping-particle" : "", "family" : "Carlsson", "given" : "Fredrik", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Kataria", "given" : "Mitesh", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lampi", "given" : "Elina", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Environmental and Resource Economics", "id" : "ITEM-1", "issue" : "1", "issued" : { "date-parts" : [ [ "2010", "4", "9" ] ] }, "page" : "65-89", "title" : "Dealing with ignored attributes in choice experiments on valuation of Sweden\u2019s environmental quality objectives", "type" : "article-journal", "volume" : "47" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[54]", "plainTextFormattedCitation" : "[54]", "previouslyFormattedCitation" : "[54]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[54] and studies in health DCEs have found insignificant coefficients on cost ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1111/j.1524-4733.2008.00451.x", "ISBN" : "1524-4733", "ISSN" : "1524-4733", "PMID" : "18783389", "abstract" : "OBJECTIVE: To examine women's preferences for characteristics of chlamydia screening. Chlamydia trachomatis is the most common curable sexually transmitted disease. To design effective screening programs, it is important to fully capture the benefits of screening to patients. Thus, the value of experience factors must be considered alongside health outcomes., METHODS: A self-complete discrete choice experiment questionnaire was administered to women attending a family planning clinic. Chlamydia screening was described by five characteristics: location of screening; type of screening test; cost of screening test; risk of developing pelvic inflammatory disease if chlamydia is untreated; and support provided when receiving results., RESULTS: One hundred twenty-six women completed the questionnaire. Respondents valued characteristics of the care experience. Screening was valued at 15 pound; less invasive screening tests increase willingness to pay by 7 pound, and more invasive tests reduce willingness to pay by 3.50 pound. The most preferred screening location was the family planning clinic, valued at 5 pound. The support of a trained health-care professional when receiving results was valued at 4 pound. Respondents under 25 years and those in a casual relationship were less likely to be screened., CONCLUSIONS: Women valued experience factors in the provision of chlamydia screening. To correctly value these screening programs and to predict uptake, cost-effectiveness studies should take such values into account. Failure to do this may result in incorrect policy recommendations.", "author" : [ { "dropping-particle" : "", "family" : "Watson", "given" : "V", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ryan", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Watson", "given" : "Emma", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-1", "issue" : "4", "issued" : { "date-parts" : [ [ "2009", "6" ] ] }, "note" : "From Duplicate 1 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of chlamydia screening: an application to women attending the family planning clinic\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, V; Ryan, M; Watson, E )\nAnd Duplicate 2 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, Verity; Ryan, M; Watson, Emma )\nAnd Duplicate 3 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, V; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nFrom Duplicate 4 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, V; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nFrom Duplicate 1 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, Verity; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nFrom Duplicate 2 ( \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nValuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n- Watson, Verity; Ryan, M; Watson, Emma )\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\u00a0\u00a0\u00a0\u00a0\u00a0 (50) \u00a0\u00a0 Watson V, Ryan M, Watson E. Valuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic. Value in Health 12(4):621-3, 2009 Jun. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nQualitative Methods\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThis study provides no detail about qualitative methods employed.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nAttributes were chosen &quot;Based on a literature review, policy variations, and advice from the family planning clinic\u2019s doctor, five screening attributes were identified.&quot; (p. 621)\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe DCEs were administered as a survey on arrival to clinic. Qualitative methods could have contributed to the authors' understanding of non-responses, whether the attributes were relevant and whether the respondents understood the task/attributes/levels. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThey even conclude that &quot;Future work should explore if other attributes are important to this group&quot; (p. 623) and &quot;the insignificance of the risk of pelvic inflammatory disease may reflect the difficulties that respondents had in understanding this attribute&quot; (p.623)", "page" : "621-623", "publisher" : "Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, UK. v.watson@abdn.ac.uk", "publisher-place" : "United States", "title" : "Valuing experience factors in the provision of Chlamydia screening: an application to women attending the family planning clinic.", "type" : "article-journal", "volume" : "12" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[19]", "plainTextFormattedCitation" : "[19]", "previouslyFormattedCitation" : "[19]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[19].A few women who did consider the cost of screening interpreted it as an indicator of the programme’s quality. This association could also lead to biased valuation estimates as respondents associate higher cost with more utility, which could explain positive coefficients on cost attributes in previous research ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/j.jval.2012.11.007", "ISSN" : "1524-4733", "PMID" : "23538191", "abstract" : "OBJECTIVES: To assess patients' preferences for rheumatoid-arthritis treatments with biologic agents using a discrete-choice experiment. METHODS: A discrete-choice experiment was conducted with adult rheumatoid-arthritis patients who had never been treated with biological agents from two university hospitals-public and private-in Buenos Aires, Argentina. We evaluated preferences for seven treatment attributes (with two to three levels each): effectiveness, mode of administration, frequency of administration, local and systemic adverse events, severe infections, and out-of-pocket costs.A probit regression model was used to analyze the relative importance of rheumatoid-arthritis treatment attributes. We estimated attributes' relative importance and their 95% confidence intervals. RESULTS: Survey responses from 240 patients with rheumatoid arthritis receiving conventional disease-modifying antirheumatic drugs were included in the study. All tested biological agents' attributes significantly affected the choice of treatment. Attributes' relative importance in decreasing order was the following (mean, confidence interval 95%): cost, 0.81 (0.69-0.92); systemic adverse events, 0.66 (0.57-0.76); frequency of administration, 0.61 (0.52-0.71); efficacy, 0.42 (0.32-0.51); route of administration, 0.41 (0.30-0.52); local adverse events, 0.40 (0.31-0.49); and serious infections, 0.29 (0.22-0.37). CONCLUSIONS: Different treatment attributes had a significant and different influence in rheumatoid-arthritis patients' choice of biological agents. This type of study can not only inform about patients' preferences but also about the trade-offs among different possible treatments or process-related attributes.", "author" : [ { "dropping-particle" : "", "family" : "Augustovski", "given" : "F.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Beratarrechea", "given" : "Andrea", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Irazola", "given" : "Vilma", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Rubinstein", "given" : "Fernando", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Tesolin", "given" : "Pablo", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Gonzalez", "given" : "Juan", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Lencina", "given" : "Ver\u00f3nica", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Scolnik", "given" : "Marina", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Waimann", "given" : "Christian", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Navarta", "given" : "David", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Citera", "given" : "Gustavo", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Soriano", "given" : "Enrique R", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Value in Health", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2013" ] ] }, "page" : "385-93", "title" : "Patient preferences for biologic agents in rheumatoid arthritis: a discrete-choice experiment.", "type" : "article-journal", "volume" : "16" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[55]", "plainTextFormattedCitation" : "[55]", "previouslyFormattedCitation" : "[55]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[55]. Further unexpected interpretation of the cost attribute occurred when women made choices using other people’s budget constraints. If DCE respondents employ a budget stricter than their true constraint when making their choices, this could result in a conservative estimate of actual WTP. There were also examples of women discounting the attributes, such as cost, which were related to the life-time screening programme being valued. Further research is required to establish if in different frames with shorter time horizons discounting still occurs. A few women (in both risk communication formats) reported that it took them a while to familiarise themselves with the task. The unfamiliar nature of the choice task was also reflected in women’s comments on the hypothetical situation and the way their preferences updated through the survey. This aligns with other research which has found signs of ‘learning effects’ in choice experiments ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1016/S1755-5345(13)70051-4", "ISBN" : "1755-5345", "ISSN" : "17555345", "abstract" : "This paper addresses the issue of ordering effects in choice experiments, and in particular how learning processes potentially affect respondents' stated preferences in a sequence of choice sets. In a case study concerning food quality attributes of chicken breast filets, we find evidence of ordering effects in a sequence of 16 choice sets, where the last eight choice sets are identical to the first eight. We find evidence of changes in preferences. More precisely there are differences in preferences for the price attribute for the two identical sequences. Moreover, we find a reduction in the error variance for the last eight choice sets relative to the first eight choice sets. This is mainly caused by very high error variances in the first two choice sets. These results imply that learning effects in terms of institutional learning as well as - though in our case only to a limited extent - preference learning can indeed be of significant structural importance when conducting CE surveys.", "author" : [ { "dropping-particle" : "", "family" : "Carlsson", "given" : "Fredrik", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "M\u00f8rkbak", "given" : "Morten Raun", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Olsen", "given" : "S\u00f8ren B\u00f8ye", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Journal of Choice Modelling", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2012" ] ] }, "page" : "19-37", "title" : "The first time is the hardest: A test of ordering effects in choice experiments", "type" : "article-journal", "volume" : "5" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[56]", "plainTextFormattedCitation" : "[56]", "previouslyFormattedCitation" : "[56]" }, "properties" : { "noteIndex" : 12 }, "schema" : "" }[56], and also serves as evidence of the importance of effective training materials explaining exactly why their choices are of interest and what the task will involve. There are relatively few studies which have used qualitative research methods to gain a deeper-understanding of respondents’ choices in a DCE setting ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "PMID" : "17478438", "abstract" : "BACKGROUND: Delivering effective health care within limited budgets requires an understanding of patient priorities. Discrete choice experiments (DCEs) provide patients with choices, where each choice differs in terms of certain attributes (such as waiting times, quality of care). Although this technique has significant potential in examining priorities, its use raises practical and conceptual issues. This paper describes the development of a DCE evaluating patient priorities in primary care. METHODS: Twenty patients completed a DCE using a 'think aloud' protocol, where they verbalized their thinking while making choices. The analysis examined their decision-making processes. RESULTS: There was evidence that patients reinterpreted some attributes, and related some to others outside the task. The cost attribute was interpreted in a variety of ways, dominating some patients' decision-making, being seen as irrelevant by others and being interpreted appropriately by some. The deree to which patients exhibited trading in line with theoretical assumptions also varied. Some choices in the hypothetical task were restricted by their previous experience, but more frequently patients tested the boundaries of the task in ways which directly reflected the primary care context. CONCLUSION: Patient interpretation of the discrete choice task was varied and some went beyond the formal boundaries of the task to make their choices. This highlights the importance of piloting attributes, providing clear instructions about the task and developing models of patient decision-making so that responses can be interpreted correctly.", "author" : [ { "dropping-particle" : "", "family" : "Cheraghi-Sohi", "given" : "S", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Bower", "given" : "P", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Mead", "given" : "Nicola", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "McDonald", "given" : "Ruth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Whalley", "given" : "Diane", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Roland", "given" : "Martin", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Family Practice", "id" : "ITEM-1", "issue" : "3", "issued" : { "date-parts" : [ [ "2007" ] ] }, "page" : "276-282", "title" : "Making sense of patient priorities: applying discrete choice methods in primary care using 'think aloud' technique.", "type" : "article-journal", "volume" : "24" }, "uris" : [ "" ] }, { "id" : "ITEM-2", "itemData" : { "DOI" : "10.1002/hec", "abstract" : "Stated preference methods assume respondents' preferences are consistent with utility theory, but many empirical studies report evidence of preferences that violate utility theory. This evidence is often derived from quantitative tests that occur naturally within, or are added to, stated preference tasks. In this study, we use qualitative methods to explore three axioms of utility theory: completeness, monotonicity, and continuity. We take a novel approach, adopting a 'think aloud' technique to identify violations of the axioms of utility theory and to consider how well the quantitative tests incorporated within a discrete choice experiment are able to detect these. Results indicate that quantitative tests classify respondents as being 'irrational' when qualitative statements would indicate they are 'rational'. In particular, 'non-monotonic' responses can often be explained by respondents inferring additional information beyond what is presented in the task, and individuals who appear to adopt non-compensatory decision-making strategies do so because they rate particular attributes very highly (they are not attempting to simplify the task). The results also provide evidence of 'cost-based responses': respondents assumed tests with higher costs would be of higher quality. The value of including in-depth qualitative validation techniques in the development of stated preference tasks is shown. Copyright 2008 John Wiley & Sons, Ltd", "author" : [ { "dropping-particle" : "", "family" : "Ryan", "given" : "M", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Watson", "given" : "V", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Entwistle", "given" : "Vikki", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Health Economics", "id" : "ITEM-2", "issued" : { "date-parts" : [ [ "2009" ] ] }, "page" : "321-336", "title" : "Rationalising the 'irrational': a think aloud study of a discrete choice experiment responses", "type" : "article-journal", "volume" : "18" }, "uris" : [ "" ] }, { "id" : "ITEM-3", "itemData" : { "DOI" : "10.1371/journal.pone.0090635", "ISSN" : "1932-6203", "PMID" : "24759637", "abstract" : "OBJECTIVES: This study provides insights into the validity and acceptability of Discrete Choice Experiment (DCE) and profile-case Best Worst Scaling (BWS) methods for eliciting preferences for health care in a priority-setting context. METHODS: An adult sample (N = 24) undertook a traditional DCE and a BWS choice task as part of a wider survey on Health Technology Assessment decision criteria. A 'think aloud' protocol was applied, whereby participants verbalized their thinking while making choices. Internal validity and acceptability were assessed through a thematic analysis of the decision-making process emerging from the qualitative data and a repeated choice task. RESULTS: A thematic analysis of the decision-making process demonstrated clear evidence of 'trading' between multiple attribute/levels for the DCE, and to a lesser extent for the BWS task. Limited evidence consistent with a sequential decision-making model was observed for the BWS task. For the BWS task, some participants found choosing the worst attribute/level conceptually challenging. A desire to provide a complete ranking from best to worst was observed. The majority (18,75%) of participants indicated a preference for DCE, as they felt this enabled comparison of alternative full profiles. Those preferring BWS were averse to choosing an undesirable characteristic that was part of a 'package', or perceived BWS to be less ethically conflicting or burdensome. In a repeated choice task, more participants were consistent for the DCE (22,92%) than BWS (10,42%) (p = 0.002). CONCLUSIONS: This study supports the validity and acceptability of the traditional DCE format. Findings relating to the application of BWS profile methods are less definitive. Research avenues to further clarify the comparative merits of these preference elicitation methods are identified.", "author" : [ { "dropping-particle" : "", "family" : "Whitty", "given" : "Jennifer", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Walker", "given" : "Ruth", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Golenko", "given" : "Xanthe", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ratcliffe", "given" : "Julie", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "PloS one", "id" : "ITEM-3", "issue" : "4", "issued" : { "date-parts" : [ [ "2014", "1" ] ] }, "page" : "e90635", "title" : "A think aloud study comparing the validity and acceptability of discrete choice and best worst scaling methods.", "type" : "article-journal", "volume" : "9" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[20,26,28]", "plainTextFormattedCitation" : "[20,26,28]", "previouslyFormattedCitation" : "[20,26,28]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[20,26,28], and this is the first empirical study which has focussed on how people balance risks and benefits in a DCE setting. The think-aloud method was used to limit response acquiescence or ‘yea-saying bias’ by prompting interviewees to come-up with their own accounts instead of agreeing with the interviewer ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Blamey", "given" : "RK", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Bennett", "given" : "JW", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Morrison", "given" : "MD", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Land Economics", "id" : "ITEM-1", "issue" : "1", "issued" : { "date-parts" : [ [ "1999" ] ] }, "page" : "126-141", "title" : "Yea-saying in contingent valuation surveys", "type" : "article-journal", "volume" : "75" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[57]", "plainTextFormattedCitation" : "[57]", "previouslyFormattedCitation" : "[57]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[57]. However the method has been criticised because of its reliance on what people say, rather than what they really think ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "author" : [ { "dropping-particle" : "", "family" : "Boren", "given" : "T", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ramey", "given" : "J", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "IEEE Transactions on Professional Communication", "id" : "ITEM-1", "issue" : "3", "issued" : { "date-parts" : [ [ "2000" ] ] }, "page" : "261-278", "title" : "Thinking aloud: Reconciling theory and practice", "type" : "article-journal", "volume" : "43" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[58]", "plainTextFormattedCitation" : "[58]", "previouslyFormattedCitation" : "[58]" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }[58]. A concern is whether the act of thinking-aloud disturbs the natural decision-making behaviour and therefore generates data that are not the ‘usual’; a phenomenon known as ‘nonverdicality’ ADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.3109/0142159X.2013.801938", "ISBN" : "0142-159x", "ISSN" : "0142159X", "PMID" : "23805999", "abstract" : "BACKGROUND: Whether the think-aloud protocol is a valid measure of thinking remains uncertain. Therefore, we used functional magnetic resonance imaging (fMRI) to investigate potential functional neuroanatomic differences between thinking (answering multiple-choice questions in real time) versus thinking aloud (on review of items). METHODS: Board-certified internal medicine physicians underwent formal think-aloud training. Next, they answered validated multiple-choice questions in an fMRI scanner while both answering (thinking) and thinking aloud about the questions, and we compared fMRI images obtained during both periods. RESULTS: Seventeen physicians (15 men and 2 women) participated in the study. Mean physician age was 39.5 + 7 (range: 32-51 years). The mean number of correct responses was 18.5/32 questions (range: 15-25). Statistically significant differences were found between answering (thinking) and thinking aloud in the following regions: motor cortex, bilateral prefrontal cortex, bilateral cerebellum, and the basal ganglia (p < 0.01). DISCUSSION: We identified significant differences between answering and thinking aloud within the motor cortex, prefrontal cortex, cerebellum, and basal ganglia. These differences were by degree (more focal activation in these areas with thinking aloud as opposed to answering). Prefrontal cortex and cerebellum activity was attributable to working memory. Basal ganglia activity was attributed to the reward of answering a question. The identified neuroimaging differences between answering and thinking aloud were expected based on existing theory and research in other fields. These findings add evidence to the notion that the think-aloud protocol is a reasonable measure of thinking.", "author" : [ { "dropping-particle" : "", "family" : "Durning", "given" : "Steven J.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Artino", "given" : "Anthony R.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Beckman", "given" : "Thomas J.", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Graner", "given" : "John", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Vleuten", "given" : "Cees", "non-dropping-particle" : "Van Der", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Holmboe", "given" : "Eric", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Schuwirth", "given" : "Lambert", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Medical Teacher", "id" : "ITEM-1", "issue" : "9", "issued" : { "date-parts" : [ [ "2013" ] ] }, "page" : "720-726", "title" : "Does the think-aloud protocol reflect thinking? Exploring functional neuroimaging differences with thinking (answering multiple choice questions) versus thinking aloud", "type" : "article-journal", "volume" : "35" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "[59]", "plainTextFormattedCitation" : "[59]", "previouslyFormattedCitation" : "[59]" }, "properties" : { "noteIndex" : 12 }, "schema" : "" }[59]. However, the interview schedule was designed to limit this and other sources of bias, such as reflexivity (effects of researcher presence), as much as possible. ConclusionThis study has shed light on how DCE respondents make choices and trade-off risk attributes. The risk communication format did not substantially alter women’s accounts of their choice making but icon arrays did appear to relieve some of the cognitive burden in the initial choice sets by aiding the visualisation of risk. Although most women appeared to complete the DCE in line with underpinning economic theories, some violations were detected. Researchers seeking to use DCEs to elicit preferences for healthcare goods or services should be aware of the potential for non-trading behaviour, information asymmetry and utilisation of other’s budget constraints. However, these factors could be mediated with appropriate communication of the research question and a clear explanation of the survey’s purpose. Data Availability StatementData generated in this study cannot be made available due to ethical restrictions. ADDIN Mendeley Bibliography CSL_BIBLIOGRAPHY 1. Ryan M, Bate a, Eastmond CJ, Ludbrook a. Use of discrete choice experiments to elicit preferences. Qual. Heal. Care. 2001;10(Suppl I):i55–60. 2. Lancaster KJ. A new approach to consumer theory. J. Polit. Econ. 1966;74(2):132–57. 3. McFadden D. Conditional logit analysis of qualitative choice behaviour. In: Zarembka P, editor. Front. Econom. New York: Academic Press INC; 1974. p. 105–42. 4. Hensher D. An exploratory analysis of the effect of numbers of choice sets in designed choice experiments: an airline choice application. J. Air Transp. Manag. [Internet]. 2001;7(6):373–9. 5. Smyth RL, Watzin MC, Manning RE. Investigating public preferences for managing lake champlain using a choice experiment. J. Environ. Manage. [Internet]. England: Rubenstein School of Environment and Natural Resources, Aiken Center, University of Vermont, Burlington, VT 05045, USA.; 2009;90(1):615–23. 6. Harrison M, Rigby D, Vass CM, Flynn T, Louviere JJ, Payne K. Risk as an attribute in discrete choice experiments: a systematic review of the literature. Patient [Internet]. 2014 [cited 2014 Dec 2];7(2):151–70. 7. Vass CM, Payne K. Using discrete choice experiments to inform the benefit ? risk assessment of medicines?: are we ready yet?? Pharmacoeconomics. 2017;35(9):1–21. 8. Hauber A, Fairchild AO, Johnson F. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl. Health Econ. Health Policy [Internet]. 2013 [cited 2015 May 25];11(4):319–29. 9. Ho MP, Gonzalez JM, Lerner HP, Neuland CY, Whang JM, McMurry-Heath M, et al. Incorporating patient-preference evidence into regulatory decision making. Surg. Endosc. Other Interv. Tech. 2015;29(10):2984–93. 10. Reed SD, Lavezzari G. International experiences in quantitative benefit-risk analysis to support regulatory decisions. Value Heal. 2016. p. 727–9. 11. Muhlbacher AC, Juhnke C, Beyer AR, Garner S. Patient-focused benefit-risk analysis to inform regulatory decisions: the european union perspective. Value Heal. 2016;19(6):734–40. 12. Johnson FR, Zhou M. Patient preferences in regulatory benefit-risk assessments: a us perspective. Value Heal. 2016;19(6):741–5. 13. FDA. Patient preference information voluntary submission, review in premarket approval applications, humanitarian device exemption applicationsm and de novo requests, and inclusion in decision summaries and device labeling. U.S. Dep. Heal. Hum. Serv. Food Drug Adm. Cent. Devices Radiol. Heal. 2016;FDA-2015-D. 14. Coast J, Al-Janabi H, Sutton E, Horrocks SA, Vosper J, Swancutt DR, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–41. 15. Hammitt JK, Graham JD. Willingness to pay for health protection: inadequate sensitivity to probability? J. Risk Uncertain. 1999;18(1):33–62. 16. Lipkus I. Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med. Decis. Mak. [Internet]. 2007 [cited 2011 Aug 2];27(5):696–713. 17. Lipkus I, Samsa G, Rimer B. General performance on a numeracy scale among highly educated samples. Med. Decis. Mak. [Internet]. 2001;21(1):37–44. 18. Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz L, Woloshin S. Helping doctors and patients make sense of health statistics. Psychol. Sci. public Interes. [Internet]. 2007 [cited 2015 Mar 3];8(2):53–96. 19. Watson V, Ryan M, Watson E. Valuing experience factors in the provision of chlamydia screening: an application to women attending the family planning clinic. Value Heal. [Internet]. United States: Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, UK. v.watson@abdn.ac.uk; 2009;12(4):621–3. 20. Ryan M, Watson V, Entwistle V. Rationalising the “irrational”: a think aloud study of a discrete choice experiment responses. Health Econ. [Internet]. 2009;18:321–36. 21. Lagarde M. Investigating attribute non-attendance and its consequences in choice experiments with latent class models. Health Econ. 2012;22(5):554–67. 22. Fossum M, Alexander GL, G?ransson KE, Ehnfors M, Ehrenberg A. Registered nurses’ thinking strategies on malnutrition and pressure ulcers in nursing homes: a scenario-based think-aloud study. J. Clin. Nurs. [Internet]. Wiley/Blackwell (10.1111); 2011 [cited 2018 Jul 13];20(17–18):2425–35. 23. Reicks M, Smith C, Henry H, Reimer K, Atwell J, Thomas R. Use of the think aloud method to examine fruit and vegetable purchasing behaviors among low-income african american women. J. Nutr. Educ. Behav. 2003;35(3):154–60. 24. Vass CM, Rigby D, Payne K. The role of qualitative research methods in discrete choice experiments: a systematic review and survey of authors. Med. Decis. Mak. 2017;37(3):298–313. 25. Coast J, Flynn T, Natarajan L, Sproston K, Lewis J, Louviere J, et al. Valuing the icecap capability index for older people. Soc. Sci. Med. [Internet]. 2008 [cited 2011 Jun 12];67(5):874–82. 26. Cheraghi-Sohi S, Bower P, Mead N, McDonald R, Whalley D, Roland M. Making sense of patient priorities: applying discrete choice methods in primary care using “think aloud” technique. Fam. Pract. [Internet]. 2007;24(3):276–82. 27. Cheraghi-Sohi S, Hole AR, Mead N, McDonald R, Whalley D, Bower P, et al. What patients want from primary care consultations: a discrete choice experiment to identify patients’ priorities. Ann. Fam. Med. [Internet]. United States: National Primary Care Research and Development Centre (NPCRDC), University of Manchester, Manchester, United Kingdom.; 2008;6(2):107–15. 28. Whitty J, Walker R, Golenko X, Ratcliffe J. A think aloud study comparing the validity and acceptability of discrete choice and best worst scaling methods. PLoS One [Internet]. 2014 [cited 2015 May 25];9(4):e90635. 29. Ericson K, Simon H. Protocol analysis: verbal reports as data (revised edition) [Internet]. 1993 [cited 2014 Oct 13]. 30. Baum M. Harms from breast cancer screening outweigh benefits if death caused by treatment is included. BMJ Br. Med. J. [Internet]. 2013 [cited 2013 Mar 7];346:f385. 31. Kirwan CC. Breast cancer screening: what does the future hold? BMJ Br. Med. J. [Internet]. 2013 [cited 2013 Mar 8];346(jan23 1):f87–f87. 32. Pace LE, Keating NL. A systematic assessment of benefits and risks to guide breast cancer screening decisions. JAMA [Internet]. 2014;311(13):1327–35. 33. Torjesen I. How much is too much breast screening? BMJ [Internet]. 2015;350(h139). 34. G?tzsche P, Nielsen M. Screening for breast cancer with mammography ( review ). Cochrane Libr. 2009;(4). 35. Center and center for bioethics and social sciences in medicine [Internet]. Univ. Michigan. 36. Vass CM, Rigby D, Payne K. Investigating the heterogeneity in women’s preferences for breast screening: does the communication of risk matter? Value Heal. 2018;21(2):219–28. 37. Lincoln YS, Guba EG. Naturalistic inquiry. Eval. Program Plann. Sage Publications; 1985; 38. Independent UK Panel on Breast Cancer Screening. The benefits and harms of breast cancer screening: an independent review. Lancet. Elsevier Ltd; 2012;380(9855):1778–86. 39. Snow SJ. Health and greater manchester in historical perspective. Represent. (McDougall Trust. [Internet]. Taylor & Francis; 2015 [cited 2018 Jul 13];51(4):439–52. 40. QSR. Nvivo qualitative data analysis software: version 10.0. 2012. 41. Gale N, Heath G, Cameron E. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. [Internet]. 2013 [cited 2014 Oct 13];1:117. 42. Hole AR. Small-sample properties of tests for heteroscedasticity in the conditional logit model. Econ. Bull. 2006;3:1–14. 43. Czajkowski M, Barczak A, Budziński W, Giergiczny M, Hanley N. Within- and between- sample tests of preference stability and willingness to pay for forest management. Univ. St Andrews Discuss. Pap. Environ. Econ. 2014;(6):1–24. 44. Frederick S, Loewenstein G, O’donoghue T. Time discounting and time preference: a critical review. J. Econ. Lit. [Internet]. 2002 [cited 2015 Jan 9];40(2):351–401. 45. Torgerson DJ, Raftery J. Discounting. BMJ Br. Med. J. 1999;319:914–5. 46. Hess R, Siegrist M. Risk communication with pictographs: the role of numeracy and graph processing. Judgm. Decis. Mak. 2011;6(3):263–74. 47. Galesic M, Garcia-Retamero R, Gigerenzer G. Using icon arrays to communicate medical risks: overcoming low numeracy. Heal. Psychol. [Internet]. 2009 [cited 2013 Sep 26];28(2):210–6. 48. Garcia-Retamero R, Galesic M, Gigerenzer G. Do icon arrays help reduce denominator neglect? Med. Decis. Mak. 2010;30:672–84. 49. Zikmund-Fisher BJ, Witteman HO, Dickson M, Fuhrel-Forbis A, Kahn VC, Exe NL, et al. Blocks, ovals, or people? icon type affects risk perceptions and recall of pictographs. Med. Decis. Mak. [Internet]. 2014;34(4):443–53. 50. Galesic M, Garcia-retamero R. Using analogies to communicate information about health risks. Appl. Cogn. Psychol. 2013;27:33–42. 51. Katapodi MC, Lee K, Facione NC, Dodd MJ. Predictors of perceived breast cancer risk and the relation between perceived risk and breast cancer screening: a meta-analytic review. Prev. Med. (Baltim). [Internet]. 2004 [cited 2015 Feb 2];38(4):388–402. 52. Veldwijk J, van der Heide I, Rademakers J, Schuit a J, de Wit GA, Uiters E, et al. Preferences for vaccination: does health literacy make a difference? Med. Decis. Making [Internet]. 2015;35(8):948–58. 53. Varian HALR. Microeconomic analysis. 3rd ed. Taxes. 1992. 54. Carlsson F, Kataria M, Lampi E. Dealing with ignored attributes in choice experiments on valuation of sweden’s environmental quality objectives. Environ. Resour. Econ. [Internet]. 2010 [cited 2014 May 7];47(1):65–89. 55. Augustovski F, Beratarrechea A, Irazola V, Rubinstein F, Tesolin P, Gonzalez J, et al. Patient preferences for biologic agents in rheumatoid arthritis: a discrete-choice experiment. Value Heal. [Internet]. 2013 [cited 2015 Feb 3];16(2):385–93. 56. Carlsson F, M?rkbak MR, Olsen SB. The first time is the hardest: a test of ordering effects in choice experiments. J. Choice Model. 2012;5(2):19–37. 57. Blamey R, Bennett J, Morrison M. Yea-saying in contingent valuation surveys. Land Econ. [Internet]. 1999 [cited 2015 Jan 7];75(1):126–41. 58. Boren T, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans. Prof. Commun. [Internet]. 2000 [cited 2014 Oct 13];43(3):261–78. 59. Durning SJ, Artino AR, Beckman TJ, Graner J, Van Der Vleuten C, Holmboe E, et al. Does the think-aloud protocol reflect thinking? exploring functional neuroimaging differences with thinking (answering multiple choice questions) versus thinking aloud. Med. Teach. 2013;35(9):720–6. Figure 1: Data collection processesFigure 2: Final coding tree identifying key themes and inter-theme relationships Linked Behaviour-5715000 Key issues in the empirical study Table 1: Interviewee characteristicsIDRisk format receivedAge band(years)OccupationEducationFemale 1Percentages only50+Analyst5+ O-levels/GCSEsFemale 2Percentages only35 to 44LecturerMaster’s degreeFemale 3Percentages only18 to 24Social WorkerUndergraduate degreeFemale 4Icon arrays25 to 34TeacherMaster’s degreeFemale 5Icon arrays18 to 24UnemployedUndergraduate degreeFemale 6Icon arrays25 to 34ResearcherUndergraduate degreeFemale 7Icon arrays25 to 34PhD StudentUndergraduate degreeFemale 8Icon arrays25 to 34ResearcherMaster’s degreeFemale 9Icon arrays25 to 34UnemployedUndergraduate degreeFemale 10Percentages only25 to 34AdministratorA/AS-levelsFemale 11Percentages only18 to 24Undergraduate StudentA/AS-levelsFemale 12Percentages only50+Information technology (IT)/ AdministratorMaster’s degreeFemale 13Percentages only50+AdministratorNVQs or equivalentFemale 14Percentages only50+AdministratorMaster’s degreeFemale 15Percentages only25 to 34Bar worker/ part-time studentUndergraduate degreeFemale 16Percentages only45 to 49Finance officerUndergraduate degreeFemale 17Icon arrays25 to 34AdministratorUndergraduate degreeFemale 18Icon arrays25 to 34EngineerMaster’s degreeFemale 19Icon arrays50+Retired1-4 O-levels/GCSEsTable 2: Results of the heteroskedastic conditional logit modelAttribute labelEstimatesASC (on none)-1.955**(0.76)Detect0.210***(0.04)UtilityRisk-0.101***(0.03)Costa-0.045(0.04)Icons*detect-0.044(0.12)Icons*risk-0.013(0.06)Scale termIcons0.115(0.52)Number of respondents = 19Number of observations = 627* p<0.05; ** p<0.01; ***p<0.001; ‘Icons’ identify individuals who received icon arrays and percentages; standard errors in parentheses; ASC is an alternative specific constant and represents the baseline utility from participating in screening beyond what is explained by the attributes presented; aCost attribute scaled so ?1=?100 so coefficient represents the effect of a ?100 change in the cost of the programmeAppendix A: Example choice questions with risk communicated as a percentage only or as icon arrays and percentagesAppendix B: Original coding frameworkThe original framework is shown by black square boxes, and recurring themes are displayed according to their focus: circles representing feelings and emotions; capsules showing logic and reasoning; the diamonds demonstrating expressions relating to the interviewees’ perceptions or experiences; and the white squares encapsulating opinions and beliefs. The arrows are suggested links and possible drivers of each phenomenon. 782320-1270Proposed linksFeelings/emotionsPerceptions/experiencesOpinions/ beliefsLogic/ reasoning00Proposed linksFeelings/emotionsPerceptions/experiencesOpinions/ beliefsLogic/ reasoningFigure B1: Intermediate coding tree demonstrating identified codes and themes ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download