3-day EBM workshop exercises - Home - CEBM
Some of your sessions may be at the Old Road Campus (Churchill Hospital site) – so here is a map of how to get to the relevant buildings.
[pic]
Evidence Based Medicine
Thread Course Notes
TABLE OF CONTENTS
|Topic |Pages |
|Introduction |3 |
|Program |4 |
|Lecture: Introduction to Evidence Based Medicine |5 |
|Session 1: (A) Analysis of Asbtracts & |8 |
|(B) Critical Appraisal of a Therapy Study | |
|Session 2: Systematic Rewiews & |15 |
|Session 2: Asking questions & Library session |17 |
|Session 3: Preparation instructions & |24 |
|Session 3: Short Presentations |27 |
|OTHER READINGS | |
|Finding the Gold in Medline: clinical queries |29 |
|Measures of Assocation |31 |
|Measurement Scales |33 |
|Sample Size Rules | |
|Glossary | |
| | |
An introduction to
Evidence Based Medicine
“Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values” - Dave Sackett
Welcome to the evidence-based medicine theme. The primary purpose of EBM sessions is to give you concrete experience in using searching and critical appraisal skills and in the context of you current clinical learning and future practice. Our aim is to give you some basic skills that will be useful in your other medical terms (e.g., for writing up case reports) and for your own life long learning and future medical career.
At the end of this year you should be able to:
❑ Recognise and formulate your own answerable clinical questions
❑ Identify the type of research that best answers the different classes of clinical questions.
❑ Know which of several research databases (MEDLINE, Cochrane, Embase, etc)and secondary resources (Clinical Evidence, Guidelines) are most likely to be helpful in answering different types of clinical questions.
❑ Search using multiple text words and MeSH heading connected by Booleans (AND, OR, NOT) and truncations (* and $)
❑ Appraise, and apply the results of different types of research studies to help in the management of individual patients. (Note: we will only cover therapy studies in detail; the other types – diagnosis, prognosis, aetiology - will be covered throughout the year).
❑ Express the results of clinical trials in terms of both relative and absolute risk reductions, and be able to explain the numerical results to a patient.
Recommended Text
Straus SE, Richardson WS, Glasziou PP, Haynes RB. Evidence-based Medicine: How to Practise and Teach EBM. Third Edition. Churchill Livingstone: Edinburgh, 2005 or
Badenoch D, Heneghan C. Evidence-Based Medicine Toolkit, BMJ Publishing, 2002.
Also - contains useful material, including a toolbox which has a glossary and useful breif summaries of the types of studies.
Contact
Dr Paul Glasziou Tel: 289298; email: Paul.Glasziou@dphpc.ox.ac.uk
PROGRAM
q
The program below will introduce you to the basics of Evidence-Based Medicine. During your terms in later years will include sessions on Evidence-Based Medicine and Evidence-Based Surgery. These will include sessions on diagnosis, on appraisal of surgical treatments, and further critically appraised topics of your own.
|When? Where? |What? |Why? |
|Thursday |Lecture/Tutorial |Introduction to the 4-step process of EBM |
|22nd October |Introduction to EBM & Analysis of Abstracts |(ask, search, appraise, & apply research |
|9:00-10:15 | |evidence). |
| | | |
|10:30-11:30 |Small Group tutorial |Practice of trial appraisal |
| |Critical Appraisal of Trials | |
|JR Hospital | | |
|Classroom 7 | | |
|Thursday |Lecture/Tutorial |Introduction to systematic reviews and |
|5th November |Appraising Systematic Reviews |appraisal |
|9:00-10:15 | | |
| | | |
| |Searching Session | |
|10:30-11:30 | |Search & preparation for developing your own |
| | |topic for January session. |
|JR Hospital | | |
|Classroom 7 | | |
|To be announced |Lecture |Interpretation of diagnostic test information|
| |Diagnostics | |
| | | |
| | |Each Student will present for about 10 |
| |Presentation Session |minutes on their topic of choice |
| |Presentation of CATs | |
| |(critically appraised topics) | |
|November |Lecture |Overview of study types |
| |Introduction to Epidemiology | |
| |(Adrian Smith) | |
|February |Statistics Tutorial |Basic statistics plus answering any questions|
| |(Optional session) | |
| | | |
Lecture slides here
S
ession ONE A: Analysis of Abstracts
This first session aims to familiarise you with the PICO structure of questions, and being able rapidly recognise these in research articles.
PART A. Exercise: study designs
Read the abstracts from published studies on the following pages and answer the following questions for each study:
1. What is the question (PICO) of the study?
2. What is the purpose of the study?
a. intervention
a. frequency (incidence or prevalence)
b. diagnostic accuracy
c. prognosis (or natural history)
d. aetiology and risk factors
3. Which study type would give the highest quality evidence to answer the question? (see ‘Levels of evidence table’)
4. Which is the best study type that is also feasible? (You can use the Table below as a guide)
5. What is the study type used?
Table: levels of evidence according to type of research question
|Level |Intervention |Diagnosis ** |Prognosis |Aetiology ††† |
|I |A systematic review of level II |Systematic review of level II|A systematic review of |A systematic review |
| |studies |studies |level II studies |of level II studies |
|II |A randomised controlled trial |Cross-sectional study among |A prospective inception |A prospective cohort|
| | |consecutive presenting |cohort study |study |
| | |patients | | |
|III-1 |A pseudo-randomised controlled trial |Cross-sectional study among |untreated control patients|A retrospective |
| |(eg alternate allocation or some |non-consecutive patients |in a randomised controlled|cohort study |
| |other method) | |trial | |
|III-2 |A comparative study with concurrent |Diagnostic case-control study|A retrospectively |A case-control study|
| |control group: | |assembled cohort study | |
| |Nonrandomised experimental study | | | |
| |Cohort study, case-control study, | | | |
| |interrupted time series with a | | | |
| |control group | | | |
|III-3 |A comparative study without | | | |
| |concurrent control group: | | | |
| |Historical control study | | | |
| |Comparison of two or more single arm | | | |
| |studies (ie case series from two | | | |
| |studies) | | | |
| |Interrupted time series without a | | | |
| |parallel control group | | | |
|IV |Case series |Case series |Case series, or cohort |A cross-sectional |
| | | |study of patients at |study |
| | | |different stages of | |
| | | |disease | |
Abstract 1
Voutilainen S, Rissanen TH, Virtanen J, Lakka TA, Salonen JT; Kuopio Ischemic Heart Disease Risk Factor Study. Low dietary folate intake is associated with an excess incidence of acute coronary events: The Kuopio Ischemic Heart Disease Risk Factor Study.
BACKGROUND: Although several prospective studies have shown that low folate intake and low circulating folate are associated with increased risk of coronary heart disease (CHD), the findings are inconsistent.
METHODS AND RESULTS: We studied the associations of dietary intake of folate, vitamin B(6), and vitamin B(12) with the risk of acute coronary events in a prospective cohort study of 1980 Finnish men 42 to 60 years old examined in 1984 to 1989 in the Kuopio Ischemic Heart Disease Risk Factor Study. Nutrient intakes were assessed by 4-day food record. During an average follow-up time of 10 years, 199 acute coronary events occurred. In a Cox proportional hazards model adjusted for 21 conventional and nutritional CHD risk factors, men in the highest fifth of folate intake had a relative risk of acute coronary events of 0.45 (95% CI 0.25 to 0.81, P=0.008) compared with men in the lowest fifth. This association was stronger in nonsmokers and light alcohol users than in smokers and alcohol users. A high dietary intake of vitamin B(6) had no significant association and that of vitamin B(12) a weak association with a reduced risk of acute coronary events.
CONCLUSIONS: The present work in CHD-free middle-aged men is the first prospective cohort study to observe a significant inverse association between quantitatively assessed moderate-to-high folate intakes and incidence of acute coronary events in men. Our findings provide further support in favor of a role of folate in the promotion of good cardiovascular health.
|Question |Answer |
|1. What is the question (PICO) of the study? |P |
| |I |
| |C |
| |O |
|2. What is the purpose of the study? | |
|3. Which study type would give the highest quality evidence to answer the | |
|question? | |
|4.Which is the best study type that is also feasible? | |
|5.What is the study type used? | |
Abstract 2
Lonn E, et al for the HOPE 2 Investigators. Homocysteine lowering with folic acid and B vitamins in vascular disease. N Engl J Med. 2006
BACKGROUND: In observational studies, lower homocysteine levels are associated with lower rates of coronary heart disease and stroke. Folic acid and vitamins B6 and B12 lower homocysteine levels. We assessed whether supplementation reduced the risk of major cardiovascular events in patients with vascular disease.
METHODS: We randomly assigned 5522 patients 55 years of age or older who had vascular disease or diabetes to daily treatment either with the combination of 2.5 mg of folic acid, 50 mg of vitamin B6, and 1 mg of vitamin B12 or with placebo for an average of five years. The primary outcome was a composite of death from cardiovascular causes, myocardial infarction, and stroke.
RESULTS: Mean plasma homocysteine levels decreased by 2.4 micromol per liter (0.3 mg per liter) in the active-treatment group and increased by 0.8 micromol per liter (0.1 mg per liter) in the placebo group. Primary outcome events occurred in 519 patients (18.8 percent) assigned to active therapy and 547 (19.8 percent) assigned to placebo (relative risk, 0.95; 95 percent confidence interval, 0.84 to 1.07; P=0.41). As compared with placebo, active treatment did not significantly decrease the risk of death from cardiovascular causes (relative risk, 0.96; 95 percent confidence interval, 0.81 to 1.13), myocardial infarction (relative risk, 0.98; 95 percent confidence interval, 0.85 to 1.14), or any of the secondary outcomes. Fewer patients assigned to active treatment than to placebo had a stroke (relative risk, 0.75; 95 percent confidence interval, 0.59 to 0.97). More patients in the active-treatment group were hospitalized for unstable angina (relative risk, 1.24; 95 percent confidence interval, 1.04 to 1.49).
CONCLUSIONS: Supplements combining folic acid and vitamins B6 and B12 did not reduce the risk of major cardiovascular events in patients with vascular disease.
|Question |Answer |
|1. What is the question (PICO) of the study? |P |
| |I |
| |C |
| |O |
|2. What is the purpose of the study? | |
|3. Which study type would give the highest quality evidence to answer the | |
|question? | |
|4.Which is the best study type that is also feasible? | |
|5.What is the study type used? | |
Abstract 3
Chen SM, Chang MH, Du JC, Lin CC, Chen AC, Lee HC, Lau BH, Yang YJ, Wu TC, Chu CH, Lai MW, Chen HL; Taiwan Infant Stool Color Card Study Group, 2006.
Screening for biliary atresia by infant stool color card in Taiwan. Pediatrics 117(4):1147–54.
OBJECTIVE: We aimed to detect biliary atresia (BA) in early infancy to prevent additional liver damage because of the delay of referral and surgical treatment and to investigate the incidence rate of BA in Taiwan.
METHODS: A pilot study to screen the stool color in infants for the early diagnosis of BA was undertaken from March 2002 to December 2003. We had designed an ‘infant stool color card’ with 7 numbers of different color pictures and attached it to the child health booklet. Parents were then asked to observe their infant's stool color by using this card. The medical staff would check the number that the parents chose according to their infant's stool color at 1 month of age during the health checkup and then send the card back to the stool color card registry center.
RESULTS: The average return rate was approximately 65.2% (78,184 infants). A total of 29 infants were diagnosed as having BA, and 26 were screened out by stool color card before 60 days of age. The sensitivity, specificity, and positive predictive value were 89.7%, 99.9%, and 28.6%, respectively. Seventeen (58.6%) infants with BA received a Kasai operation within 60-day age period. The estimated incidence of BA in screened newborns was 3.7 of 10,000.
CONCLUSIONS: The stool color card was a simple, efficient, and applicable mass screening method for early diagnosis and management of BA. The program can also help in estimating the incidence and creating a registry of these patients.
|Question |Answer |
|1. What is the question (PICO) of the study? |P |
| |I |
| |C |
| |O |
|2. What is the purpose of the study? | |
|3. Which study type would give the highest quality evidence to answer the | |
|question? | |
|4.Which is the best study type that is also feasible? | |
|5.What is the study type used? | |
S
ession One B: Critical Appraisal of a Therapy Study
Scenario: A young woman presents having had a tick bite after walking in the New Forest 3 days ago. She removed most of the tick, and feels well but is concerned about developing Lyme disease which she has heard can be transimitted by ticks. She asks you if there is anything she should do?
What question(s) does this scenario raise? (try to PICO these)
Now work though the critical appraisal worksheets for the NEJM article, and:
❑ decide what question (PICO) the study asked and answered
❑ whether the internal validity of the study is sufficient to allow firm conclusions (all studies have some flaws; but are these flaws sufficient to discard the study?)
❑ if the study is sufficiently valid, look at and interpret the results – what is the relevance or size of the effects of the intervention?
decide whether and how the results would apply to the original scenario or patient question.
THERAPY STUDY: Are the results of the trial valid? (Internal Validity)
|R- Was the assignment of patients to treatments randomised? |
|What is best? |Where do I find the information? |
|Centralised computer randomisation is ideal and often used in |The Methods should tell you how patients were allocated to groups |
|multi-centred trials. Smaller trials may use an independent |and whether or not randomisation was concealed. |
|person (e.g, the hospital pharmacy) to “police” the | |
|randomization. | |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|R- Were the groups similar at the start of the trial? |
|What is best? |Where do I find the information? |
|If the randomisation process worked (that is, achieved comparable|The Results should have a table of "Baseline Characteristics" |
|groups) the groups should be similar. The more similar the groups|comparing the randomized groups on a number of variables that |
|the better it is. |could affect the outcome (ie. age, risk factors etc). If not, |
|There should be some indication of whether differences between |there may be a description of group similarity in the first |
|groups are statistically significant (ie. p values). |paragraphs of the Results section. |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|A - Aside from the allocated treatment, were groups treated equally? |
|What is best? |Where do I find the information? |
|Apart from the intervention the patients in the different groups |Look in the Methods section for the follow-up schedule, and |
|should be treated the same, eg., additional treatments or tests. |permitted additional treatments, etc and in Results for actual |
| |use. |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|A - Were all patients who entered the trial accounted for? - and were they analysed in the groups to which they were randomised? |
|What is best? |Where do I find the information? |
|Losses to follow-up should be minimal - preferably less than 20%.|The Results section should say how many patients were randomized |
|However, if few patients have the outcome of interest, then even |(eg., Baseline Characteristics table) and how many patients were |
|small losses to follow-up can bias the results. Patients should |actually included in the analysis. You will need to read the |
|also be analysed in the groups to which they were randomised – |results section to clarify the number and reason for losses to |
|‘intention-to-treat analysis’. |follow-up. |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|M - Were measures objective or were the patients and clinicians kept “blind” to which treatment was being received? |
|What is best? |Where do I find the information? |
|It is ideal if the study is ‘double-blinded’ – that is, both |First, look in the Methods section to see if there is some mention|
|patients and investigators are unaware of treatment allocation. |of masking of treatments, eg., placebos with the same appearance |
|If the outcome is objective (eg., death) then blinding is less |or sham therapy. Second, the Methods section should describe how |
|critical. If the outcome is subjective (eg., symptoms or |the outcome was assessed and whether the assessor/s were aware of |
|function) then blinding of the outcome assessor is critical. |the patients' treatment. |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
What were the results?
|How large was the treatment effect? |
|Most often results are presented as dichotomous outcomes (yes or not outcomes that happen or don't happen) and can include such |
|outcomes as cancer recurrence, myocardial infarction and death. Consider a study in which 15% (0.15) of the control group died and |
|10% (0.10) of the treatment group died after 2 years of treatment. The results can be expressed in many ways as shown below. |
|What is the measure? |What does it mean? |
|Relative Risk (RR) = risk of the outcome in the treatment |The relative risk tells us how many times more likely it is that an event|
|group / risk of the outcome in the control group. |will occur in the treatment group relative to the control group. An RR of|
| |1 means that there is no difference between the two groups thus, the |
| |treatment had no effect. An RR < 1 means that the treatment decreases the|
| |risk of the outcome. An RR > 1 means that the treatment increased the |
| |risk of the outcome. |
|In our example, the RR = 0.10/0.15 = 0.67 |Since the RR < 1, the treatment decreases the risk of death. |
|Absolute Risk Reduction (ARR) = risk of the outcome in the|The absolute risk reduction tells us the absolute difference in the rates|
|control group - risk of the outcome in the treatment |of events between the two groups and gives an indication of the baseline |
|group. This is also known as the absolute risk difference.|risk and treatment effect. An ARR of 0 means that there is no difference |
| |between the two groups thus, the treatment had no effect. |
|In our example, the ARR = 0.15 - 0.10 = 0.05 or 5% |The absolute benefit of treatment is a 5% reduction in the death rate. |
|Relative Risk Reduction (RRR) = absolute risk reduction / |The relative risk reduction is the complement of the RR and is probably |
|risk of the outcome in the control group. An alternative |the most commonly reported measure of treatment effects. It tells us the |
|way to calculate the RRR is to subtract the RR from 1 (eg.|reduction in the rate of the outcome in the treatment group relative to |
|RRR = 1 - RR) |that in the control group. |
|In our example, the RRR = 0.05/0.15 = 0.33 or 33% |The treatment reduced the risk of death by 33% relative to that occurring|
|Or RRR = 1 - 0.67 = 0.33 or 33% |in the control group. |
|Number Needed to Treat (NNT) = inverse of the ARR and is |The number needed to treat represents the number of patients we need to |
|calculated as 1 / ARR. |treat with the experimental therapy in order to prevent 1 bad outcome and|
| |incorporates the duration of treatment. Clinical significance can be |
| |determined to some extent by looking at the NNTs, but also by weighing |
| |the NNTs against any harms or adverse effects (NNHs) of therapy. |
|In our example, the NNT = 1/ 0.05 = 20 |We would need to treat 20 people for 2 years in order to prevent 1 death.|
|How precise was the estimate of the treatment effect? |
|The true risk of the outcome in the population is not known and the best we can do is estimate the true risk based on the sample of |
|patients in the trial. This estimate is called the point estimate. We can gauge how close this estimate is to the true value by |
|looking at the confidence intervals (CI) for each estimate. If the confidence interval is fairly narrow then we can be confident |
|that our point estimate is a precise reflection of the population value. The confidence interval also provides us with information |
|about the statistical significance of the result. If the value corresponding to no effect falls outside the 95% confidence interval |
|then the result is statistically significant at the 0.05 level. If the confidence interval includes the value corresponding to no |
|effect then the results are not statistically significant. |
Will the results help me in caring for my patient? (ExternalValidity/Applicability)
|The questions that you should ask before you decide to apply the results of the study to your patient are: |
|Is my patient so different to those in the study that the results cannot apply? |
|Is the treatment feasible in my setting? |
|Will the potential benefits of treatment outweigh the potential harms of treatment for my patient? |
S
ession TWO: Appraisal of Systematic Reviews
Case Study
A 55 year old man with known osteoarthritis presents because his pain is getting much worse. In particular the right knee is much worse than the left. Examination reveals some warmth, but no effusion.
He is currently taking regular paracetamol and he cannot tolerate non-steroidal anit-inflammatories (NSAIDs) because of his past gastric ulceration. He asks if an injection is possible, as he has had friends who have had this. So you consider whether a corticosteriod injection* might be worthwhile.
Questions
1. What are you clinical questions (PICOs)?
2. What sort of evidence would be helpful to answer the question(s)?
3. Where would you search for evidence and what terms would you use?
4. Now read and appraise the attached systematic review. How does their search compare to yours?
SYSTEMATIC REVIEW: Are the results of the review valid?
|What question (PICO) did the systematic review addressed? |
|What is best? |Where do I find the information? |
|The main question being addressed should be clearly stated. The |The Title, Abstract or final paragraph of the Introduction |
|exposure, such as a therapy or diagnostic test, and the |should clearly state the question. If you still cannot |
|outcome(s) of interest will often be expressed in terms of a |ascertain what the focused question is after reading these |
|simple relationship. |sections, search for another paper! |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|F - Is it unlikely that important, relevant studies were missed? |
|What is best? |Where do I find the information? |
|The starting point for comprehensive search for all relevant |The Methods section should describe the search strategy, |
|studies is the major bibliographic databases (e.g., Medline, |including the terms used, in some detail. The Results section|
|Cochrane, EMBASE, etc) but should also include a search of |will outline the number of titles and abstracts reviewed, the|
|reference lists from relevant studies, and contact with experts, |number of full-text studies retrieved, and the number of |
|particularly to inquire about unpublished studies. The search |studies excluded together with the reasons for exclusion. |
|should not be limited to English language only. The search |This information may be presented in a figure or flow chart. |
|strategy should include both MESH terms and text words. | |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|A - Were the criteria used to select articles for inclusion appropriate? |
|What is best? |Where do I find the information? |
|The inclusion or exclusion of studies in a systematic review |The Methods section should describe in detail the inclusion |
|should be clearly defined a priori. The eligibility criteria used|and exclusion criteria. Normally, this will include the study|
|should specify the patients, interventions or exposures and |design. |
|outcomes of interest. In many cases the type of study design | |
|will also be a key component of the eligibility criteria. | |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|A - Were the included studies sufficiently valid for the type of question asked? |
|What is best? |Where do I find the information? |
|The article should describe how the quality of each study was |The Methods section should describe the assessment of quality|
|assessed using predetermined quality criteria appropriate to the |and the criteria used. The Results section should provide |
|type of clinical question (e.g., randomization, blinding and |information on the quality of the individual studies. |
|completeness of follow-up) | |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
|T - Were the results similar from study to study? |
|What is best? |Where do I find the information? |
|Ideally, the results of the different studies should be similar |The Results section should state whether the results are |
|or homogeneous. If heterogeneity exists the authors may estimate |heterogeneous and discuss possible reasons. The forest plot |
|whether the differences are significant (chi-square test). |should show the results of the chi-square test for |
|Possible reasons for the heterogeneity should be explored. |heterogeneity and if discuss reasons for heterogeneity, if |
| |present. |
|This paper: Yes ( No ( Unclear ( |
|Comment: |
What were the results?
|How are the results presented? |
|A systematic review provides a summary of the data from the results of a number of individual studies. If the results of the |
|individual studies are similar, a statistical method (called meta-analysis) is used to combine the results from the individual |
|studies and an overall summary estimate is calculated. The meta-analysis gives weighted values to each of the individual studies |
|according to their size. The individual results of the studies need to be expressed in a standard way, such as relative risk, odds |
|ratio or mean difference between the groups. Results are traditionally displayed in a figure, like the one below, called a forest |
|plot. |
|[pic] |
|The forest plot depicted above represents a meta-analysis of 5 trials that assessed the effects of a hypothetical treatment on |
|mortality. Individual studies are represented by a black square and a horizontal line, which corresponds to the point estimate and |
|95% confidence interval of the odds ratio. The size of the black square reflects the weight of the study in the meta-analysis. The |
|solid vertical line corresponds to ‘no effect’ of treatment - an odds ratio of 1.0. When the confidence interval includes 1 it |
|indicates that the result is not significant at conventional levels (P>0.05). |
|The diamond at the bottom represents the combined or pooled odds ratio of all 5 trials with its 95% confidence interval. In this |
|case, it shows that the treatment reduces mortality by 34% (OR 0.66 95% CI 0.56 to 0.78). Notice that the diamond does not overlap |
|the ‘no effect’ line (the confidence interval doesn’t include 1) so we can be assured that the pooled OR is statistically |
|significant. The test for overall effect also indicates statistical significance (p 1 there is possible heterogeneity. If Cochran Q is not statistically significant |
|and Q/df is < 1 then heterogeneity is very unlikely. In the example above Q/df is ................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.