Evidence-Based Medicine in Developing Countries



Sample Facilitator’s Manual

Table of Contents

Page

Program 2

Workshop 1

Objectives 3

Clinical Scenario 5

Facilitator’s Guide 6

Workshop 2

Objectives 21

Clinical Scenario 22

Facilitator’s Guide 23

Appendix

Tips for Facilitation 34

EBM Booklet 36

P r o g r a m

7:30 - 8:00 Registration

8:00 - 11:00 Workshop 1 – Making Therapeutic Decisions

11:00 - 11:45 Lecture: EBM and Obstacles to EBM

11:45 - 12:00 Open forum

12:00 - 1:00 Lunch

1:00 - 3:30 Workshop 2 – Making Diagnostic Decisions

3:30 - 4:00 Computer demo on Literature Search

4:00 - 4:45 Teaching and applying EBM

4:45 - 5:00 Open forum

Module 1

Making Therapeutic Decisions

GENERAL OBJECTIVE:

To apply the various rules of evidence in arriving at decision regarding the effectiveness and efficiency of treatment.

SPECIFIC OBJECTIVES:

1. To discuss the rationale for each of the user’s guides pertaining to the validity of claims on effectiveness;

2. To recognize the strengths and weaknesses of randomized controlled trials, as compared to other non-experimental study designs;

3. To define and differentiate the concepts of validity and precision, as they pertain to the results of clinical trials;

4. To recognize the strengths and weaknesses of intention-to-treat analyses as opposed to efficacy analyses;

5. To differentiate between dichotomous, continuous, and other scales of treatment outcome;

6. To recognize the differences between various measures of treatment effect including absolute risk reduction, relative risk, relative risk reduction, number needed to treat (NNT), and mean difference;

7. To define and differentiate between a point estimate and interval estimate of a treatment effect;

8. To differentiate clinically relevant endpoints from mechanistic endpoints in the evaluation of treatment effects.

REFERENCES:

1. Guyatt GH, Sackett DL, and Cook DJ for the Evidence-Based Medicine Working Group. User’s guides to the medical literature, II: how to use an article about therapy or prevention.

A. Are the results of the study valid? JAMA 1993; 270:2598-2601.

2. Guyatt, GH, Sackett DL, and Cook DJ for the Evidence Medicine Working Group. User’s guides to

The medical literature, II: how to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? JAMA 1993; 271:59-63.

CLINICAL SCENARIO (Making Therapeutic Decisions)

A 58 year old male patient, s/p coronary artery bypass grafting (CABG) three months ago, came to your clinic for follow-up. He is asymptomatic and has already reported back to office work. Apart from his present medications, he inquires whether Vit. E should be added to his regimen (an officemate informed him of the "beneficial" effects of Vit. E). Unsure about the answer, you search the literature and found the article by Stephens, et. al. which was published in Lancet 1996 and is entitled "Randomised controlled trial of Vitamin E in patients with coronary disease."

A. Read the article and critically appraise its validity using the Users' Guides on for an Article on Therapy.

NOTE: After appraising study validity, decide if you want to go on and read/discuss the rest of the article.

B. Appraise the results of the study, discussing the rationale for each.

C. Decide if you will prescribe Vitamin E for this patient.

A Sample Facilitators’ Guide for Workshop 1: Making Therapeutic Decisions

(2-3 Hours)

Note: In this guide, sentences in bold letters refer to the specific article being discussed. Sentences in plain text refer to parts of the discussion which are generic in nature, ie, they do not refer to a specific article. When preparing facilitator’s guides for new articles, we just replace the bold segments. The rest of the guides remain unchanged.

Title: Randomised Controlled Trial of Vitamin E in Patients with Coronary Disease: Cambridge Heart Antioxidant Study (CHAOS)

REMINDER TO THE FACILITATOR: KILL THE TEACHER IN YOU!

1. Was the assignment of patients to treatment randomized?

(Spend about 20 minutes here)

Answer: YES [p. 782, METHODS, 1st paragraph, 1st sentence. - "prospective, double-blind, placebo-controlled, randomized, single-center trial in the East Anglican region of UK…"]

Follow-up questions:

a. Why do we need to randomize?

Point to extract: To make the two groups equal.

b. What are the consequences of 2 groups being unequal?

Point to extract: Treatment can look better or worse. This is called BIAS.

c. So why do the 2 groups need to be equal?

Point to extract: To make sure the differences are really due to the

treatment.

d. How does randomization make sure groups are identical as to baseline characteristics?

Point to extract: Through sheer numbers. Suggest toss of coin.

e. Can we make 2 groups equal without randomization?

Point to extract: Only for known factors.

f. Should we insist on randomized trials for all treatment decisions we make?

Points to extract: Exceptions include: (1) Illnesses with uniformly fatal

or adverse outcome; (2) No known options for

treatment; (3) Treatment of few subjects reverses

uniform adverse outcome.

f.1 Think of specific conditions where you wouldn’t do an RCT?

Extract some general rules regarding exceptions (e.g. would you

do an RCT to decide if surgery is okay for ruptured

appendicitis?). Suggest the concept of Equipoise.

2. Was follow-up adequate? (Spend about 10 minutes here)

Answer : YES [p. 783, 2nd column, 2nd paragraph, 3rd sentence –

"complete follow-up data were available in 98%

participants…there were no differences between the

groups in completeness of follow-up”.

Follow-up questions:

a. Which are worrisome drop-out rates? (Illustrate using the table below and let participants choose).

|Control |Treatment |Control |Treatment |

|Drop-out rate |Drop-out rate |Death rate |Death rate |

|A. 1 % | 1 % |20 % |10 % |

|B. 1 % | 1 % | 2 % | 1 % |

|C. 10 % |10 % |50 % |10 % |

|D. 10 % |10 % |10 % | 5 % |

|E. 1 % |10 % |5 % | 5 % |

a.1 Which study worries you more because of their drop-out

rates, study A or B? (They should be able to work out that the

drop-outs in B are more worrisome).

a.2 Ask the same question about study B or C, giving them time to

think things over. Then ask them if they should worry about

the results of study E.

b. Ask participants to formulate general rules on when to worry, based on these examples.

Points to Extract: (1) When there is gross imbalance in drop-out rates

between Groups (e.g. study E); (2) When drop-out

rates are greater than the event rates (e.g. study D

and E); and (3) When worst assumptions on what

happened lead to opposite conclusions (eg-study D

and E).

3. Were the patients analyzed in the groups to which they were randomized?

(Spend about 15 minutes here)

Answer : YES [Refer: p. 783, 1st column, 3rd paragraph]

Follow-up questions:

a What is the difference between a censored analysis and an ITT analysis? Illustrate through this hypothetical example:

|N |Treatments compared |Failure Rate (Compliers) |Failure Rate |% Failure |

|(2,000) | |Analysis A |(Non-compliers) |(Total) |

| | | | |Analysis B |

|1,000 |6-mo. Course |100/1000 = 10% |0/0 |100/1000 = 10% |

|1,000 |1-yr course |45/900 = 5% |90/100 |135/1000 = 13.5% |

a.1 Which one is an ideal world analysis, which one is a real world

analysis?

Point to extract: A is ideal, B is real.

a.2 Which one addresses the question, "can the drug work?”, which

one addresses the question "will the drug work?"

Point to extract: A and B respectively.

a.3 Which one is ITT, which one is per-protocol analysis?

Point to extract: B and A respectively.

a.4 Which one should clinicians be interested in?

Point to extract: B

a.5 Who’s interested in B?

Point to extract: researchers, drug companies.

4. Were the patients, health workers, and study personnel blind to treatment?

(Spend about 5 minutes)

Answer : YES [See abstract]

a. Why blind patients? If they know they're on placebo, are they likely to feel better or worse?

Point to extract: Worse

a.1 Will this make the treatment look good or bad?

Point to extract: Good

b. Why blind health care providers (i.e. Doctors) ? If they know a patient is on placebo, might this affect their outcome assessment?

Point to extract: Depends

b.1 Will this make the treatment look better or worse?

Point to extract: Depends, usually better.

c. Why blind study personnel (i.e. people measuring the outcome)? If they know

a patient is on placebo, might this affect their outcome assessment?

Point to extract: Depends

c.1 Will this make the treatment look better or worse?

Point to extract: Depends, usually better.

d. Why is this a minor validity criterion? Can all studies be blinded?

Point to extract: Because you can’t blind all studies and it may not

necessarily invalidate the results of the study (e.g.

studies in which the outcome is objectively

determined, such as death).

5. Were the groups similar at the start of the trial? (Spend about 5 minutes here)

Answer : YES [See Table 1 - although slight difference is evident as explained in p. 783, 2nd column]

a. Why is this important?

Point to extract: This counterchecks if randomization was successful.

(NOTE: The magnitude of a difference is more

important than the p—value).

6. Aside from the experimental intervention, were the groups treated equally?

(Spend about 5 minutes here)

Answer : YES [p. 782, last sentence, 1st column. … “There was actually no planned clinic follow-up. An initial supply of medications was dispensed at recruitment and ALL patients were asked to request for follow-up study medications. This was mailed to them when necessary .”]

a. Why is this important?

Point to extract: differences in care may shift results in favor of or

against treatment.

b. Why is this a minor criterion?

Point to extract: it merely double checks success of randomization.

STOP AT THIS POINT! Make a decision - are the study results valid? Should we go on and read? Point out - 1) There are no perfect studies, 2) distinguish between fatal and minor flaws.

7. How large was the treatment effect? (Spend about 30 minutes here)

Lead questions:

a. If you weighed 80 KG after the Christmas holidays, and 60 kg after a summer diet, what would be the ways of expressing your weight loss”?

Points to extract:

a. I lost 25% of my weight.(my relative weight reduction).

b. I am now 75% of what I used to weigh (my relative weight).

c. I lost 20 KG (my absolute weight reduction).

(This may be tabulated as column headings for the study exercise. An analogy with RRR, ARR and RR may then be made. Participants can probably create their own formulas).

|Control |Experimental |RRR |RR |ARR |

|80 KG |60 KG |25 % |75 % |20 KG |

|My past weight |My present weight |The percent weight I lost |The percent weight that |The absolute weight I lost|

| | | |remained | |

|Rc |Rt |(Rc-Rt)/Rc |Rt/Rc |Rc-Rt |

|My past risk |My present risk |The percent risk I lost |The percent risk |The absolute risk I lost |

| | | |remaining | |

(Proceed to work on 2 sample endpoints. Estimate RRR, RR and ARR then

ask the participants to express in English).

|Endpoint: Non-fatal MI |

|Control |Treatment |RRR |RR |ARR |

|Placebo |Vitamin E | | | |

|(n=967) |(n=1035) | | | |

|41 |14 |.042 - .014 |.014/.042 |.042 - .014 |

|Rc = 41/967 = .042 |Rt = 14/1035 = .014 |.042 |= .3 (30%) |= .028 (2.8 %) |

|= 4.2 % |= 1.4 % |= .70 (70 %) | | |

|Rate of endpoints in |Rate of endpoints in |70% reduction in Non-fatal |The rate of having a |Non-fatal MI is prevented in|

|control group |treatment group |MI when taking Vit. E |Non-fatal MI is now 30% of |2.8% of patients. |

| | |compared to those taking |what it used to be. | |

| | |placebo. | | |

Facilitators may want to stop after RRR, ask “will you use the drug?” Then ask again after calculating ARR. A difference in reponse may be useful to point out the importance of understanding the differences in these estimates of effectiveness.

|Endpoint: Non-cardiac Deaths |

|Placebo |Vitamin E |RRR |RR |ARR |

|Rc = 3/967 = .0031|Rt = 9/1035 |= -1.8 OR -180% (a relative |= 2.8 OR 280% |= -0.0056 or |

|= 0.31 % |= .0087 |risk increase!) | |-0.56 % (an absolute risk |

| |= 0.87 % | | |increase) |

|Rate of Control |Rate of Treatment |Instead of a relative risk |The rate of having a |There's a 0.56% increase in |

| | |reduction, this now becomes a|Non-cardiac death is now |Non-cardiac Deaths. |

| | |relative risk increase. The |280% of what it used to | |

| | |risk of having a non-cardiac |be. | |

| | |death is 1.8X more compared | | |

| | |than the control. | | |

SUMMARY:

a. In which measure are MDs more likely to adopt treatment?

Point to extract: RRR

b. In which measure are MDs more likely to reject treatment?

Point to extract: ARR :……. even if the results are from the same trial!

c. illustrate: Compute for RRR, ARR; Which drug is helping more patients?

| |Control Deaths |Treatment Deaths |RRR |ARR |

|Drug A |10% |8% |0.2 (20%) |.02 (2%) |

|Drug B |20% |16% |0.2 (20%) |.04 (4%) |

|Drug C |50% |40% |0.2 (20%) |.10 (10%) |

8. How precise was the estimate of the treatment effect? (Spend about 15 minutes here)

[p. 784, 1st column - 2nd column, Relative Risks]

Major cardiovascular event 0.53 [95% CI 0.34-0.83]

Non-fatal MI 0.23 [95% CI 0.11-0.47]

Cardiovascular death 1.18 [95% CI 0.62-2.27]

NOTE: (analysis was done using Cox analysis), therefore results may differ from

our own calculations of RR).

a. What is the difference between a point estimate and an interval estimate?

a.1 Suggested approach:

Ask for an estimate of the average height or weight (or some other

parameter) of people in the room. Ask them regarding the probability that

the estimate is correct. Now ask for an interval estimate. Then, ask them

about the probability that this is correct.

Points to extract:

(1) interval estimates are humbler because they accept a range of possibilities; (2) interval estimates are more likely to be correct; (3) more useful because aside from suggesting statistical significance, they convey a message regarding magnitude of effect, i.e. the best and worst scenario; (4) all points have interval estimates.

b. What does the p-value mean when reported with point estimate of a treatment effect?

Point to extract: p is the probability that the observed differences are due to

chance

c. How does this form of reporting relate with interval estimates of treatment effects?

Point to extract (OR teach): p ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download