University of Alberta
Quality improvement in primary care – The tyranny of the acute and the changes required – Theory and principles
Bodenheimer T. Heping patients improve their health-related behaviors: What system changes do we need? Disease Management, 8(5): 2005.
- Problems with the management of chronic diseases
o Diabetes – HbA1c > 7.0% - 63% prevalence
o Only 65% eat the prescribed diet, 19-30% perform regular exercise, and 7% carry out all recommended self-management behaviors.
- What sort of problem is this?
o Physician? Counter example – rushed atmosphere, competing demands.
o Patient? Counter example – inability to pay, overwhelmed with fraught agenda that is not patient-centered, inadequate enlistment, etc..
o System?
- Objective: Argument – encouraging patients to become informed and activated is crucial to the adoption of healthy behaviors and the improvement of clinical outcomes – this requires redesigned primary care on the system level.
- CCM
o Wagner EH, Austin BT, Davis C, et al. Health Aff (Millwood), 20: 2001
o Goals
▪ Productive interactions
▪ Informed and activated patient
▪ Prepared, proactive practice team
- Informed, activated patient
o Information giving necessary, but not sufficient condition for promoting healthy behaviors and improved outcomes.
o An additional factor is needed (Norris SL, Engelgau MM, Narayan KMV. Effectiveness of self-management training in type 2 diabetes. Diabetes Care, 24: 2001).
o Collaborative decision making and goal setting
▪ Improves concordance of goals (Golin et al) (17, 23)
▪ Improves understanding
▪ Improves self-efficacy (Goli CE, DiMatteo MR, Gelberg L. Diabetes Care, 19: 1996) (Roter DL. Health Educ Monogr, 5: 1997).
• Self-efficacy = patients’ level of confidence that they can make improvements in their lives.
• RCT – participatory model of self-management training – improved self-efficacy and health outcomes (Lorig KR, Ritter P, Stewart AL, et al. Med Care, 39: 2001) (Anderson RM, Funnell MM, Butler PM, et al.. Diabetes Care, 18: 1995)
• E.g.: DPP
• See this chapter in the book by Bodenheimer and Grumsbach.
o “Enhancing patient-provider communication and shared decision making have been shown to result in greater patient satisfaction, adherence to treatment plans, and improved health outcomes … The consistency of these studies’ findings … is impressive” (Heisler M, Bouknight RR, Hayward RA, et al. J Gen Intern Med, 17: 2002).
- Current system
o Essence = the 15-minute office visit
o Information giving
▪ Patients interrupted after only 23 seconds. In only 28% of visits were patients able to express concerns completely.
▪ 76% of type 2 diabetes patients received limited or no diabetes education (Clement S. Diabetes self-management education. Diabetes Care, 18: 1995).
▪ As many as 50% of patients leave an office visit not understanding what they were told (Roter DL, Hall JA. Annu Rev Public Health, 10: 1989).
▪ Closing the loop works, but is rarely done (12%) (Schillinger D, Piette J, Grumbach K, et al. Closing the loop. Physician communication with diabetic patients who have low health literacy. Arch Intern Med, 163: 2003),.
o Collaborative decision-making
▪ In the vast majority of visits, patient did not receive information required for informed consent (options and consequences).
o Limiting factor = visit-length
▪ Average duration of primary care visit in the US may actually have increased.
▪ But the number of tasks required has outpaced time.
▪ It would take a physician 7.4 hours per day to provide all recommended preventive services to a typical patient panel (Chobanian AV, Barkis GL, Black HR, et al.. JAMA, 289: 2003), let alone the demands of acute care.
▪ Longer visits associated with more health education and preventive care.
▪ Time – a major barrier preventing physicians from providing sufficient information to their patients, and blocking their ability to incorprate shared decision-making into office practice.
▪ Time – prevents patients from seeking the information they want, or being engaged as they would want.
▪ Time = a system problem
- Redesigned system
o Pre-activation
▪ A process of assisting patients to be more assertive during them edical care visit.
▪ Pre-activated patients in RCTs were more assertive, received more information, and reported better overall functional capacity.
▪ However, time limits and physician dominance may curtail the impact of pre-activation – pre-activated patients in one study were more anxious and angry, and less satisfied with care.
▪ Most important study (Greenfield S, Kaplan SH, Ware JE, et al. Patients’ participation in medical care. J Gen Intern Med, 3: 1988)
• Pre-activation – RA for 20 minuts prior to the physician visit – discuss medical decisions and rehearse negotiation skills.
• Average HbA1c decreased, and was lower in the intervention group vs control group – significant.
• However, visit lengths were 30 minutes – unrealistic.
▪ Impact on visit length mixed. Emerging theme – tight time constraints limits impact of pre-activation (Lang F. Arch Fam Med, 9: 2000).
- Planned visits
o A visit understood by physician and patients to have a limited agenda, focused on the chronic management of disease.
o Often led by nurses – individual or group settings.
o Several studies – group nurse or multidisciplinary team-led planned visit clinics – lower HbA1c in all studies. Lower CVD RFs, mortality or health care utilization has been observed.
o Planned chronic care visits, when compared with the standard 15-minute unplanned physician visit, improves clinical outcomes for patients with chronic conditions.
- Regular and sustained follow-up
o Sustaining major behavior change requires ongoing support.
o Follow-up by regular case managers improves HbA1c. Contact can be as simple as periodic telephone calls (E.g.: Piette JD, Weinberger M, Kraemer FB, et al. Diabetes Care, 24: 2001). However, another study concluded that more intensive and sustained support may be necessary (Glasgow RE, Toobert KJ. Brief, computer-assisted diabetes dietary self-management counseling. Med Care, 38: 2000).
o The impact of education diminish over time without sustained follow-up at regular intervals (Norris SL, Lau J, Smith SJ, et al. Diabetes Care, 25: 2002).
o In contrast, education without regular follow-up is unlikely to result in long-term success (Clement S. Diabetes Care, 18: 1995).
- Mechanism – information-giving and collaborative decision-making improve knowledge and self-efficacy = informed, activated patient, who then engages in healthier behaviors, and have improved clinical outcomes.
- Instead of seeing chronic care as inappropriate for the 15-minute visit, we should see the 15-minute visit as inappropriate for chronic care!
- If we take the 15-minute physician visit as fixed, then implementing these innovations will require additional staff and clear divisions of labor, leveraging community resources, and conducive organizational environments.
Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for patients with chronic illness. JAMA, 288(14): 2002.
- The majority of patients with common chronic conditions are inadequately treated.
- Tyranny of the urgent
o The acute symptoms and concerns of the patient crowd out the less urgent need to bring chronic illness under optimal management.
o An uninformed, passive patient interacts with an unprepared practice team, resulting in frustrating, inadequate encounters.
- CCM – 6 pillars in a tri-galactic universe – objective – informed, activated patients interacting with a prepared, proactive practice team, resulting in high-quality, satisfying encounters and improved outcomes.
o Chronic care take place within 3 overlapping galaxies – “tri-galactic universe”
▪ Entire community – myriad resources
▪ Health care system, including payment structures
▪ Provider organization
o Community resources and policies
▪ Closer to patient during their every day lives.
▪ More resources for lifestyle change and education.
o Health care organization – Structure, goals, and values; and reimbursement relations are foundational – a business case, as well as a leadership case, is needed.
o Self-management support
▪ Patients themselves become the principal care-givers for chronic conditions.
▪ Patients live with chronic illness for many years.
▪ Self-management support = collaboratively helping patients and their families acquire the skills and confidence to manage their chronic illness.
o Delivery system design
▪ Practice teams with a clear division of labor, separating acute care (physicians), from the planned management of chronic conditions (physicians and non-physician personnel).
▪ Panned visits.
o Clinical information systems
▪ Reminder systems
▪ Feedback
▪ Registries – lists of all patients with a particular chronic condition.
- CCM – began at Lovelace Health Systems in Albuquerque and Group Health Cooperative of Puget Sound.
o Lovelace innovation retrenched due to fiscal constraints – FFS made a comeback, and start-up capital diminished.
- Examples – numerous before-and-after studies
o Primier Health Partners
o HealthPartners Medical Group
o Clinica Campesina
Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for patients with chronic illness: the chronic care model, Part 2. JAMA, 288(15): 2002.
- Does the implementation of the CCM actually improve chronic care?
- Not subject to rigorous evaluation – but many of its elements have been included in interventions assessed by adequate study designs.
- SR of diabetes care programs featuring elements of the chronic care model, based on the Cochrane review by Renders et al..
- Additional quick and dirty Medline search for cost studies relating CCM components.
- Intervention classification
o Self-management (most common)
o Decision support (most frequently educational materials and meetings)
o Delivery system design (most frequently case managers, care teams, and scheduling of planned diabetes follow-up visits)
o Clinical information systems (reminder systems and performance feedback)
o Health care organization and community resources were not addressed in the Cochrane review.
- Outcomes
o Processes of care
o Patient outcomes
- No quantitative meta-analysis due to heterogeneity.
- 39 studies included. Most frequent – 2 CCM components implemented.
- Results
o 32 of 39 studies found that the intervention improved at least 1 process or outcome measure.
o Unable to determine whether the number of CCM components implemented, or specific CCM component combos, were more likely to be effective (lack of co-variation).
o Cost studies
▪ CHF – 3 of 7 – reduced health care use, costs or both.
▪ Asthma – 8 of 12 positive for reduced health care use, costs, or both.
▪ Diabetes – 7 of 9 positive for reduced health care use, costs, or both.
▪ For CHF and asthma, illness severity may play an important role – risk-stratifying chronic illness may be required for efficiency.
- Review supports the conclusion that the chronic care model, when implemented through multifaceted interventions, can improve process and outcome measures for diabetes.
- Business case for chronic care
o Requires ability to internalize the benefits of CCM implementation.
o E.g.: Fewer hospitalizations means reduced income unless the system receives capitated payments.
o E.g.: Savings should accrue to those who bear the costs of innovation for incentives to be aligned.
- Hospital systems
o Cost savings achievable from fewer hospital days and less ED use.
o No business case under per-diem, FFS, or DRG payments, where income is lost due to reductions in utilization.
o Also, the insurer, not the organization paying for the improvements, saves money – incentives mis-aligned.
o E.g.: Pioneering CHF program showed cost savings (Rich MW, Beckham V, Wittenberg C, Leven CL, Freedland KE, Carney RM. NEJM, 333: 1995). Discontinued due to financial concerns.
- Ambulatory systems
o Use of CCM innovations should result in increased utilization or complexity of care = higher FFS revenues, especially if complex care visits are billable.
- Payment reform
o Insurers need to reward providers who improve the care of chronic illness.
o Medicare is a good place to start.
- Barriers to implementation of the CCM
o Need favorable conditions in the 3 galaxies – institution, health care system, community.
o Visionary clinical champions needed (internal).
o Financial environment must help, or at least not hinder (external).
o Other internal barriers
▪ Clinical information systems – expensive, hard to install.
▪ Registries – require upfront investments so that they become an active tool.
▪ Physicians – must be rescued from the hamster syndrome, and convinced that change is necessary.
- Must take baby first steps appropriate to the context.
Grumbach K, Bodenheimer T. A primary care home for Americans: putting the house in order. JAMA, 288(7): 2002.
- Patients prefer to seek initial care for common problems from their primary care physician – survey data.
- Functions of a primary care home – “Primary care is …” and “The primary care home is where …”
o Fist-contact care
o Comprehensive
o Longitudinal
o Integrated
o Accessible
- The endangered primary care home
o Unrewarding work
▪ Physicians feel that they cannot spend sufficient time with patients.
▪ The number of US medical school graduates entering primary care is decreasing.
o Patient-physician concerns
▪ Patients are having difficulty gaining timely access to primary care
▪ Many ED visits are not urgent.
o Primary care practice is inadequately managing chronic illnesses.
▪ Only half of smokers are counseled about smoking cessation, etc..
▪ PCPs appear to perform as well as do specialists caring for patients with common chronic illnesses.
o Why is primary care endangered?
▪ External forces
• Medical culture of specialization.
• Managed care – administrative hassles, challenges to clinical autonomy, income reductions.
▪ Heightened expectations – changing societal demands on science and medicine.
▪ “Hamster health care” (Morrison I, Smith R. BMJ, 321: 2000).
- Value of a primary care home
o Cross-national studies – primary care-oriented systems have better health outcomes and lower costs.
o Within the US – states with more PCPs, but not specialists, have better population health indicators.
o Continuity of care is associated with better processes and outcomes.
o Generalists provide comparable quality of care at lower cost.
o PCPs play an important role – most patients value their PCPs, and most physician visits for prevalent, serious conditions, occur in PCP offices.
- Difficulties facing primary care
o Comprehensiveness
▪ Far more to do in the last few decades – preventive services, cancer screening, management of chronic illnesses.
▪ Increasing prevalence of chronic illnesses – mainly type 2 diabetes.
▪ Increasing capability, and therefore expectation, that these patients will be managed in the community – e.g.: previous practice was a period of hospitalization for a nonketotic patient with hyperglycemia.
▪ Increasing medicalization – depression, ADHD, substance abuse, domestic violence, etc..
o First-contact care
▪ Primary care organized to respond to acute and urgent problems – now suffer from tyranny of the urgent – acute illnesses crowding out chronic illness management and preventive care.
▪ New forms of communication may need to be implemented = challenging.
o Continuity and coordination
▪ Growing complexity of health care organization.
▪ Difficult to know what other providers are doing.
- Accountability – increasingly, physicians are subject to scrutiny in the form of audit and feedback, benchmarking, report cards.
- Paradox of time
o No evidence that the actual length of office visits in the US is shorter – in contrast, mean length is up from 16.3 to 18.3 minutes, 1989 to 1998.
o But increasing demands relative to time.
- Putting the primary care house in order
o Less saturated work environment needed. Must contain systems that improve access and quality while relieving physician workloads. Must occur without major increases in costs.
o Alternative – scenario – Relying exclusively on specialists
▪ Those with more than one chronic disease (50% of all chronic disease patients) – separate disease-specific programs.
▪ Patients responsible for initiating and arranging preventive care.
▪ Emphasis on disease and organ systems, not care of the whole person.
o Alternative scenario – vacate the primary care home to non-physicians
▪ New generation of clinicians would struggle the same way.
- Conclusions
o All health care systems need a sturdy primary care home.
o Future care models, even team care models, will require strong and continued physician presence.
o Solutions that do not address the central paradox of time may be counter-productive, and increase burn-out, or the sense of hamster medicine.
Davidson MB. How our current medical care system fails people with diabetes. Diabetes Care, 32(2): 2009.
- Diabetes care gaps
o Only half of US patients meet glycemic goal - NHANES.
o In other studies …
▪ 22-46% of patients have HbA1c > 9.5%
▪ 22-46% met LDL goal
▪ 29-33% met the BP goal
▪ Only 2-10% met the combined glycemia, lipd, and BP goals.
- Lack of time major issue
o One approach effective – using specially trained nurses of pharmacists, with authority to make medication changes according to approved algorithms – impressive results compared to CM in which nurses do not have prescriptive authority.
o Critical factor – timely and appropriate clinical decision are made.
o Appropriate – based on algorithms.
o Timely – more time can be spent with patients, outside of the usual 15 minutes every 3 months physician visit – stabilization will happen within 1 month.
o If this is correct, then replacing infrequent face-to-face time with more frequent tele-health communications could also be effective.
o Home glucose monitoring may be faxed to a nurse, who can decide on insulin dose changes – baseline A1c levels averageing 8.4% fell by 0.7% in 20 weeks (Davidson MB, Lewis G. Diabetes Care, 23: 2000). However, the business case could not be made (no billable code for this activity) and it was dropped.
o Many other examples can be found, of web-based monitoring successes. The key is conjectured to be a case manager making treatment decision in real time, based on published literature – case management studies without this role tend to be ineffective.
- Need – more frequent interactions with knowledgeable providers – timely and appropriate clinical decisions need to be made – perhaps by specially trained, appropriately supervised nurses and pharmacists with prescription authority based on approved treatment algorithms.
Quality improvement in primary care – Self-management training
Norris SL, Engelgau MM, Narayan KM. Effectiveness of self-management training in type 2 diabetes: a systematic review of randomized controlled trials. Diabetes Care, 24(3): 2001.
- 72 studies included. Interventions categorized as knowledge or information (didactic), knowledge or information (collaborative), lifestyle interventions (collaborative), or skills teaching (collaborative).
- Results – knowledge
o Improved knowledge in most studies.
o 7 studies showed improved knowledge for both control and intervention groups – contamination – infeasibility of blinding.
o Regular reinforcement or repetition of the intervention seemed to improve knowledge levels at variable lengths of follow-up.
- Results – self-care
o SMBG – more frequent or accurate, but no improvement in HbA1c.
o Foot lesions – mixed results.
- Results – lifestyle behaviors
o Most studies examining dietary changes were positive for self-reported changes.
o A few – corresponding improvements – weight (3 studies) or glycemic control (1 study)
o Physical activity – variable results – increased in some, no change in others.
- Results – psychological and QoL
o Mixed findings – improvements noted in problem solving and anxiety levels.
o 2 of 3 studies – no change in QoL.
- Results – glycemic control
o HbA1c tended to improve in both control and intervention groups.
o Improvement intervention > control – 14 studies,
o Percentage change – HbA1c = -26% to +4% in the intervention groups, and -33+ to +15% in the control groups.
o Improvement control > intervention – 3 studies, but NS in 2.
o Shorter length of follow-up had greater effectiveness. Longer follow-up studies – mixed effects.
o Prolonged interventions – mixed results – most studies NS despite regular patient contact.
o Method of delivery – Compared with didactic delivery, collaborative interventions produced somewhat more favorable results, particularly if interventions were repetitive and ongoing.
o Content delivered – studies focusing on lifestyle generally failed to show improvements compared with control.
o Improved glycemic control and increased knowledge were not consistently correlated.
o Use of computers – mixed results, balance towards positive effects. Only 4 studies.
- CVD risk factors
o Studies with positive results mostly involved regular contacts or reinforcement sessions, or very short follow-up periods. Weight decrease, average of 2kg, detected in 13 studies, vs control.
o However, an equal number of studies appeared to be NS.
o Only 3 studies involved didactic interventions – only 1 of 3 showed a decrease in weight.
o Lipid results – mixed – some positive, others initially positive but NS at final follow-up. Positive studies involved interactive, generally individualized, repetitive interventions.
o 3 didactic studies – 0 of 3 resulted in improved lipid profiles.
o BP control – mixed results as well.
o CVD events or mortality – 2 studies – Both NS.
- Results – Economic and health care utilization
o Most studies did not demonstrate improvement.
o No CBAs identified.
o Not cost-savings. CEA measured in terms of % reduction in HbA1c in one study. Costs per patient measured in one other study.
- Didactic interventions focusing on the acquisition of knowledge and information demonstrated positive effects on knowledge, but mixed results on glycemic control and BP, and no effect on weight.
- Collaborative interventions focusing on knowledge tended to demonstrate positive effects on glycemic control in the short term, with mixed results in follow-up > 1 year. Effects on lipids, weight, and BP were mixed.
- Factors other than knowledge needed. Suggestion that …
o changes in attitude and motivation, and a
o minimum threshold of knowledge are required.
- Group versus individual therapy – both demonstrated mixed results for interventions focusing on knowledge, lifestyle, or skills.
- Important elements from previous reviews
o Social learning theory
o Behavior change strategies more effective than didactic methods
o Education most effective when combined with health care provider medication adjustment and reinforcement of educational message.
o Non-complex, individualized to patient’s lifestyle, reinforced over time, respect individual habits and routines, include social support.
o Collaborative problem definition, targeting, goal setting, and planning (Von Korff et al.)
o A continuum of self-management training and support services (Von Korff et al.)
o Active and sustained follow-up (Von Korff et al.).
o Psycho-educational programming (Wagner et al.)
o Respond to individual needs, readiness to change, and self-efficacy (Wagner et al.).
o Individualization, relevance, feedback, reinforcement, and facilitation.
- Limitations
o Study quality
▪ Lack of blinding of patients and assessor
▪ High attrition
▪ Contamination of the control group
▪ Unintended co-interventions
▪ Lack of detail on allocation concealment
▪ Response bias on self-reported outcomes
▪ Un-validated instruments.
▪ Comparison group = basic care and education
o Inadequate description of study interventions and participants
o Generalizability limited, as well, by volunteer nature of study populations.
o Unable to determine which intervention theories are most advantageous.
o No NRSs included.
o Emphasis on efficacy, excluding adoption, reach, and institutionalization, which may affect effectiveness.
- Objectives of the ideal self-management intervention
o Practical and feasible in a variety of settings
o Population willing to participate
o Effective for long-term important outcomes
o Patients satisfied.
o Low cost and cost-effective.
Norris SL, Lau J, Smith SJ, Schmid CH, Engelgau MM. Self-management education for adults with type 2 diabetes: a meta-analysis of the effect on glycemic control. Diabetes Care, 25(7): 2002.
- Health People 2010 – increase to 60%, fro the 1998 baseline level of 45%, the proportion of individuals with diabetes who receive formal diabetes education (Healthy People 2010).
- Large body of literature on diabetes education and its efficacy – however, techniques have evolved and shifted from didactic presentations to interventions involving patient “empowerment”, with participation and collaboration.
- Objective: review reports of published RCTs to ascertaint he efficacy of DSME in adults with type 2 education, provide summary measures of its effect on HbA1c, and identify predictors of effect.
- 1980 tp 1999.
- Studies
o RCTs
o Type 2 diabetes, in any setting.
o Able to examine the effect of the educational component separately. Comparison group could be basic intervention or no intervention, so long as it was mirrored for the intervention group.
- DSME covariates
o Content foci – knowledge or information, lifestyle behaviors, skill development, and coping skills.
o Knowledge-focused studies sub-divided – didactic or collaborative
▪ Didactic – patient attended to the information but did not interact with the instructor or participate actively in teaching sessions.
▪ Collaborative – patient participated actively (e.g.: group discussions), or explicit mention of empowerment, individualized goal-setting, or modeling.
▪ All other content foci considered to be necessarily collaborative.
o Individual vs group education
o Use of SMBG
o Number of educator contacts
o Total contact time
o Time frame of intervention delivery
o Personnel delivering the intervention
o Computer-assisted instruction
o Health care system characteristics – interface with PCP documented.
o Setting
- Studies stratified by follow-up interval – outcomes diminish over time – during or immediately after intervention, 1-3 months from intervention end, or >= 4 months from intervention end.
- Outcome = mean difference in change scores.
- Analysis – random-effects models, Der-Simonian and Laird – imputed before-after correlation to calculate standard errors of change scores – r = 0.50.
- 72 RCTs – 40 examined HbA1c – 37 were included – 3731 participants.
- Results
o Heterogeneity significant for the immediate FU interval.
o Difference in HbA1c improvement, where negative values = increase in HbA1c.
▪ Immediate = 0.76% [0.34, 1.18]
▪ 1-3 months = 0.26% [-0.21, 0.73]
▪ >= 4 months = 0.26% [0.05, 0.48]
o Meta-regression
▪ Total contact time – HbA1c further reduced by 0.04% [0.01, 0.08] per additional hour, over a range of 1 to 28 hours of contact.
▪ On average, 24 hours of contact are needed for a 1% reduction in HbA1c.
▪ No other covariates were significantly associated with the outcome.
- DSME is effective
- Effect diminishes over time, and is improved more by interventions with more contact time.
- A variety of teaching techniques may be effective.
Quality improvement and primary care – Case management
Norris SL, Nichols PJ, Caspersen CJ, Glasgow RE, Engelgau MM, Jack L, Isham G, Snyder SR, Garande-Kulis VG, Garfield S, Briss P, McCulloch D. The effectiveness of disease and case management for people with diabetes. A systematic review. American Journal of Preventive Medicine, 22(4 Suppl): 2002.
- Traditional health care delivery
o Providers react to patient-initiated complaints.
o Care is fragmented, disorganized, duplicative, and focused on managing established disease and complications
o Provider practice what they know.
o Goals generally short term
o Management is provider-directed, provider-centered.
- Objective – examine the extent and quality of the evidence of effectiveness for disease and case management when applied to people with diabetes.
- Community Guide SR.
o Logic model – Disease management = organized, proactive, population-based, integrated care. Case management overlaps with disease management. DM/CM affects health care delivery system, provider KAB, and patient KAB. These three constructs interact. Patient KAB leads to short-term outcomes, which then lead to long-term outcomes.
o Studies – primary comparative (including before and after studies) investigations providing information on one or more outcomes of interest, meeting minimum quality standards. Exclusion – studies with inadequate descriptive information of the study population or intervention characteristics
- Disease management – an organized, proactive, multi-component approach to healthcare delivery that involve all members of a population with a specific disease entity. Care is focused and integrated – across spectrum of disease, across prevention and treatment, and across delivery system silos. Inclusion criteria – intervention involves:
o Identification of a population at risk
o Guidelines / performance standards identified
o Management
o Tracking and monitoring through information systems
o Additional interventions may be incorporated
- Case management – Assignment of authority to a professional who is not the provider of direct health care, but who oversees and is responsible for coordinating and implementing care.
o Identification of eligible patients
o Assessment
o Development of an individual care plan
o Implementation of the care plan
o Monitoring of outcomes
- Results – Disease management
o 27 studies
o HbA1c improved in 18 of 19 studies, median net change = -0.5%, IQR [-1.35, -0.1]
o Improvement in percentage of providers performing annual HbA1c monitoring and retinopathy screening – strong evidence (median +9%).
o Improvement in screening for foot lesions, peripheral neuropathy, lipid levels, and proteinuria – sufficient evidence (median +9% to +24%, approx).
o Other patient outcomes – weight, BMI, BP, lipid levels – inconsistent evidence.
o Economic – 2 studies – Additional $42 to $84 adjusted to the community guide reference case, for increased rates of process outcomes, and improved BP and visual acuity in one study – other study involved pregnant women.
- Results – Case management
o 15 studies
o HbA1c – case management with disease management and without disease management – sufficient evidence of improvement. Median effect = -0.53% IQR [-0.65, -0.46]
o Process outcomes – provider monitoring of HbA1c – sufficient evidence for case management with disease management – insufficient for case management alone.
o Lipid levels, weight, BMI, BP – insufficient evidence.
o Economic studies – none.
- Barriers identified in substrate studies and evaluated as important by the SR team.
o Organizational
▪ Leadership deficiency
▪ Financial resource unavailability
▪ Lack of practice guidelines
o Providers – switch to proactive, organized management requires the redesign of much of the practice and approach to patient care
▪ Appointment and follow-up scheduling
▪ Allocation of clinic time to review registries and CPGs
▪ Delineation of the roles of support staff and providers
▪ Delegation of care
▪ Team organization
▪ Use of planned visits and patient reminders
▪ Provider perception of time-consumming, and uncomfortable-ness with information systems.
▪ CPG barriers (from other sources, not explicated by SR substrate studies).
• Lack of awareness or familiarity
• Disagreement
• Lack of confidence that patient outcomes can be improved
• Inertia of previous practice
• External barriers, like inconvenience, and insufficient time
• See Cabbana et al., JAMA.
▪ Identification of patients
o Patients
▪ Difficulties in maintaining major behavioral change.
▪ Complexity of self-management required.
Davidson MB. How our current medical care system fails people with diabetes. Diabetes Care, 32(2): 2009.
- Diabetes care gaps
o Only half of US patients meet glycemic goal - NHANES.
o In other studies …
▪ 22-46% of patients have HbA1c > 9.5%
▪ 22-46% met LDL goal
▪ 29-33% met the BP goal
▪ Only 2-10% met the combined glycemia, lipd, and BP goals.
- Lack of time major issue
o One approach effective – using specially trained nurses of pharmacists, with authority to make medication changes according to approved algorithms – impressive results compared to CM in which nurses do not have prescriptive authority.
o Critical factor – timely and appropriate clinical decision are made.
o Appropriate – based on algorithms.
o Timely – more time can be spent with patients, outside of the usual 15 minutes every 3 months physician visit – stabilization will happen within 1 month.
o If this is correct, then replacing infrequent face-to-face time with more frequent tele-health communications could also be effective.
o Home glucose monitoring may be faxed to a nurse, who can decide on insulin dose changes – baseline A1c levels averageing 8.4% fell by 0.7% in 20 weeks (Davidson MB, Lewis G. Diabetes Care, 23: 2000). However, the business case could not be made (no billable code for this activity) and it was dropped.
o Many other examples can be found, of web-based monitoring successes. The key is conjectured to be a case manager making treatment decision in real time, based on published literature – case management studies without this role tend to be ineffective.
- Need – more frequent interactions with knowledgeable providers – timely and appropriate clinical decisions need to be made – perhaps by specially trained, appropriately supervised nurses and pharmacists with prescription authority based on approved treatment algorithms.
Davidson MB. The effectiveness of nurse- and pharmacist-directed care in diabetes disease management: a narrative review. Current Diabetes Reviews, 3(4): 2007.
- Most diabetic patients do not meet recommended goals.
- HbA1c will be used here as the primary outcome measure by which to judge the effectiveness of attempts to improve diabetes care – prevent or delay the microvascular complications of diabetes – most available measure.
- Most approaches ineffective or inconsistently effective – according to HbA1c (The review of Renders et al. found that few studies at the time had measured HbA1c)
o Patient appointment reminders
o Patient information feedback to physicians
o Case management – without independent treatment decisions
o Physician education
o Multifaceted interventions in the practice setting.
- Limitations of providing patient information
o (Lobach DF, Hammond WE. Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med, 102: 1997)
o 6 month RCT, 30 clinicians analyzed.
o 8 process measured agreed upon by physician group – prompt page attached to charts at time of patient visit – the decision support system resulted in a two-fold increase in clinician compliance (p = 0.001) – median compliance measures performed over those noted as due – 32.0% vs 15.6%.
o However, measures due were ordered only 1/3 of the time – despite statistically significant difference, overall compliance rate low.
o Reasons
▪ Time constraints
▪ Over-whelming amount of other clinical information to process
▪ Insufficient time to document an intervention performed outside of the practice
▪ Intervention potentially painful or dangerous (PPV)
▪ Recommendations not considered appropriate
o But overall guideline adherence higher – proportion of all recommendations performed – median 65% vs 41%.
- One approach consistently shown to be effective – specially trained nurses and pharmacists given the authority to make independent treatment decisions which follow approved protocols.
o Over 14 nurse controlled studies tabled, others mentioned.
o Over 5 pharmacist controlled studies tabled, others mentioned.
o Almost all showed significant improvements in the intervention vs control group.
o Weighted average – change in HbA1c = -2.2% vs -0.7% – baseline usually near 10%, weighted average – HbA1c lowered approximately 3-fold more!
o Several non-glycemic outcome measures also observed to improve, significantly
▪ LDL
▪ Trig
▪ BP
▪ Diabetic retinopathy in one study!
- In one study in two academically affiliated VA medical centers …
o A diabetes case management program in which NP med changes required physician approval showed no changes in average HbA1c – exit HbA1c = 9.3% vs 9.2%, p = 0.65, although satisfaction with providers was improved = 92% vs 64% (p = 0.04)
▪ (Krein SL, lamerus ML, Vijan S, Lee JL, Fitzgerald JT, Pawlow A, Reeves P, Hayward RA. Am J Med, 116(11): 2004).
▪ RCT
▪ 246 veterans at 2 sites, followed for 18 months
▪ NP case managers – telephone contacts, collaborative goal setting, and treatment algorithms – med change recommendations forwarded to PCP. Targeted self-management, reminders for recommended screening, help with scheduling appointment, monitoring home glucose and BP, identify and initiate med changes.
▪ Barriers
• Patients selected were those with persistently poor control – may have suffered poor social circumstances and support. E.g.: Difficulty contacting patients for FU due to “disruptive” living situations, over 70% of attempted phone contacts unsuccessful at one site.
• Competing demands – patient agenda – repeatedly surfacing theme in semi-structured patient interviews.
• Case managers reported some PCPs unresponsive to contact attempts, possibly due to overloaded schedules or unrealized intentions – unable to approve changes.
o However, a clinical pharmacist with independent decision-making abilities – improved HbA1c levels significantly over 12-24 months – reduction in HbA1c = 2.1% vs 0.9%, p = 0.03 – 3 of 5 process measure conducted significantly more frequently = LDL, retinal exam, monofilament foot screening – study took place in the same setting.
▪ RCT, 80 patients
▪ Pharmacist – evaluation and modification of drug therapy, self-management diabetes education, reinforcement of complications screening via clinic visits and telephone follow-up.
▪ (Choe HM, Mitrovich S, Dubay D, Hayward RA, Krein SL, Vijan S. Am J Manag Care, 11(4): 2005)
- Traditional visits are short and infrequent (3 months in between)
- Also, clinical inertia – HbA1c levels when therapy is added on, dosages increased, or switched are far above ADA guideline levels – patients spend a substantial amount of time with high HbA1c levels.
- Additional staff following detailed treatment algorithms circumvent both of these barriers. These providers have more time, and only have to focus on diabetes and related issues. Timely, appropriate clinical decisions are made on an ongoing basis, leading to improved outcomes.
Quality improvement in primary care – General reviews of QI strategies
Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ, 153(10): 1995.
- Broad definition of CME – ways by which physician learning and clinical practice might be altered by educational or persuasive means.
- Current to 1993
- Studies – involved health care providers, objective outcome assessment, random or quasi-randomized assignment.
- Quality assessment – custom checklist.
- 102 trials
- Results
o Educational materials only – most trials failed to demonstrate changes.
o Conferences those during which no explicit effort was made to determine practice needs of to facilitate practice changed failed to demonstrate change.
o Outreach visits – Reduced inappropriate prescribing, and, to a lesser extent, increasing preventive services – small increases, e.g.: 5 to 27%.
o Local opinion leaders – Range of effects.
o Patient-mediated interventions - Improvements, especially when combined with physician education.
o Audit, feedback, and reminders – results vary across a wide range – nil to moderate.
o Marketing – Unable to separate market research impact from counter-detailing.
o Multifaceted interventions consistently demonstrate changes in professional performance – less so for patient outcomes.
o Local consensus processes – importance unclear
- Because of variations in interventions / settings / study designs and inadequacies in reporting, it is not possible to compare of make definitive conclusions about the effects of specific types of interventions.
- Still, some interventions have been shown to work – others, unexamined, or found wanting.
- Analogy to pharmacotherapy – there are no wonder drugs - several drugs may be needed, along with lifestyle or environment changes.
- Rational drug prescribing analogy also implies appropriate diagnostic strategies to select interventions and monitor their effects.
- “There are no ‘magic bullets’ for improving the quality of health care. There are, however, a wide range of interventions available that, if used appropriately, could lead to substantial improvements”.
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O’Brien MA. Changing provider behavior: an overview of systematic review of interventions. Medical Care, 39(8 Suppl 2): 2001.
- Objective – Use systematic methods to identify, appraise, and summarize SRs of professional education or quality assurance interventions to improve quality of care.
- Current to 1998.
- 41 SRs.
- Variable quality using 9 criteria, with a summary score of 1-7 – median score = 4 (Oxman AD. Checklists for review articles. BMJ, 309: 1994) – worthwhile issues included failure to report inclusion criteria adequately, failure to avoid bias in selection of studies, failure to report criteria to assess study validity, unit of analysis errors, inappropriate meta-analysis given substantial heterogeneity.
- SRs of broad strategies – mostly CME, and programs to ehance the dissemination and implementation of guidelines.
o Most outreach interventions effective
o Lomas – Little evidence that passive dissemination alone resulted in provider behavior change (Lomas J. Annual Review of Public Health, 12: 1991).
o Compliance to CPGs lower for recommendations that were more complex, less trialable (Grilli R, Lomas J. Med Care, 32: 1994).
o Guidelines can change clinical practice – especially if they took account of local circumstances, were disseminated by active educational interventions, and were implemented by patient-specific reminders (Bulletin) (Effective Health Care, 1(8): 1994).
o Oxman review – 102 interventions (Oxman AD, Thomson MA, Davis DA, Haynes RB. CMAJ, 153: 1995).
▪ Dissemination activities – little or no change
▪ More complex interventions –effective – but usually only moderate effects.
o Multifaceted interventions tended to be more effective for introducing guidelines in primary care settings.
- SRs of interventions to improve specific behaviors – e.g.: vaccinations.
o Most studies show significant improvements in processes of care – but few studies report significant improvements in patient outcomes, even when measured.
- SRs of particular interventions
o Dissemination of education materials – no significant improvements.
o Educational outreach visit – Most studies observed improvements in care – small to moderate size.
o Local opinion leaders – mixed results.
o Audit and feedback – mixed results.
o Computer based decision support systems – Might enhance clinical performance for most aspects of care, but not diagnosis.
o Computerized systems – aspects of performance improved, but consultation time may be lengthened (Sullivan F, Mitchell E. BMJ, 331: 1995).
o CQI – before and after studies – effective, but RCTs – no effect.
- Conclusions
o Passive dissemination generally ineffective – may be useful for awareness.
o Active approaches
▪ Audit and feedback, use of local opinion leaders – variable.
▪ Education outreach and reminders – generally effective.
▪ Multifaceted interventions based on assessment of potential barriers – more likely to be effective.
o Guideline implementation may be more effective in hospital settings than in primary care.
DL – Note that many of these studies used may have used a vote-counting method of synthesis, which does not account for magnitude or precision of estimates. Reporting the median and IQR of estimates provides more information, and may prevent inappropriate dismissal of interventions … it is most likely that even passive interventions have an important awareness role to play. (Grimshaw J, McAuley LM, Bero LA, Grilli R, Oxman AD, Ramsay C, Vale L, Zwarenstein M. Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care, 12(4): 2003.)
Renders CM, Valk GD, Griffin SJ, Wagner EH, Eijk Van JT, Assendelft WJ. Interventiosn to improve the management of diabetes in primary care, outpatient, and community settings: a systematic review. Diabetes Care, 24(10): 2001.
- Shifting of diabetes care from hospitals to primary care (Goyder EC, McNally PG, Drucquer M, Spiers N, Botha JL. BMJ, 316: 1998).
- Primary care management can be as good or better than hospital outpatient care if regular review of patients is guaranteed.
- However, compliance with national guidelines for management inadequate in primary care.
- Wide range of interventions implemented.
- Objective – Determine the effectiveness of interventions targeted at health care professionals and/or the structure of care in order to improve the management of patients with diabetes in primary care, outpatient, and community settings.
- EPOC SR, no quantitative meta-analysis.
- 41 studies, 27 of which were RCTs, and 12 of which were CBAs – current through 2000.
- Methodologic limitations – inadequate concealment, randomization within a clinic or practice (contamination risk), lack of similar baseline measurements, outcomes not usually blinded or objective, attrition problems in most studies, and unit of allocation problems in 18 studies.
- Almost all studies involved postgraduate education.
- Only 15 of 41 studies reported both patient outcomes and process measures.
- Results – Professional interventions vs UC
o 12 studies
o Clinician education, reminders, audit and feedback.
o Improved provision of care in all studies that showed deficient care at baseline.
o Patient outcomes – rarely assessed. Improveds described for glycemic control and foot lesions (3 studies), although only 1 of 3 achieved statistical significance.
- Results – Organizational interventions versus UC
o 9 studies
o Most common intervention – computerized tracking or nurse involvement in contacting patients for follow-up.
o Studies (2 studies) in which a nurse or pharmacist assumed part of the physician’s role and provided diabetes care in combination with a patient-oriented intervention – small, beneficial effect on glycemic control – but poor quality.
o Arrangements for follow-up (e.g.: computer tracking, nurse telephone contacts without assuming physician responsibilities) improved processes of care.
o Effectiveness of follow-up arrangements on patient outcomes rarely assessed.
o 2 studies – multidisciplinary teams with patient education and arrangements for follow-up – HbA1c and cholesterol improved significantly – these studies considered not to be professional interventions – no physician oriented component – supplementary management clinic.
- Results – Combined professional and organization interventions vs UC
o 20 studies – complex intervention – combo.
o Combo interventions were commonly medical record systems changes for arranging follow-up, audit and feedback, and/or reminders.
o Computerized reminders, audit and feedback, or both improved process measures – but mixed results in only 2 studies of patient outcomes.
o Centrally organized computerized database – associated with improvements in process measures.
o Patient outcome studies – those that featured a greater involvement of nurses reported positive effects on outcomes.
o Studies that reported a positive effect on patient outcomes often included patient education.
- Multifaceted professional interventions and organizational interventions that facilitate structured and regular review of patients were effective in improving the process of care.
- The addition of patient education, and the enhancement of the role of nurses, led to improvements in patient outcomes, as well as processes of care.
Foy R, Eccles MP, Jamtvedt G, Young J, Grimshaw JM, Baker R. What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Services Research, 5: 2005.
- By examining interventions in a range of settings and circumstances, reviews aim to produce generalisable messages about the effectiveness of these interventions.
- Question – Can we operationalize audit and feedback for improving the quality of diabetes primary care from existing review data?
- Audit and feedback
o Cochrane SR definition – The provision of any summary of clinical performance over a specified period of time.
o NICE document, Principles for Best Practice in Clinical Audit, provides more specific principles for audit and feedback, including creating the right organizational structures and culture of success.
- But how should audit and feedback be performed?
o Does audit and feedback work for diabetes in primary care?
o Does it work equally across all dimensions of care?
o How should it be prepared? Anonymized? Benchmarked?
o How intensive should feedback be? Who should receive feedback (individuals or groups)? Frequency and length of feedback? Continuation of intervention over time?
o How should it be delivered?
o What activities should accompany feedback?
o What should be done about the poorest performers?
- SR, 85 RCTs included, across different diseases (Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Library (2): 2004).
o Audit and feedback can improve professional practice – but effects small to moderate.
o Effectiveness varies substantially among studies.
o Variation may be related to different methods of audit and feedback, or contextual factors.
o Only 5 head-to-head comparisons of different methods of providing feedback.
o No economic evaluation.
o 14 studies of audit and feedback alone vs multifaceted interventions with audit and feedback – no evidence that combination interventions worked better.
- Diabetes care results
o 4 studies, 3 in primary care.
o 2 studies – effectiveness for diabetes care – Audit and feedback, with or without other interventions, was more effective that usual care.
o 2 studies – partially addressed 3 questions about how to conduct audit and feedback.
▪ No difference between continuing feedback and withdrawal of feedback in accuracy of capillary BG monitoring
▪ No small benefit of feedback given by doctor or nurse compared with feedback alone.
o No relationship between study effect size and
▪ Feedback intensity
▪ Co-intervention use
▪ Complexity of targeted behavior
- Review evidence – limited use in informing the operationalisation of evidence based audit and feedback.
o Lack of direct evidence
o Few studies, potential confounding of indirect comparisons.
o Heterogeneity
o Problems with sub-group interpretations in the full review (end up comparing across diseases, etc.)
- Inadequate description of intervention and understanding of causal mechanism – unable to assess applicability to a particular service setting.
o Adopt a conceptual framework – enable identification of features of systematic influence – generalizable concepts that can be used across contexts.
o E.g.: If perceived peer pressure was predictive of adherence to CPGs, then feedback incorporating peer comparison might enhance effectiveness.
o Reduces necessity of multiple direct or pragmatic trials.
Quality improvement in primary care – Effectiveness of the Chronic Care Model
Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the chronic care model in the new millennium. Health Affairs, 28(1): 2009.
- SR of studies evaluating a CCM-based intervention – based on the CCM = integrate changes that involved most, or all, of the six areas of the model – community-based in primary care.
- 2000 to 2008 – since the CCM was developed.
- 82 articles
o 14 reviews
o 21 studies correlating organizational characteristics and QI
o 11 RCTs
o 6 cost or cost-effectiveness studies
o 30 quasi-experimental or observational evaluations.
- CCM implementation – RAND – 3 quarters of practices sustained CCM changes 1 year later – 51 organizations participating in a RAND study (Cretin S, Shortell SM, Keeler EB. Evaluation Review, 28(1): 2004).
- Improved care? CCM implementation significantly improves at least some process and outcome measures compared to controls, in before-and-after evaluations, and in observational CBAs (e.g.: community health center evaluations of the Health Disparities Collaboratives of the Health Resources and Services Administration).
- Is a complex, multicomponent model really necessary?
o CCM designed to build on interrelationships between the 6 pillars.
o RAND evaluation – practice teams implement some elements with more ease – information systems received the most attention, and community linkages, the least.
o Several observational studies have found that composite measures of CCM implementation were significantly associated with higher quality of care.
o Although simpler interventions would be attractive, observational studies suggest that high-performing practices make changes across multiple elements of the CCM.
▪ HbA1c and 10-year risk of CVD – UKPDS risk engine – Assessment of Chronic Care Illness survey – 20 primary care clinics – 30 patients sampled per clinic – 1 point increase in ACIC associated with a 16% relative decrease in risk – absolute 1—year CVD risk was 16.2% – Parchman ML et al., Medical Care, 42(12): 2007.
▪ Average scores on the 6 HEDIS diabetes measures – cross-sectional study – survey returned by 90 MCOs – Top quartile differed from bottom quartile – many univariate differences – e.g.: compuer-gererated reminders, physician champions, etc. – practitioner input and use of clinical-guidelines software independent predictors of composite diabetes quality scores after adjustment for structural and geographic variables, namely NPO status and number of beneficiaries – Fleming B, et al.. American Journal of Managed Care, 10(12): 2004.
- Is the CCM cost-effective?
o Some evidence that it may reduce costs.
o However, cost savings take time to materialize.
o CCM costs money in the short term.
o Evidence on cost-effectiveness is just beginning to emerge.
- The CCM should continue to inform systematic efforts to improve care, and those efforts should be rigorously evaluated.
Quality improvement in primary care – Improving research relevance
Davis H, Nutley S, Walter I. How knowledge translation is misconceived for applied social research. Journal of Health Service Research & Policy, 13(3): 2008.
- What is “evidence”?
o Hierarchy of epidemiologic evidence – synthesis = technical task largely free of values or judgment.
o Practical knowledge – Rycroft-Malone’s “context sensitive” knowledge.
o Critical knowledge – strongly theoretical, intended to destabilize established problem framings and ways of thinking.
- What are sources of evidence
o Academic specialists
o Local evaluators, auditors, or management exercises.
o Practitioners
- Synthesis – unlikely to be acontextual, value-free.
o “Knowledge transfer” and “knowledge translation” suggest gathering, integrating, condensing convergent knowledge into a neat package for dissemination.
o Such a view belies the challenges of synthesizing any but the most simple and incontrovertible of findings.
- Complexities of research use
o Instrumental uses.
o Symbolic uses, and other uses.
o Continual process, in which interpersonal and social interactions are key to accessing and interpreting research knowledge.
o A highly contingent process. An on-going, creative, and unfolding process, rather than a clearly delineated event.
- Knowledge transfer
o Rational-linear models of research use.
o Three sorts of activities
▪ Knowledge push
▪ Knowledge pull
▪ Linkage and exchange
- Suggested terms – “knowledge interaction”, or “knowledge intermediation”.
Rycroft-Malone J, Seers K, Titchen A, Harvey G, Kitson A, McCormack B. What counts as evidence in evidence-based practice? Journal of Advanced Nursing, 47(1): 2004.
- The nature of evidence
o OED – serving to indicate or attest; indication, sign, facts making for a conclusion.
o Unequivocal understanding of evidence infrequent (see, legal evidence).
o Evidence should be independently observed and verified.
- Clinical evidence
o Evidence-based medicine – Sackett – “The conscientious, explicit and judicious use of current best evidence about the care of individual patients”.
o Evidence assumed to be research evidence – quantitative hierarchy.
o Powerful perspective given
▪ Gaps in effectiveness and efficiency.
▪ Need to decrease costs.
▪ Managerialism and accountability.
o Relative neglect of other forms of evidence.
- Alternative definition – evidence = “knowledge derived from a variety of sources that has been subjected to testing and has been found to be credible.”
o What is knowledge? Tripartite definition – justified, true belief.
o Alternatively, “an awareness or familiarity gained by experience”
o Propositional vs non-propositional.
▪ Propositional – formal, explicit, concerned with generalizability.
▪ Non-propositional – informal, implicit, derived through practice.
▪ Non-propositional may become propositional knowledge once it has been articulated, then debated, contested, and verified – theory generation.
o Practitioners must draw on both.
- Alternative sources of evidence captured
o Research evidence
▪ In truth, this is less than objective
• Under-determination of theory by observations
• Affirming the consequent
• Duhem-Quine thesis – requirement of auxillary hypotheses.
• Modus tollens
• Theory-laden nature of observations
▪ Contested uses of research evidence
▪ Simply “pushing out” research evidence is unlikely to lead to use.
▪ Must paticularise, and engage with the context of application.
o Clinical experience
▪ Embedded and accrued in practice, may be tacit and intuitie.
▪ Assumption that these sources are idiosyncratic and subject to bias.
▪ Authors argue that such knowledge, however, is required for individualized evidence-based health care.
▪ This requires affirmed experience – explication, dissemination, and evaluation.
▪ Research evidence is more powerful when it matches clinical experience – a mismatch will lead to variable use in practice.
▪ Improving practice requires integrating new knowledge into practitioners’ existing knowledge frameworks.
o Patients, clients, and carers
▪ How does technological information fit into the context of the world of the person?
▪ E.g.: Prostate cancer, family history, affecting Steve Hrudey’s decision to act aggressively, and forego watchful waiting.
▪ E.g.: Policy decision not to fund beta-interferon (modest benefit outweighted by high cost) drew public outcry – did not engage (as oppose to dismiss) individuals’ positive experiences of using the drug.
▪ Must acknowledge individual values and personal experiences as sources of knowledge that inform the evidence base of practice.
▪ Two types of usable evidence
• Specific to the practitioner-patient encounter
• General body of composite stories
o Local context – E.g.: Local audit data, or PDSA findings.
▪ Has it been systematically collected?
- Must integrate all sources – experiential and scientific – to inform decision making.
o Conflicts
o Ensuring that each source of evidence is robust – discipline specific methods exist.
- In reality, practitioners draw on multiple sources of knowledge.
- The delivery of effective, evidence-based patient-centered care will only be realized when a broader definition of what counts as evidence is embraced.
Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annual Review of Public Health, 28: 2007.
- Barriers to translation and the importance of context – emphasis on those proximal to knowledge use in public health
o Intervention
▪ Efficacious interventions often intensive and demanding
• Costs
• Time demands
• Expertise required
• Learning curve
• Intervention not packaged, “manualized”, or adaptable.
• Unsustainable
• Specific to particular settings and populations
• Lack of process data to disentangle necessary components.
▪ Need to emphasize “minimum intensity” required for change.
▪ Less intensive, more extensive public health approaches, in the vein of Geoffrey Rose, may have value – however, these may be discouraging.
o Target settings
▪ Characteristics that may be barriers
• Competing demands
• External impositions (resentment or gaming if program is imposed)
• Financial relations (e.g.: reimbursement policies)
• Specific client needs
• Limitations – resources, time, organization support
• Unable to implement intervention as intended.
▪ Time is central – some settings find themselves unable to allocate “ one minute for prevention”.
o Research design
▪ Design issues
• Not relevant or representative (external validity) – e.g.: Inclusion of women and minority participants.
• Failure to evaluate cost, reach, setting adoption, implementation, maintenance, or sustainability.
o Interactions between the 3 categories
▪ Examples
• Staffing pattern does not match intervention needs
• Because of setting participation barriers, program reach is low.
• Intervention not appropriate for target population
• Knowledge or intervention not seen as appropriate.
• Unable to implement as intended, leading to failure and abandonment.
▪ Lack of “fit” between intervention and setting, or congruence between information provided and information valued, are major paths to low adoption and implementation.
o Other barriers – e.g.: lack of funds, social expectations.
- CBPR – may enhance the relevance and effectiveness of interventions
o Collaborative partnership – involves partners in all aspects of the research process.
o Builds expertise and capacity in the community – may make a lasting impact, beyond the program at hand.
- Addressing research design factors – Concepts of evidence – must broaden – address a wider range of relevant questions in a rigorous fashion – Develop knowledge according to needs, not needs according to knowledge – Assess multiple aspects of program theory and effectiveness.
o Many types of evidence. Need work on integration – current discussion has tended to degrade into paradigm wars.
o Researchers should deploy mixed methods – mutually beneficial.
o Examples of successful mixed method research projects provided.
- Addressing setting factors – Context and external validity
o External validity = inferences about the extent to which a causal relationship holds over variations in persons, settings, treatments, and outcomes.”
o Information needed
▪ Program reach and sample representativeness
▪ Program or policy implementation and adaptation
▪ Outcomes for decision-making
▪ Maintenance and institutionalization
- Addressing research design factors – Practical trials – adjust research design to encourage attention to contextual factors.
o Purpose – provide information that will make health research more relevant.
o Heterogeneity is encouraged and purposeful, rather than minimized, as is typical in efficacy research.
o Practical trials reflect more of the complexity and context of the real world.
o Key attributes
▪ Answer questions of key stakeholders
▪ Assess multiple and relevant outcomes, including process outcomes and costs
▪ Compare clinically meaningful treatment alternatives
▪ Recruit a diverse, heterogeneous sample – evaluate robustness across key subgroups.
▪ Include multiple settings and interventions
▪ Attend to particular issues of importance, such as training, patient preferences, and algorithms for intervention tailoring.
- Addressing intervention factors – Connectedness of intervention strategies across levels of influence
o Behavior change is complex, has multiple determinants (see PHCHC), and requires long-term follow-up.
o Practices require community linkages and redesign to provide effective interventions.
o The 5 As model – Emphasizes the importance of collaborative goal setting (Agree), and identification of specific behaviors to be adopted (Advise).
▪ Assess
▪ Advise
▪ Agree
▪ Assist – identify barriers, problem solving techniques, and environmental support.
▪ Arrange – specify follow-up plan – require community supports.
o Chronic Care Model – Essential elements of a system that delivers high-quality chronic disease care
▪ Health system / organizational support
▪ Self-management support
▪ Delivery system design
▪ Decision support
▪ Clinical information systems
▪ Community resources
- When do we have enough evidence to translate? Highly context-specific judgment required – not all evidence should be translated (See Graham and Tetroe).
- To enhance integration of research and practice, we need to change how we perform and report research.
o Greater attention to context and external validity
o Partnership with target audiences
o Greater use of practical trials
o Use of systems-based models for conceptualizing change and translation
o Iterative, interactive approach to translation.
Lau, D. Examples where qualitative information contributed to interpretation of trial results.
- Much is to be learnt from the conclusions of semi-structured interviews following failed interventions. However, these data are often badly analyzed, and left to the discussion section of efficacy studies.
- E.g.: Regenstrief system – escape key – too many reminders
o Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. effects on physician compliance. Med Care. 1986;24(8):659-66.
o Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003 Dec;18(12):967-76.
o Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005 Apr;40(2):477-97.
- E.g.: Disappointing results – no improvement in influenza vaccination rates after administration of a paper-based self-assessment / provider reminder (A/R) tool – due to large number of vaccinations (8) prompted – Providers unwilling, or did not have the time, to consider all eight vaccinations.
o 238. Fishbein DB, Willis BC, Cassidy WM, Marioneaux D, Winston CA. A comprehensive patient assessment and physician reminder tool for adult immunization: Effect on vaccine administration. Vaccine. 2006;24(18):3971-83.
- E.g.: Nurse case management intervention in which nurses were unable to adjust medications independently – ineffective because of time constraints on physicians.
o Lobach DF, Hammond WE. Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med, 102: 1997
Quality improvement in primary care – The role of behavioral theory in research
Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. Journal of Clinical Epidemiology, 58(2): 2005.
- Context
o About 30% to 40% of patients do not receive care according to current scientific evidence – about 20% to 25% of care provided is not needed or is potentially harmful (Schuster M, McGlynn E, Brook RH. How good is the quality of health care in the United States? Milbank Q, 76: 1998).
o Implementation research = scientific study of methods to promote the uptake of research findings, including the study of influences on healthcare professionals’ behavior.
- Problem
o Interventions designed to improve implementation – moderate improvements with considerable variation – consistent finding of reviews of implementation research.
o UK MRC framework – need to establish the theoretical basis of interventions.
o Analogy to drug evaluation
▪ Pre-clinical = theory
▪ Phase I = modeling
▪ Phase II = exploratory trial
▪ Phase III = Definitive RCT
▪ Phase IV = long-term implementation
o However, implementation studies are built on investigator interpretation of other empirical studies – like exploring the role of an antihypertensive drug …
▪ Without understanding the pharmacology of the drug, or the pathophysiology of HTN; and
▪ Without phase I trials in animal models or healthy human volunteers.
o Generalizability problems – we do not know what might influence the effectiveness of different interventions.
o Unable to guide the choice or adaptation of complex interventions in practice.
- Theory
o Assumes that clinical practice = a form of human behavior – may be described by behavioral theories.
o Theory = A coherent and non-contradictory set of statements, concepts, or ideas that organizes, predicts and explains phenomena.
o In only 10% of studies of guideline implementation strategies did authors refer to an explicit theory.
o Levels of theory
▪ Individual health professional
▪ Health care groups / teams
▪ Organizations providing health care
▪ Health care system or environment
o Types of individual or group behavioral theories
▪ Motivational (How individuals come to intend to change behavior, with intentions taken as a good predictor of behavior)
▪ Action (Predictors of behavior in those already motivated)
▪ Stage (Orderly progression through discrete stages towards change)
o Choice of theory
▪ Demonstrated effectiveness in explanation, prediction, and control.
▪ Explain behavior in terms of actionable variables – e.g.: age is purely descriptive, cannot be changed.
▪ Must include non-volitional components – e.g.: perceptions of external factors.
o Using theory
▪ Explain variations in clinical practice as functions of theory-based constructs or processes – potential targets for intervention.
▪ Develop and test interventions targeting theoretical constructs
- Theory-based factors underling clinical practice
o Constructs, to measures, to instruments, and actual behavior.
o E.g.: Studies that identify belies with the strongest relationship to behavioral intention.
- Designing interventions
o Select a technology and a delivery method – e.g.: audit and feedback, clinician reminders. Then select and implement a measurement model
o Modeling studies with interim outcomes, e.g.: behavioral intention are useful.
- Use of theory-based measures in the context of a trial provides a peek into the “black box” of understanding why a trial intervention has or has not worked.
- Theory-driven checklists may assist implementers in matching important characteristics of their situation and needs to the available evidence.
- Implementation research lives in a context where clinicians and decision-makers may erroneously believe that they already know what is best to do – a coherent approach to implementation interventions will avoid ineffective quick fixes, and ensure that implementation advances the wider discipline of implementation research.
Eccles MP, Grimshaw JM, Johnston M, Steen N, Pitts, NB, Thomas R, Glidewell E, Maclennan G, Bonetti D, Walker A. Applying psychological theories to evidence-based clinical practice: Identifying factors predictive of managing upper respiratory tract infections without antibiotics. Implementation Science, 2: 2007.
- When implementation studies are reviewed, the lack of any common underlying framework means that they provide little detailed information to guide the choice, or optimize the components, of complex interventions when introduced into routine care settings.
- A generalizable framework can minimize the number of costly pragmatic implementation trials, and will identify the “active ingredients” in interventions.
- Objective – identify theoretical constructs that predict clinical behavior, behavioral simulation, and behavioral intention, with respect to managing upper respiratory tract infections without antibiotics.
- Design – Cross-sectional study
- Data source
o Theory-based cognitions – postal survey
o Behavior – rate of antibiotic prescribing rates per patient as a proxy for managing URTIs without antibiotics – Administrative prescribing data
▪ Potentially relevant antibiotics prescriptions (i.e.: not all antibiotics, only those likely to be prescribed for URTI)
▪ Measured for the 6 months before cognitions survey, and the 6 months after – behavior over 1 year.
o Behavioral simulation = 5 clinical scenarios – postal survey.
o Behavioral intentions – 3 questions – 7-point scale – summed = intention to prescribe score.
- Theories – theory-based measures developed from semi-structure interviews with 14 GPs
o Theory of Planned Behavior
o Social Cognitive Theory
o Operant Learning Theory
o Implementation Intentions
o Common Sense Self-Regulation Model
o Stage Models
- 230 GPs
- Behavior, intention, and simulation were significantly correlated with each other, r ranging from 0.17 to 0.44, with highest r for the relation between behavioral simulation and behavioral intention – as it turns out, only “small” correlations were demonstrated between intention and behavior, and simulation and behavior.
- Results – behavior
o Theory-level analysis – TPB R2 = 3%, SCT R2 = 5%, OLT R2 = 6%
o Cross-theory multiple regression – only evidence of habitual behavior (OLT) was retained - R2 = 6%.
- Results – behavioral simulation
o Theory-level analysis – TPB R2 = = 31%, II R2 = 6%, OLT R2 = 24%, knowledge R2 = 4.5%. Stage model predictive of differences in simulation score.
o Cross-theory analysis – TPB PBC (strongest beta), evidence of habitual behavior, cause (chance/bad luck), and behavioral intention – R2 = 36%.
- Results – behavioral intention
o Theory level analysis – TPB R2 = 30%, SCT R2 = 29%, CS-SRM R2 = 27%, II R2 = 9%, OLT R2 = 43%, knowledge and attitudes R2 = 22%. Stage model predictive of intention score.
o Cross-theory regression – Evidence of habitual behavior (highest beta), TPB attitudes, risk perception, CS-SRM control by doctor, TPB PBC, and CS-SRM control by treatment retained – R2 = 49%.
- Knowledge was related not to behavior, but to simulation an intention – but did not enter any of the cross-theory regressions – other constructs are consistently more important – changing knowledge alone unlikely to be effective.
- Main constructs were habit, with additional influence from control, attitudes, and risk perceptions.
- GPs have considered this frequently performed behavior (results from stage model) and operate in a predominantly habitual manner backed up by beliefs that support their habit.
- Intention and simulation may be useful proxy indicators for behavior – however, the correlation was weak.
- Stepwise decrease in variance explained by models from intention to simulation to behavior. Higher variance of behavior explained has been seen in studies of relating intention and behavior in health care professionals, suggesting that it is not the case that these theories simply do not apply to GP behaviors – rather, there are likely problems with prescriptions as a proxy for the behavior.
- Psychological models can be useful in understanding and predicting clinical behavior.
Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annual Review of Public Health, 31: 2010.
- Effective public health programs usually require behavior change at many levels.
o Behavior
o Context
- Interventions with and explicit theoretical foundation are more effective than those lacking a theoretical base – strategies that combine multiple theories and concepts may have larger effects.
- Objective – Overview contemporary behavioral science theory use for development and implementation of public health and health promotion interventions.
- Multiple determinants and multiple levels of health behavior
o Many factors contribute to health behavior patterns – their development, maintenance, and change.
o Ecologic perspective – interventions should not only target the individual, but should also consider and affect interpersonal, organizational, and environmental factors influencing health behavior.
- What is theory?
o Theory – a set of interrelated concepts, definitions, and propositions that explain or predict events or situations by specifying relations among variables.
▪ Description
▪ Prediction
▪ Intervention
o Generality is important – theories are abstract, and not content- or topic-specific (See Levin-Rozalis M. The Canadian Journal of Program Evaluation, 18(2): 2003)
▪ Research findings intended to serve as a base for general laws – widest possible application of the findings.
▪ Defined, general, and abstract variables – which area also less useful for evaluation purposes – however idiographic techniques like ethnography have been developed – but purpose is still the formulation of generalized laws.
o Explanatory theory – e.g.: understanding why an employee smokes.
o Change theory – e.g.: understanding how to change employee smoking.
- Most often used theories – demonstrated by recent review of journal articles – Trans-Theoretical Model, Social Cognitive Theory, and the Health Beliefs Model. Other common theories – Theory of Reasoned Action / Theory of Planned Behavior and self-efficacy theory.
- HBM
o Developed to help understand why people did or did not use preventive services offered by public health departments in the 1950s.
o People’s beliefs influence their readiness to take action
▪ Perceived susceptibility
▪ Perceived severity
▪ Perceived benefits
▪ Perceived barriers
▪ Cues to action
▪ Self-efficacy (more recent)
o Typically applied to health concerns that are asymptomatic and prevention-related – beliefs as or more important than overt symptoms.
- TTM
o Long term changes involve actions and adaptations over time – people are at different “stages” in readiness and change.
o A heuristic model that describes a sequence of steps in successful behavior change
▪ Precontemplation
▪ Contemplation
▪ Preparation
▪ Action
▪ Maintenance
o People do not always move through stages linearly – e.g.: relapse.
o E.g.: Smoking cessation.
- SCT
o Explains human behavior in terms of a three-way, dynamic, reciprocal model in which personal factors, environmental influences, and behavior continually interact.
o People learn not only through their own experiences, but also by observing the actions of others and the results of those actions.
o Reciprocal determinism – a person can be both an agent for change and a responder to change – much in line with social ecological models – changes in environment, examples of role models – may be used to promote healthier behavior.
o Key constructs – observational learning, reinforcement, self-control, self-efficacy.
o This is a basis for interventions that achieve behavior modification with methods to improve self-efficacy.
▪ Goal-setting (Set incremental and achievable goals)
▪ Self-monitoring (Monitor and reinforce – keep records)
▪ Behavioral contracting (Use formalized contracts to establish goals and rewards)
o Self-efficacy = a person’s confidence in his or her ability to take action and to persist in that action despite obstacles or challenges.
- Social ecological model
o Emphasize multiple levels of influence
o Behaviors both shape and are shaped by the social environment
o Creating an environment conducive to change is important to facilitate adoption of healthy behaviors.
- Use of theory
o Task Force on Community Preventive Services SRs have captured theoretical bases of interventions, where reported.
o 11 SRs since 2000 reporting on theory use.
o Most-often used theories
▪ SCT
▪ TTM/stages of change
▪ HBM
▪ TPB
▪ PRECEDE/PROCEED
o Several (7 cited) reviews concluded that interventions based on theory or explicitly describe theoretical constructs were more effective than those not using theory – larger effects.
▪ E.g.: Ammerman et al., 2002 – Dietary fat, fruit and vegetable consumption.
▪ E.g.: Legler et al., 2002 – Mammography promotion in historically underserved groups.
o Why? Unknown – use of theories that fit well? Theory-based strategies developed with greater care, implemented with greater fidelity?
o Few addressed organizational change or provider behavior.
- How has theory been used?
o Painter et al. – continuum of theory use
▪ Informed by theory – Theoretical framework identified.
▪ Applied theory – Several constructs applied.
▪ Tested theory – More than half the constructs measured or tested, or two or more theories compared
▪ Building/creating theory – Theory expanded or revised using constructs.
o Most common use is informing.
o Need more testing, more thorough application.
- Some examples – WISEWOMAN and Minnesota Statewide Health Improvement Program.
o Modest impact, however.
o Use of ecologic models in SHIP – insufficient detail – behavior-specific models may need to be better articulated.
o Use of behavioral psychology theories in WISEWOMAN – interventions may not adequately consider contextual factors.
o Evaluation complicated – mediating influences in the context of communities not adequately recognized, measured, or reported.
o Interventions based on integrated theory of ecological change need to involved community mobilization that goes beyond community advisory board meetings.
o Also need to account for a longer time frame – better implemented in large programs.
- Constructs and issues across theories
o Environmental influences – Social, organizational, and physical environments are determinants of behavior.
o Behavior change as a process, not an event – Multiple actions and adaptations over time.
o Intentions versus action – TTM makes a clear distinctionb tween contemplation and preparation and oert action. TPB also proposes that intentions are the best predictors of behavior – “Implementation intentions”
o Changing behaviors versus maintaining behavior change – require different types of strategies.
- Challenges and unresolved issues
o Selecting the right theory or theories
▪ Identify the problem, goal, and units of practice – start with a logic model, and work towards solutions.
▪ Practitioners – Pragmatic criterion of usefulness and consistency with everyday observations
▪ Researchers – Is the theory borne out?
▪ Test theories iteratively.
o When is a new theory needed? Careful thought may lead instead to the choice of a suitable theory, with adaptations.
o Population-focused programs and individual-focused strategies
▪ Population-focused program – limited value to adopt a program oriented solely towards modifying individuals’ behaviors.
▪ Environmental change should be performed alongside individual skill training.
- Strongest interventions may be built from multiple theories.
- Rigorous tests of theory-based interventions are the building blocks of the evidence base .
- Know the audience – participatory action improves the odds of success.
- Be creative. Interventions must attract and retain interest and enthusiasm.
Godin G, Belanger-Gravel Ariane, Eccles Martin, Grimshaw J. Healthcare professionals’ intentions and behaviours: A systematic review of studies based on social cognitive theories. Implementation Science, 3: 2008.
- Social cognitive theory = theories where individual cognitions/thoughts are viewed as processes intervening between observable stimuli and responses in real world situations.
- Objectives – Quantify the extent to which social cognitive theories …
o Explain intention of health care professionals to adopt clinical behaviors
o Predict health professionals’ clinical behavior.
- Included studies – examined predictive value of clearly specified social cognitive theories for clinician intentions (cross-sectional or prospective) and/or clinical behaviors (prospective studies only) – theories of motivation only.
- Current to 2007.
- Analysis – Frequency weighted pooled R2 for theories; vote counting for individual variables, categorized according to a modified Michie framework; testing of a priori specified effect modifiers
o Type of professional
o Type of behavior (e.g.: prescribing, wearing gloves, guideline compliance)
o Main theory used
o Sample size (>= 150 vs < 150)
o Psychometric qualities (Cronbach’s alpha >= 0.60)
o Measurement method – dependent variable (e.g.: direct observation, self-report)
o Level of correspondence between intention and behavior (theoretical – correspondence of measures in action, target, context, and time).
- 76 included studies (20259 participants)
- Results – Behavior
o Behavior R2
▪ Behavior – range from 0.001 to 0.58
▪ Mean = 0.31 – 15 studies, 2112 participants.
o Most common theory = TRA or its extention, the TPB, in 14 of 15 studies – significantly higher R2 than the single study of OLT (p < 0.0001).
▪ TRA/TPB R2 = 0.35
o Cognitive factors most consistently associated with behavior
▪ Beliefs about capabilities
▪ Intention – R2 = 0.46
▪ Beliefs about consequences – R2 = 0.18
▪ Social influences
▪ Past behavior
▪ Knowledge only assessed in 2 studies, no conclusions drawn (min. >= 3)
o Prediction better for professions other than physicians and nurses
o Prediction of self-reported behavior better than objectively assessed behavior – however, “objective” measures were often proxies, so that intention and behavior did not fully correspond.
o Prediction better when intention appropriate corresponds to behavior
- Results – Intention
o Intention R2
▪ Intention – range from 0.14 to 0.91.
▪ Mean = 0.59 – 64 studies, 14986 participants.
o Most common theories = TRA/TPB, the theory of interpersonal behavior, the technology acceptance model, and others – TIB was best predictor, p < 0.0001.
▪ TIB R2 = 0.81 (3 studies)
▪ TRA/TPB R2 (56 studies) = 0.59
▪ Others – not far behind – approximately 0.47, 0.42.
o Cognitive variables most consistently associated with intention
▪ Beliefs about capabilities
▪ Beliefs about consequences
▪ Moral norm
▪ Social influences
▪ Role and identify
▪ Others
▪ Knowledge assessed in 8 studies, significant in 12.5%, the lowest vote count.
o Prediction best for nurses
o Prediction better with good psychometric values.
- Differences within professionals by type of behavior.
- Accuracy of intention may be reduced by the complexity of clinical-related behaviors – modulated by aspects of the case.
- The TPB is an appropriate theory to predict behavior, whereas Triandis’ theory better capture the dynamic underlying intention, based on variables associated.
- Even though habit did not emerge, it has been added, because its effect should be controlled.
Foy R, Eccles MP, Jamtvedt G, Young J, Grimshaw JM, Baker R. What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Services Research, 5: 2005.
- By examining interventions in a range of settings and circumstances, reviews aim to produce generalisable messages about the effectiveness of these interventions.
- Question – Can we operationalize audit and feedback for improving the quality of diabetes primary care from existing review data?
- Audit and feedback
o Cochrane SR definition – The provision of any summary of clinical performance over a specified period of time.
o NICE document, Principles for Best Practice in Clinical Audit, provides more specific principles for audit and feedback, including creating the right organizational structures and culture of success.
- But how should audit and feedback be performed?
o Does audit and feedback work for diabetes in primary care?
o Does it work equally across all dimensions of care?
o How should it be prepared? Anonymized? Benchmarked?
o How intensive should feedback be? Who should receive feedback (individuals or groups)? Frequency and length of feedback? Continuation of intervention over time?
o How should it be delivered?
o What activities should accompany feedback?
o What should be done about the poorest performers?
- SR, 85 RCTs included, across different diseases (Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Library (2): 2004).
o Audit and feedback can improve professional practice – but effects small to moderate.
o Effectiveness varies substantially among studies.
o Variation may be related to different methods of audit and feedback, or contextual factors.
o Only 5 head-to-head comparisons of different methods of providing feedback.
o No economic evaluation.
o 14 studies of audit and feedback alone vs multifaceted interventions with audit and feedback – no evidence that combination interventions worked better.
- Diabetes care results
o 4 studies, 3 in primary care.
o 2 studies – effectiveness for diabetes care – Audit and feedback, with or without other interventions, was more effective that usual care.
o 2 studies – partially addressed 3 questions about how to conduct audit and feedback.
▪ No difference between continuing feedback and withdrawal of feedback in accuracy of capillary BG monitoring
▪ No small benefit of feedback given by doctor or nurse compared with feedback alone.
o No relationship between study effect size and
▪ Feedback intensity
▪ Co-intervention use
▪ Complexity of targeted behavior
- Review evidence – limited use in informing the operationalisation of evidence based audit and feedback.
o Lack of direct evidence
o Few studies, potential confounding of indirect comparisons.
o Heterogeneity
o Problems with sub-group interpretations in the full review (end up comparing across diseases, etc.)
- Inadequate description of intervention and understanding of causal mechanism – unable to assess applicability to a particular service setting.
o Adopt a conceptual framework – enable identification of features of systematic influence – generalizable concepts that can be used across contexts.
o E.g.: If perceived peer pressure was predictive of adherence to CPGs, then feedback incorporating peer comparison might enhance effectiveness.
o Reduces necessity of multiple direct or pragmatic trials.
Grimshaw JM, Zwarenstein M, Tetroe JM, Godin G, Graham ID, Lemyre L, Eccles MP, Johnston M, Francis J, Hux J, O’Rourke K, Legare F, Presseau J. Looking inside the black box: a theory-based process evaluation alongside a randomized controlled trial of printed educational materials (the Ontario printed educational message, OPEN) to improve referral and prescribing practices in primary care in Ontario, Canada. Implementation Science, 2: 2007.
- TPB constructs chosen due to primary hypothesis that OPEM intervention causes changes in physicians’ intentions due to improved attitudes or subjective norms, with little or no change in perceived behavioral control.
- Surveys with Dillman method to measure TPB constructs, pre- and post-intervention.
- Instrument development
o Behavioral intention – specified to correspond TACTfully (target, action, context, and time) with the behavior of interest.
o Attitude measures – common stem – “For me, [behavior] would be” – bipolar adjectives on a 7-point scale – “good practice … bad practice”, “necessary … unnecessary”, “satisfying … not satisfying”
o Subjective norms – 7 point Likert response format – “Most people who are important to me think that [behavior]”.
o PBC – Difficulty and controllability – “Doing the [behavior] is difficult for me” and “There are factors outside my control that prevent me from [behavior]”
- Interpretations – Improvements in behaviors and the theory-based constructs – Educational materials may have changed behavior through the hypothesized mediators.
- Interpretations – Improvements in behaviors, but not in hypothesized mediators.
o Other mediating mechanisms.
o Theoretical measures used insensitive predictors of behavioral change (e.g.: by resulting in data with limited variance – floor or ceiling effects)
o Post-intentional factors important (TPB = motivational, not action, theory)
o Selection-bias – those responding to TPB construct surveys are dissimilar from those in whom behavior was measured.
- No improvement in behaviors, improvements in mediators
o Changes in mediators insufficient to alter behavior (threshold hypothesis)
o Post-intentional factors (e.g.: environmental barriers) moderated the effects of the intervention (intention-behavior gap hypothesis).
o Selection bias
- No improvement in behavior or mediating constructs
o Intervention ineffective. If baseline measures of TPB constructs are high, then barriers to practice were not KAB-intention related, and an educational intervention may have been inappropriate to begin with.
DL – The protocol published by Grimshaw et al. (2007) can be used to demonstrate the knowledge that is missing if only effectiveness is examined – look at the broader range of hypotheses that may be distinguished by crossing effectiveness results with those of a TPB construct survey!
Quality improvement in primary care – Research designs
Eccles M, Grimshaw J, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and improvement strategies. Quality and Safety in Health Care, 12(!): 2003.
- Randomized designs
o Gold standard method for health care interventions.
o Randomization ensures that known and unknown factors that may independently affect outcomes are likely to be distributed evenly between trial groups.
o Cluster randomization
▪ Individual patient randomized trials may not be ideal – treatment given to control patients may be affected by the organization or professional’s experience applying the intervention to those in the experimental group.
▪ Randomize providers instead – but collect data at the individual patient level.
o Level of randomization – Need to ensure adequate numbers of clusters – the balance of randomization is a large numbers effect.
o Baseline measurement – check adequacy of allocation, estimate extent of problem to begin with (e.g.: exclude ceiling effects), stratification factor for effect modification or for matching, which increase power – but possible Hawthorne effects , i.e.: sensitization.
o Sample size
▪ Must be adjusted for dependence within clusters.
▪ Extra number of patients required more efficiently obtained by increasing number of clusters, as opposed to patients per cluster.
▪ Little additional power is gained from increasing number of patients per cluster above 50.
o Analysis
▪ Cluster level – summary statistic per cluster.
▪ Standard tests – adjust patient-level statistics for correlation.
▪ Advanced hierarchcical methods – model inherent correlation explicitly.
o Two arm trials
o Multiple arm trials – allow head-to-head comparisons of interventions or levels of intervention under similar circumstances – but loss of statistical power – require sample size increases compared to two arm trials.
o Factorial designs – Evaluate the relative effectiveness of more than one intervention vs control.
▪ Independent analyses to estimate the effect of interventions separately – allows conduct of two randomized trials for the same sample size as a two arm trial.
▪ However, power is diminished by interaction effects.
o Balanced incomplete block designs
▪ A randomized complete block design occurs when all treatments are randomly assigned to each individual within a block.
▪ Incomplete – cannot fit all treatments in each block
▪ Balanced – each pair of treatments occur together to similar extents across blocks.
▪ E.g.: Two intervention trial with different outcomes, each intervention group acting as a control group for the other intervention – removes non-specific effects of being under observation, or non-causal effects of treatment (e.g.: negative demotivation effects in usual care control subjects, positive attention effects in treatment subjects).
- Non-randomized designs – quasi-experimental
o Different from quasi-randomized, which refers simply to methods of randomization that generate pseudo, as opposed to true, patters of allocation.
o Quasi-experimental designs are useful where there are political, practical, or ethical barriers to conducting a genuine experiment.
▪ Researchers have little control over delivery
▪ Must plan an evaluation.
o Before and after study, uncontrolled.
▪ Superior to observational designs (less confounding by site or participant dissimilarities), but are vulnerable to secular trends.
▪ Lipsey and Wilson – overview of meta-analyses – separate pooling for controlled and uncontrolled studies – observed effects from uncontrolled before and after studies were greater.
▪ Avoid this design! Interpret cautiously!
o Controlled before and after study
▪ Identify a control population with similar characteristics to the study population and collect data in both populations before and after the intervention is applied.
▪ Protect against secular trends.
▪ But difficulty identifying a comparable control group.
▪ May address with “within group” analyses (e.g.: comparison of change scores between groups) – but may be inappropriate
• If baseline imbalances exist, suggest that the control is not truly comparable – differences in improvement may be spurious.
o Time series designs
▪ Detect whether an intervention has an effect significantly greater than the underlying secular trend.
▪ Useful when it is difficult to randomize or identify an appropriate control group.
▪ Data collected at multiple time points before and after intervention.
• Before intervention time-points allow estimation of the underlying trend and cyclical effects.
• May then adjust these effects out of the after-intervention data.
▪ Need sufficient time-points before intervention – stable estimation of trends.
▪ Autocorrelation – when data points collected close in time are likely to be more similar to each other than to data points collected far apart – require special methods.
▪ Does not protect against changes concomitant with intervention.
- A range of research designs is available for studies evaluating the effectiveness of change and improvement strategies.
- Design chosen should maximize internal validity and generalizability within the contextual constraints of the intervention.
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- university of minnesota college of education
- university of minnesota school of social work
- wharton school of the university of pennsylvania
- cost of university of scranton
- university of minnesota school of education
- university of scranton cost of attendance
- university of south florida college of medicine
- university of minnesota masters of social work
- ecampus of university of phoenix
- university of minnesota college of continuing education
- university of illinois college of nursing
- university of north texas college of nursing