INTRODUCTION



Managed Care and Vulnerable Populations:

Adults with Serious Mental Illness

Core Paper 1: Sample Survey Component

A Grant Funded Program Sponsored by:

CSAT CSAP CMHS

SAMHSA

|CONTRIBUTORS |

|OREGON |Bentson H. McFarland, M.D. Ph.D. |

|Department of Psychiatry |Principal Investigator |

|Oregon Health & Science University | |

|Portland, OR |Jacqueline Bianconi, MS |

| |Lori Danker, RN |

| |Jo Mahler, MS |

|FLORIDA |David L. Shern, Ph.D. |

|Florida Mental Health Institute |Principal Investigator |

|University of South Florida | |

|Tampa, FL |Roger Boothroyd, Ph.D. |

| |Julienne Giard,M.S.W. |

| |Mary Murrin, M.A. |

| |Susan Ridgely |

| |Patricia Robinson |

| |Kristen Snyder |

| |Paul Stiles, J.D., Ph.D. |

|HAWAII |A. Michael Wylie, Ph.D. |

|Department of Psychology |Principal Investigator |

|University of Hawaii at Manoa | |

|Honolulu, HI |Jeffery H. Nathan, Ph.D. |

|PENNSYLVANIA |Aileen Rothbard, Sc.D. |

|University of Pennsylvania |Principal Investigator |

|Center for Mental Health Policy and Services Research | |

|Philadelphia, PA |Jeffrey Draine, PhD. |

| | |

|VIRGINIA |Joseph P. Morrissey, Ph.D. |

|Cecil G. Sheps Center for Health Services Research |Principal Investigator |

|University of North Carolina at Chapel Hill | |

|Chapel Hill, North Carolina |Scott Stroup, M.D., MPH |

|and |Elizabeth Merwin, Ph.D. |

|Southeastern Rural Mental Health Research Center |Yasar Ozcan, Ph.D. |

|University of Virginia | |

|Charlottesville, Virginia | |

|COORDINATING CENTER |H. Stephen Leff, Ph.D. |

|Human Services Research Institute |Study Leader |

|Cambridge, MA | |

| |Mathew Hoover |

| |Barbara Raab |

| |Dow Wieman, Ph.D. |

TABLE OF CONTENTS

I. INTRODUCTION 1

II. BACKGROUND 2

Managed Behavioral Health in the Public Sector 2

Hypothesized Effects of Managed Care 4

III. LITERATURE REVIEW 5

Evaluations of Public Sector Managed Behavioral Health Care 5

IV. METHODS 9

Testing the Managed Care Hypotheses 9

Equivalence Analysis 11

Statistical Difference and Equivalence 12

Conceptualization of Managed Care 13

Conceptualization of Time 14

Sites and Programs 14

Subjects 17

Subject Selection Procedures 17

Subject Characteristics 19

Attrition 22

The Managed Care Interview 22

Background Variables 24

Impact Variables 25

V. STATISTICAL METHODS 29

Adjustment for Covariates 29

Difference Analysis. 29

Equivalence Analysis 31

Possible Differences Attributable to Models of Managed Care 32

Possible Differences Attributable to Time 32

Homogeneity 32

VI. RESULTS AND DISCUSSION 33

Difference-Equivalence Findings by Domain 36

Homogeneity 39

Summary Of Difference-Equivalence Relationships 39

Selective Enrollment and Retention 41

Subgroup Analyses 42

Changes Over Time 43

VII. SUMMARY 44

Study overview 44

Summary of support for managed care hypotheses 45

Further Research 48

VIII. REFERENCES 50

APPENDIX A: THE MANAGED CARE INTERVIEW

APPENDIX B: COMMON SERVICE CATEGORIES

APPENDIX C: MEASURE PROFILES

APPENDIX D: COVARIATE CORRELATIONS WITH STUDY DOMAINS

APPENDIX E: SUB-GROUP DIFFERENCE-EQUIVALENCE ANALYSES

INTRODUCTION

Managed care refers to a set of strategies for controlling service utilization. The most notable of these is capitation, a system of financing whereby providers receive a fixed payment to provide care as needed for an enrolled population. The alternative to capitation is fee for service financing, whereby providers are reimbursed for delivering service on the basis of how much they provide. This paper describes a multi-site study that compares managed and fee for service programs for delivering public sector behavioral health care to adult persons with serious mental illness at five locations throughout the United States.

The study, known as the Adults with Severe Mental Illness (SMI) Study, is part of a larger study, entitled the Managed Care and Vulnerable Populations Evaluation Project. The large study encompasses a total of 21 sites in 10 states includes and includes, in addition to adults with SMI, three additional target populations: adults with chemical dependency, adolescents with substance abuse disorders, and children and adolescents with severe emotional disorders, (Coordinating Center for Managed Care and Vulnerable Populations Evaluation Project 1998).

The Adult SMI study comprises three major components: 1) a sample survey study of service use, quality, outcomes, and satisfaction for samples of individuals enrolled in managed care and fee for service groups, based on data from two consumer interviews six months apart data; 2) a population data study focusing on service use and costs as determined from claims and encounter data for all Medicaid recipients in the target service areas prior to, and after the introduction of managed care; and 3) a study to develop a taxonomy of managed care organizations, focusing on the managed care strategies and organization arrangements in each of the study sites.

This paper presents findings from the sample survey component of the Adult SMI study. Findings include the numbers of persons receiving services, the types and amounts of mental health, psychosocial, alcohol and other drug (AOD) and health services received, the types of medication received, service quality, and consumer outcomes and satisfaction. A subsequent paper will present findings from administrative and claims data from the prospective study.

BACKGROUND

Managed Behavioral Health in the Public Sector

The transition from fee for service to managed care in the public sector occurred relatively recently, largely since the mid-1990's after it was nearly complete in private insurance. The process began with states seeking to control Medicaid costs, which had been rising rapidly in the 1980's, by making use of the Health Care Financing Administration (HCFA) "waiver" regulations to implement managed care programs for general health care. Within a few years nearly every state operated some form of Medicaid managed care program (Lewin Group 1998). Between 1991 and 1998, the number of Medicaid recipients enrolled in managed care programs increased six-fold, from 2.7 to 16.6 million (Kaiser Commission on Medicaid and the Uninsured 1999).

Initially, states implementing Medicaid managed care programs tended to retain mental health benefits under fee for service arrangements, especially for beneficiaries whose eligibility was based on disability (i.e. recipients of Social Security Disability Insurance) (Callahan, Shepard et al. 1995). As the states gained experience managing general health care for the Aid to Families with Dependent Children/Temporary Assistance to Needy Families (AFDC/TANF), however, they began to manage mental health benefits and to enroll disabled in managed care programs. Typically these were structured as carve-out programs, i.e. specialty programs established to manage mental health and substance abuse benefits separately from general health care (Frank, Huskamp et al. 1996).

These programs may be operated by private for-profit managed care organizations (MCO's), by public agencies at the state, county or even city level, or by some combination of private for-profit and public non-profit organization. In the study described here, two of the five sites represent for-profit MCO's, two represent non-profit managed care programs, and one is a mixed model, with a non-profit organization sub-contracting with a for-profit MCO.

In other respects managed care programs are heterogeneous, with much variation in characteristics such as relationship to general health care benefits (e.g. integrated versus carve-out), techniques of utilization management, manner and extent of risk sharing, specifics of benefits packages, and provisions of the contracts between public agencies and private MCO's, and between MCO's and providers (Gold and Hurley 1997; Lewin Group 1998; Rosenbaum, Shin et al. 1998; Rosenthal 1999). This heterogeneity characterizes the managed care programs included in the present study as well (Mulkern 1999)

The states have implemented these programs in advance of much empirical evidence concerning their effects (Mechanic, Schlesinger and McAlpine, 1995). HCFA requires, as a condition of receiving a waiver, that states obtain an independent evaluation, and the health services research community, including federal funding agencies and private foundations has demonstrated considerable interest in these policy initiatives. The pace of implementation, however, combined with the considerable challenges for research design has resulted in findings beyond the level of case studies being relatively sparse to date, as indicated below (Leff and Woocher 1998). The multi-site study described in this paper will provide the most comprehensive data on public sector managed care to date.

Hypothesized Effects of Managed Care

Managed care developed as a response to rising health care costs. Proponents of managed care assert that it contains cost while improving service access, quality, and outcomes. Detractors assert that the financial incentives in capitated managed care lead to poorer quality of care. These assertions may be construed as hypotheses. Thus, we refer to the assertion that managed care controls costs while improving care as the panacea hypothesis. The countervailing view that it results in poorer care across the board is the perverse incentive hypothesis. Other hypotheses are possible, as well. One is that managed care has no effect on quality and outcomes due to the distance between changes in organization and financing and provider practices (the no difference hypothesis). Another is that managed care effects vary for different impact areas or subgroups (the mixed effect hypothesis) due to the complex patterns of financial and clinical risk associated with different subpopulations. One version of the mixed effects hypothesis, tested here, is that the panacea hypothesis applies to persons who are more severely disordered, while the perverse incentive or no difference hypotheses might apply to less-disordered individuals (Leff, Lieberman et al. 1996). The rationale for this prediction is that the needs of the most severely ill are so evident and so pressing that even a minimally adequate system will be compelled to respond appropriately, whereas the less apparent needs of less severely ill consumers may be neglected in a substandard system of care.

Although these hypotheses have been tested by some outcomes research and evaluation (Mechanic, Schlesinger et al. 1995; Leff, Lieberman et al. 1996), policy makers have relied more upon qualitative guides based on summaries of state activities and case studies produced by government agencies and policy institutes (National Academy for State Health Policy 1997; Rosenbaum, Shin et al. 1998; Substance Abuse and Mental Health Services Administration 1998), as well as anecdotal evidence and the experience of the consulting firms that often serve functions such as readiness assessment, actuarial projections, program design, and contracting (Bailit and Burgess 1999).

LITERATURE REVIEW

Evaluations of Public Sector Managed Behavioral Health Care

Leff, Wieman, and Woocher conducted a systematic literature search for reports on evaluations of public sector managed behavioral healthcare programs. From the results of this search, they included quantitative studies involving comparative analyses of public-sector behavioral health programs for adults with serious mental illness (Leff and Woocher 1998; Leff and Wieman 2000). The search produced 20 published and two unpublished articles that passed our inclusion and exclusion criteria. A recent search by Grazier and Eselius (1999) similar to this, but limited to carve-out programs and broadened to include private sector plans, produced similar results. The 22 reports were the products of 15 separate studies, involving eight different managed care plans. (Findings cited in multiple reports from a single research project are combined in the following summary.) The reported studies involved eight managed care plans in the following states: Arizona, California, Colorado, Massachusetts, Minnesota, New York, Wisconsin and Utah. These plans were implemented between 1987 (3 plans) to 1995 (1 plan). Thus, they represent a relatively early phase in the development of public sector managed care, demonstrating one of the obstacles to knowledge application: the lag time to publication.

Table 1 presents characteristics of the studies reviewed. One-third employed randomization, with the remaining ten relying on quasi-experimental or pre-experimental designs. The largest number employed claims data (n=9) or other administrative data (n=5). The predominance of this type of data is likely because it is the most accessible, and also because it lends itself to the cost estimates and projections that are a major interest of policy makers and managed care organizations. Likewise, of the domains assessed, utilization and cost predominate (eleven and nine studies respectively). All fifteen of the reports involved comparison between managed care and fee for service systems; none compared different types of managed care organization. Two-thirds reported having applied some form of risk adjustment in the data analysis.

|Table 1. Characteristics of studies reviewed |

|Characteristics of Studies |Number |

| |of Studies |

| |(N=15) |

|Research Design | |

| Experimental |5 |

| Cross-sectional/Cohort Study |5 |

| Nonequivalent Comparison Groups |4 |

| Single Group Pre-Post |1 |

|Source of Data | |

| Claims/Encounter |9 |

| Other Administrative Records |5 |

| Consumer |5 |

| Clinician/Staff |4 |

| Medical Records |2 |

| Family |2 |

| Provider |1 |

|Impacts/Effects Measured | |

| Service Utilization |11 |

| Costs |9 |

| Quality |7 |

| Access |5 |

| Consumer Functioning |5 |

| Consumer Symptoms |4 |

| Consumer Satisfaction |2 |

| Consumer Health Status |2 |

| Family Experiences |2 |

| Other |2 |

| Provider Satisfaction |1 |

|Risk/Case Mix Adjustment? | |

| Yes |10 |

| No |5 |

Table 2 summarizes findings from the fifteen studies reviewed here. This summary shows only whether the effect of managed care is statistically significant and if so, in which direction. It is evident that the most consistent finding is decreased utilization and cost of inpatient services, identified in eight of the 11 studies examining this variable. This finding reflects the primary goal of most managed care programs and the number of studies examining this variable reflects the primary interest of most policy makers. With respect to the four hypotheses cited above (panacea, perverse incentive, no difference and mixed effect), the findings of decreased hospital utilization are inconclusive since the studies report no data related to need for treatment or outcomes.

|Table 2. Summary of findings from 15 studies of public sector managed behavioral health care programs |

| | |Effect of Managed Care (Number of Studies) |

|Domain |

| | |Impact Area |

| |

| |

| |

|Hypothesis |

| |

|Site |

|2 Some capitated, others fee for service |

|3 Risk pool created from four percent withhold |

|4 State contracts with non-profit MBHO, which subcontracts with for-profit for service delivery |

Subjects

Persons in managed care and fee for service programs were interviewed at two times, six months apart. Samples were identified and recruited at each of the sites as described below. Table 5 shows the numbers of persons and percent of the total interviewed at each site, by program at Time 1.

| |

|Table 5. Number of persons and percent of multi-site total interviewed |

|in managed care and fee for service programs at each site (Time 1) |

|Site | |Managed Care | |Fee for Service |

| | |N (%) | |N (%) |

|FL (HMO) | |264 (22.7) | | |187 (16.2) | |

|FL (Carve-out) | |237 (20.4) | | |N/A1 | |

|HI | |207 (17.8) | | |356 (30.8) | |

|OR | |139 (11.9) | | |166 (14.4) | |

|PA | |140 (12.4) | | |138 (12.0) | |

|VA | |176 (15.1) | | |308 (26.7) | |

|Total | |1163 (100) | | |1155 (100) | |

|1Florida site includes two managed care study groups with one fee-for-service |

|comparison group |

Subject Selection Procedures

None of the study sites randomly assigned subjects to the managed care and fee for service condition. One of the sites drew a stratified random sample from the pool of eligible persons in managed care and fee for service. In one site, all subjects in managed care were contacted to participate in the study; all who consented were enrolled; and a matched sample was drawn from persons in fee for service. Three of the sites obtained convenience samples by interviewing persons from lists of eligible subjects until they obtained a requisite number. The following is a more detailed description of these procedures.

Florida: Investigators mailed a preliminary survey to a stratified random sample of approximately 7,500 Medicaid recipients, ages 21 to 64, both users and non-users of services, selected from the state's Medicaid eligibility file as of April, 1997. Respondents to the survey who were enrolled in either of the two managed care plans or the fee for service program and who indicated that they receive SSI because of a severe mental illness were contacted by a field interviewer who explained the study, obtained consent and conducted the baseline interview.

Hawaii: Investigators received information about potential managed care and fee for service subjects from the Department of Human Services. This information included name, gender, date of birth, diagnosis, contact information, and date of most recent mental health service utilization. Persons eligible for the study were those with a serious mental illness who were in active treatment. Research assistants contacted all persons in managed care individuals by telephone if possible, or else by mail, or finally through a designated care provider, to explain the study. All those in managed care who agreed to participate were scheduled for an appointment to obtain consent and conduct the baseline interview. A group of fee for service recipients was matched with the managed care subjects according to diagnosis, ethnicity, age and gender.

Oregon: The state mental health agency provided investigators with a random sample of clients (i.e. persons with serious mental illness) in counties with managed care systems and those with fee for service. This list was linked to a database of persons who were Medicaid eligible for the year of 1995. From this sampling frame, investigators chose a random sample, stratified on ethnicity. Minorities were sampled with a sampling fraction of one, and Caucasians with a sampling fraction chosen to attain the requisite sample size. Investigators interviewed persons from this list until a targeted number of interviews was obtained.

Pennsylvania: Subjects were Medicaid-eligible mental health service consumers with SMI between the ages of 18 and 64 in either managed care or fee for service, drawn from two sampling frames. One consisted of persons receiving outpatient services in selected settings, but with no hospitalization within the previous year. The second consisted of consumers with SMI recruited in specified inpatient units. Persons were recruited on a convenience basis until adequate numbers of subjects were interviewed.

Virginia: The study consisted of subjects selected from two sample frames. The first is an outpatient frame, consisting of persons with serious mental illness who were Medicaid-only eligible (i.e. not with Medicare and well as Medicaid) and between the ages of 18 and 64, randomly selected from a list of eligible clients supplied by public sector Community Service Boards (some sub-capitated under MCO's contracting with the state, and others fee for service). The second was an inpatient frame, consisting of persons meeting the same criteria admitted to selected hospitals. Investigators then interviewed persons in the two frames who agreed to participate until they had obtained a convenience sample of acceptable size.

Subject Characteristics

Table 6 shows the demographic and clinical background characteristics of the subjects, by site and program at Time 1. The last column in this table also shows whether, across sites, the MC and FFS were groups were statistically different or not, and equivalent or not (as determined by the method described above) on each of the demographic and clinical variables. Across sites, managed care and fee for service programs differed (different, non-equivalent) on six of the demographic and clinical variables: gender, ever married, have children, number of children at home, percent Hispanic, and age of first emotional problem. Dependent measures were residualized on all demographic and clinical variables that were fixed, i.e. that would not change from Time 1 to Time 2. These were: gender, ever married, now married, education (at least high school or not), race (white or non-white), Hispanic, disabled, and age at first treatment.

|Table 6. Demographic and clinical characteristics of subjects by site and program, with difference-equivalence relationship |

| | |Managed Care | |Fee for Service | | |

|Site |

|2 Florida fee for service repeated to indicate comparisons with both Managed Care Plan 1 and Managed Care Plan 2 |

Attrition

The overall attrition rates from the study were 5 percent for fee for service subjects and 15 percent for managed care subjects. Residualizing dependent measures at time 2 as well as time 1 statistically controlled any between-condition effects of this differential attrition. However, residualization does not address why attrition rates from the study differed under the two conditions. The reasons for attrition reported by the sites included inability to locate/no response, relocation, refusals, and death. The most frequent reason reported was inability to locate/no response. Further analysis of the differential attrition is beyond the scope of this report.

The Managed Care Interview

To develop the interview form for the sample survey component of the common protocol for the five sites, a workgroup including representatives from each site participated in an extensive consensus process. This process provided for standardized data collection to allow for cross-site comparisons, including comparisons with sites focusing on the other target populations, notably adults with chemical dependency. The common interview produced by the consensus process is contained in Appendix A.

The workgroup first identified fifteen broad domains, as shown in Table 7. In developing measures for each domain, whenever possible the workgroup selected an existing instrument with demonstrated reliability and validity (see measure profiles in Appendix C). For measures lacking estimates of test-retest reliability, investigators administered interviews to 20 subjects at each site at two-week intervals. All measures were found reliable. A table of reliability measures generated by this study can be found in Appendix C.

|Table 7. Domains, instruments and measures in the prospective study interview |

|Protocol Domain | |Instruments | |Variables |

|Demographics | |Not applicable | |See Table 6 |

|Ethnocultural identity | |Ethnocultural Identity Behavioral Index | |See Table 6 |

|Clinical history | |Schizophrenia PORT | |See Table 6 |

|Service use | |Not Applicable | | |

|Access | |MHSIP Consumer Oriented Report Card | | |

| | |Consumer Survey | | |

|Appropriateness | |MHSIP Consumer Oriented Report Card | | |

| | |Consumer Survey | | |

|Functioning (health status) | |SF12 | |Physical component summary |

| | | | |Mental Component Summary |

|Mental health symptoms | |Brief Symptom Inventory, Global Severity | |Somatization, obsessive-compulsiveness, |

| | |Index | |interpersonal sensitivity, depression, anxiety, |

| | | | |hostility, phobic anxiety, paranoid ideation, |

| | | | |psychoticism. |

|Problem Severity | |Managed Care Problem Severity Scale | |Disability, age at first problem, age at first |

| | | | |treatment |

|Quality of life | |Lehman Quality of Life Inventory | |Living situation, family, finances, work and |

| | | | |school, legal and safety, health |

|Alcohol and drug use | |Alcohol Severity Index | |16 items related to alcohol and drug use in past |

| | | | |30 days |

|Perception of stigma | |Campbell Perception of Stigma Scale | |Scale consisting of 7 items |

| | | | | |

|Empowerment | |MHSIP Consumer Oriented Report Card | |Full MHSIP survey |

| | |Consumer Survey | | |

|Satisfaction | |MHSIP Consumer Oriented Report Card | | |

| | |Consumer Survey, Consumer Assessment of | | |

| | |Behavioral Health Systems (CABHS) | | |

Background Variables

Demographic Information: In addition to standard demographic items (sex, age, language, religion, marital status, family size, education, employment, race/ethnicity, living situation) the interview included three questions about ethnocultural identity from the Ethnocultural Identity Behavior Index (Horvarth and Marsella 1994) that were the most closely associated with health care and help-seeking behavior, to be asked of persons self-identified as non-white.

Clinical History: Many of the questions used were adapted from the Schizophrenia PORT (Agency for Health Care Policy and Research, Center for Mental Health Services Research, et al. 1995) and from studies conducted by Ruth Ralph (Ralph, Kidder et al. 2000).

Service Types and Amounts: Therapeutic and psychosocial services utilization was measured on the basis of self-report. Subjects were asked if they had received a service in the last three months, the number of times they had received the service, and the average amount of time they received the service. The responses were used to compute penetration rates and amounts of service for the three-month periods. Consumers were queried about services. The services are listed and defined in Appendix B Numbers and types of medications received were also based on self-report in response to open-ended questions. A specific question was asked about medications identified as atypical anti-psychotic medications.

Service Quality: Service quality was measured by means of the Access and Appropriateness Scales of the Mental Statistics Improvement Program (MHSIP) Consumer survey. For the full 40-item MHSIP Consumer-Oriented Report Card Consumer Survey. A combination of exploratory and confirmatory factor analysis has produced a smaller set of variables that could be scored as four scales: Access, Appropriateness, Outcomes, and Overall Satisfaction.

Impact Variables

Symptom Severity: The Brief Symptom Inventory (BSI) (Derogatis 1993) was selected as the primary measure of symptom severity, upon its well-established psychometric properties, widespread use and relative brevity.

A brief version of the SCL-90, the BSI is a 53-item self-report symptom inventory, consisting of nine dimensions and three global indices. The nine symptom dimensions are somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. The BSI also yields a global score, the Global Severity Index (GSI). The GSI is the most sensitive single indicator of a respondent’s distress level. The GSI is calculated by adding the sums of the nine symptom dimensions along with additional items, and dividing this number by the total number of responses.

For an additional measure of symptom severity, the interview also includes several questions about suicidality.

Health Status: To measure health status, the group selected the SF-12 Health Survey (Ware, Kosinski et al. 1996) with some additional questions about serious physical illnesses or disabilities. The SF-12 is a 12-item shortened version of the SF-36 originally developed for the Medical Outcomes Study, widely used as a measure of health outcomes in large studies with multiple populations. The SF-12 incorporates eight concepts (physical functioning, physical role, emotional role, bodily pain, general health, energy/fatigue, social functioning, mental health) and two summary measures, a Physical Component Summary (PCS) and a Mental Component Summary (MCS).

For this report, we use the PCS and the MCS as measures of health status. These are scored using norm-based methods with regression weights derived from the general US population. All scores above and below 50 are said to be above and below the average, respectively. A one-point difference in scores is said to represent a true difference because the standard deviation is set at 10 (Ware, Kosinski et al. 1996). For our study population, scores ranged from 13 to 69 for the PCS and 10 to 70 for the MCS.

Functioning/Level of Care: The section of the Managed Care Interview addressing level of functional status consists of combination of questions from the Cross-Disability Integrated Health Outcome Survey and the Schizophrenia PORT questionnaires (Lehman 1988). For thirteen areas of functioning, the Interview asks whether the respondent needed help and if so, whether he or she received it, how much and from whom.

A Level of Care Scale was constructed using items taken from the Cross Disability Health Outcomes Survey (Evaluation Center HSRI 1996) as well as several questions regarding violence towards self or others. Respondents were assigned a score based on their lowest level of functioning indicated in the protocol. Scores range from one to six with higher scores indicating that the respondent has a lower level/need for care, and higher scores indicating a higher level of functioning.

Problem Severity: The workgroup created an additional symptom severity index based on prior and current emotional/psychiatric problems and treatment. The four items in the scale are summed to obtain a score from zero to four, with zero indicating low severity and four indicating high severity.

Alcohol and Drug Use: To measure consumer alcohol and drug use, the workgroup adapted the Addiction Severity Index (ASI) (McLellan, Luborsky et al. 1980), adding a simple lifetime prevalence question and making some modifications to account for issues specific to persons with severe mental illness. One of these was to ask of persons who acknowledged taking drugs, whether these were prescribed, and if so, how often they took at least one extra dose.

The Managed Care Interview incorporated sixteen items from the Addiction Severity Index (ASI), asking about alcohol and drug use for the past 30 days. From responses to these questions, two ASI composite scores are calculated, one for alcohol and the other for drugs. The composite scores are mathematically computed combinations of items ranging from 0 (no problem) to 1 (extreme problem). Normative data for the ASI is available on a number of populations. It has demonstrated excellent reliability and predictive, concurrent and discriminate validity (Sederer, Dickey et al. 1996).

Quality of Life: To measure quality of life, the workgroup selected items from the Lehman Quality of Life Interview (QOLI) (Lehman 1988). Of the eight domains of quality of life addressed by the QOLI, the managed care interview includes six determined to be particularly relevant to managed care: living situation, family, finances, work and school, legal and safety issues, and health. This variables analyzed in this report are based on the QOLI items with preferable distributional characteristics and least overlap with other variables. Since previous research has demonstrated the reliability of QOLI variables, these were not included in the test-retest analyses.

Satisfaction: Satisfaction was measured using the satisfaction scale from the MHSIP Consumer-oriented Report Card Consumer Survey. Some of these items were modified slightly to broaden the focus to a “mental health plan” rather than a “program.” Additionally, investigators included some questions adapted from the draft of the Development of a Consumer Assessment of Health Plans Survey for Behavioral Health and Substance Abuse (Cleary and Eisen 1998) and several open-ended questions about barriers to receiving needed mental health and substance abuse services. Finally, this section includes a global rating of satisfaction with services on a scale of from zero to ten. As noted above, the MHSIP consumer survey scales have been shown to have acceptable reliability. Acceptable internal consistency reliabilities for the MHSIP have been reported, using data from five states and a total sample of 6,900 persons.

An overall Global Rating ranging from zero to ten was also used as an additional measure of satisfaction.

Perception of Stigma: This domain was designed to obtain information about consumer perceptions of how mental health care providers view recipients of services. These questions were suggested by Ruth Ralph and were originally used in Jean Campbell’s Well-Being Project (Campbell and Schraiber 1989; Ralph, Kidder et al. 2000). The Perception of Stigma Scale is made up of seven items intended to measure how mental health providers perceive recipients of services. Items are reverse scored, and a mean score calculated for each individual. Higher scores indicate more stigma perceived by the recipient. Scores can range from 1 to 6.

STATISTICAL METHODS

Adjustment for Covariates

Because persons were not randomized to the managed care or fee for service conditions within their sites, we attempted to control statistically for possible compositional differences between conditions. For each site, individual outcome scores were residualized on demographics and clinical variables prior to analysis. Separate ordinary least squares regressions, including demographic and clinical history variables, were computed for each site, from which the unstandardized residuals were retained for further analysis.

Correlations between background variables and dependent measures are uniformly low, with only one exceeding 0.2. Although a number were statistically significant at the .05 level, this is probably inflated by the number of tests performed. Appendix D contains tables showing these relationships for time 1. Findings for time 2 were similar.

Difference Analysis.

To test for significant multi-site differences between the impacts of fee-for-service and managed care, after controlling for demographic and clinical differences, we employed a meta-analytic algorithm by DerSimonian and Laird (1986) implemented in the Comprehensive Meta-analysis software developed by Bornestein (1999). “Meta-analysis” refers to the assortment of methods and techniques for quantitatively synthesizing research findings (Lipsey and Wilson 2001). Meta-analysis is usually thought of as a method for combining the results of independently conducted, experimental studies, which in health research are known as randomized clinical trials. However, it can also be used to combine the results of coordinated, multi-site studies, which may be experimental, quasi-experimental, or even simple pre-post designs.

One advantage of both multi-site trials and meta-analysis is that they respond to the problem that individual studies may have small numbers of individuals and therefore low statistical power. Meta-analysis also has the advantage of providing comparable information on group differences within individual sites as well as group differences when studies are combined. This information can be used to judge the wisdom of combining results and to investigate the causes of differences among studies or sites (Lipsey and Wilson 1996). Additionally, meta-analysis references group means and variances, which translate into commonly understood concepts like percentages and averages.

One disadvantage of traditional meta-analysis is that methods have not been established for analyzing time trends in repeated measures, which are frequently collected in multi-site studies, although there is no logical reason why measures such as slopes could not be used as dependent variables.

The DerSimonian and Laird algorithm is a random effects approach. Such approaches assume that interventions of the same type, employed in different sites or studies, even when implemented under highly controlled conditions, may vary in their implementation and outcomes as a function of random factors and that this variation should be included in estimating the effects of a type of intervention. Additionally, DerSimonian and Laird differentially weights site and study results as a function of their sample sizes and variability in a manner referred to as precision weighting. Thus the analytic model for this study considers potential variation or difference between the five sites while simultaneously estimating the between-condition effects. This method yields precision weighted estimates (means) for each of the conditions based on within and between differences and variability at each site and for the overall difference between conditions. The standard value of p ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download