Summaries of Functional Assessment Instruments



Summaries of Functional Assessment Instruments

for Child and Adolescent Outcomes

Child and Adolescent Functional Assessment Scale (CAFAS):

Description: The intended purpose of the CAFAS is to measure day-to-day functioning among children and adolescents with mental health impairment. According to its author, the primary uses of the CAFAS include (a) tracking clinical outcomes for individual children, (b) assigning children to appropriate levels of treatment, service, or care, (c) documentation for assisting in case management activities and the development of treatment plans, and (d) program evaluation. The CAFAS is designed for children 6 to 17 years old. A companion instrument, the PECFAS is designed for children under the age of 6.

The CAFAS is completed by a case manager, clinical staff member, or other professional who is familiar with the child. The CAFAS assesses functioning in eight interest areas or domains:

School/Work Role Performance

Home Role Performance

Community Role Performance

Behavior toward Others

Moods/Emotions

Self-harmful Behavior

Substance Use

Thinking

Behavioral descriptors are provided to assist in assigning an impairment level for the child in each of the domains. Impairment is rated on a four-point scale from Minimal or No Impairment, to Severe Impairment while scores in domains use ten point increments. For example, under the School/Work subscale, a behavioral descriptor corresponding to “severe impairment” includes “Harmed or made serious threat to hurt teacher/peer-coworker/supervisor.” Total scores are obtained from a simple sum of scores for all eight domains. Rule-of-thumb interpretations of severity have been proposed and are available in the training manual.

A relatively new development in the instrument is the inclusion of descriptive phrases for each domain that are offered as strengths or goals statements that may be documented on the CAFAS form. These strengths/goals are proposed for use as part of the treatment planning process for the child. A method of profile analysis to describe individual children, or to disaggregate large quantities of data is now available. Training manuals are available and the author offers opportunities for initial and refresher training. Training is particularly recommended when the instrument is used to assess outcomes or consumer functioning over time.

Psychometrics:

Published research has confirmed the reliability of the CAFAS when proper training is provided. Inter-rater reliability was established in two studies with reliability correlations of .79 to 1.00 for domains and total CAFAS score correlations from .91 to .96. Alpha reliability was established in the results of two studies for the total score ranging from .63 to .78 from intake to18 month follow-up.

Published research has investigated and demonstrated the concurrent validity of the CAFAS by examing scores for children placed at different levels of care. Higher total scores have also been associated with problems with social relationships, involvement in juvenile justice, and school related problems. Predictive validity of the CAFAS has been investigated with the total score predictive of total cost of services, restrictiveness of care, utilization of services, contact with law enforcement, school attendance, and recidivism.

Published research has demonstrated that the CAFAS is sensitive to change with significant reduction in scores from intake to 12 months for children receiving inpatient and outpatient care. The psychometrics available for the instrument as whole do not include the use of the strength/goals statements, which are intended for use as part of the service planning process.

Additional notes: The CAFAS is a proprietary measure that must be purchased from the author. The instrument has been widely used in state mental health systems and has a large body of published literature regarding its development, interpretation, and use. A computer based process for scoring the instrument is available. The instrument includes the capacity to identify CAFAS profiles of youths to visually assess change over time. The CAFAS TIERS offers an ordered arrangement of categorical groupings of youth based on the severity of their functioning. The author has worked in close cooperation in some MH systems in the implementation and analysis of outcome data. In Michigan, data reports are provided to regional mental health entities on a monthly basis and allow them to determine who they are serving, the outcomes of the service, and how they are doing in relation to other regions in the state.

CANS-MH

Description: The intended purpose of the CANS-MH as described by the author is to support communication regarding the child’s service needs and functioning for decision- making purposes. The primary uses of the instrument are for (a) decision support regarding service intensity and residential placement, (b) quality improvement/quality assurance to assess congruence between identified needs and services and for retrospective file reviews, and (c) outcomes monitoring. The CANS-MH is completed by a staff member or professional who is familiar with the child. Training manuals are available. Different versions of the CANS instrument (CANS-MH, CANS-DD, CANS-JJ, CANS-CW) are available for mental health, developmental disabilities, juvenile justice and child welfare populations. While the structure of the measure is similar for each of these populations, domains and individual items vary for each instrument. Age groups have not been identified for the CANS-MH.

The CANS-MH assesses functioning in six interest areas or domains:

Problem Presentation

Risk Behaviors

Functioning

Care Intensity and Organization

Caregiver Capacity

Strengths

Within each domain, from four to ten areas are rated by the clinical professional on a four-point scale for a total of 43 questions. Impairment ranges from “0” reflecting “no evidence” of the issue listed, to “4”, reflecting the “need for immediate or intensive action.” For example, ten dimensions are listed under the domain Problem Presentation; examples include psychosis, depression/anxiety, and oppositional behavior. For oppositional behavior, a rating of “2” would indicate that the child “has moderate problems with compliance to rules or adult instructions.” The rating “3” would indicate that the child “has severe problems with compliance to rules or adult instructions.” The instrument is designed to provide a profile of the needs and strengths of the child. As a result, the instrument is not designed to produce a total score based on any summary of scores on individual dimensions. Ratings are typically based on the last thirty days.

As with the CAFAS, the CANS-MH has been structured to facilitate individual service planning and case management. Training manuals are available.

Psychometrics:

The reliability of the CANS-MH has been assessed in studies comparing raters’ assessments of vignettes and in comparisons of first-hand and chart reviews. Reliability estimates range from .75 to .84 for these studies. Estimated reliabilities of individual items are approximately .70 or higher, with reliabilities of domains approximately .90. Sensitivity to change was assessed in two populations of youth over a six-month period. Effect sizes ranged from .15 (small) to .82 (large) indicating that the instrument may be used to assess outcomes over time.

With regard to the instrument’s validity, the CANS-MH has been found to be significantly correlated (.61) with the CAFAS total score. Individual domain correlations were also high ranging from .54 to .73. A second study conducted using discriminant analysis found that the CANS-MH correctly classified 63% of all cases into three levels of care. In a concurrent validity study the CANS-MH was shown to agree with expert clinical review 81% of the time.

Additional notes: The CANS-MH is available in the public domain. The instrument has been widely used in state mental health systems and has published literature regarding its development, interpretation, and use. The author has a history of working in cooperation with MH systems in the implementation of the instrument.

CALOCUS

Description: The intended purpose of the CALOCUS is to measure impairment in functioning among children and adolescents with an emphasis on the level-of-care needs of the child. The primary uses of the CALOCUS include quantifying the clinical severity and service needs of children with psychiatric disorders, substance use disorders, and developmental disorders with emphasis on initial placement decisions, and determining the level and intensity of services needed. The CALOCUS is completed by a staff member or professional who is familiar with the child. Training manuals are available.

Age groups have not been identified for the CALOCUS.

The CALOCUS assesses functioning in six subscales or dimensions. Two subscales are further divided into two categories. The six subscales include:

Risk of Harm

Functional Status

Co-morbidity

Recovery Environment (Environmental stressors, Environmental supports)

Resiliency and Treatment History

Acceptance and Engagement (Child/adolescent, Parent/caregiver)

An impairment level of from “1” indicating little or no problem in the area being assessed, to “5” indicating a very serious problem in the area being assessed is assigned for each domain. Behavioral descriptors are provided for each level in each of the subscales. For example, for Risk of Harm (dimension I), a rating of “2, some risk of harm” would reflect “past history of fleeting suicidal or homicidal thoughts with no current ideation, plan, or intention, and no significant distress.” Total scores are obtained from a simple sum of scores for all domains. This composite score is used with a decision tree to determine an appropriate level of care or placement from level “0 - Basic services for prevention” to level “6 – Secure, 24 hour psychiatric monitoring.” In addition, to the total score, a high score on any of three of the subscales is also considered indicative of a higher level of care placement.

Psychometrics: Reliability of the CALOCUS was assessed in a study comparing the ratings of vignettes by child psychiatrists and social workers. Intraclass correlations for the participants on CALOCUS domains ranged from .57 to .95 for non-physicians, and from .73 to .93 for physicians. In a study of instrument stability, correlations across three months ranged from .53 to .64 for individual scales, with a correlation of .71 for the total score. In a validity study of the CALOCUS, subscale scores were correlated with the CAFAS composite score from .11 to .52. The composite CALOCUS score was correlated .62 with the CAFAS composite score. In a test of concurrent validity, CALOCUS scores were compared to total service hours, out of home placement, service restrictiveness, and total cost. Generally low correlations from -.10 to .23 were found. No validation of the decision tree or placement process was identified in the review.

CFARS

Description: The intended purpose of the CFARS is to measure the functioning of children and adolescents with an emphasis on the provision of data for performance based planning and accountability. The primary uses of the CFARS include (a) assessing domains relevant for evaluating children, (b) provide information helpful to both clinicians and agencies delivering services, and (c) describe the changing status of children. The CFARS is completed by a staff member or professional who is familiar with the child. Training materials are available. The CFARS is in the public domain and may be used without cost. No lower bound for the age range has been identified for the CFARS.

The CFARS assesses functioning in 16 interest areas or domains:

Depression

Hyperactivity

Cognitive Performance

Traumatic Stress

Interpersonal Relationships

ADL Functioning

Work/School Performance

Danger to Others

Anxiety

Thought Process

Medical/Physical

Substance Use

Behavior in “Home” Setting

Socio-Legal

Danger to Self

Security/Management Needs

Each domain of the instrument is assessed on a scale of 1 to 9, with 1 indicating “No problem” and 9 indicating “Extreme problem.” Behavioral adjectives are provided to assist in assigning an impairment level for the child in each of the domains. For example, under Depression, some of the adjectives listed include depressed mood, sad, irritable, happy, withdrawn, hopeless, and withdrawn. Total scores are obtained from a simple sum of scores of all 16 domains.

Psychometrics: An interrater reliability study of the CFARS demonstrated reliabilities of .5 or greater for fourteen of the sixteen domains. The lowest inter-rater reliabilities were obtained for traumatic stress, and behavior in the home. Validity for the instrument has been addressed by comparing mean severity ratings of children placed in different levels of care. As would be predicted, results suggest that children in more intensive settings scored higher on average than children in less intensive settings.

Additional notes: The CFARS is available at no cost. The instrument has been used in state mental health systems, and the authors have worked in cooperation with MH systems in the implementation of the instrument.

MHSIP: YSS and YSSF

Description: The purpose of the YSS and the YSSF are to obtain feedback from children and adolescents and their parents or caregivers regarding the quality of mental health services they have received. The YSS is completed by the adolescent receiving care, while the YSSF is completed by a family member, caregiver, or parent. Training materials are available for the implementation and use of the instrument.

The YSS/YSSR consists of 21 items assessing five domains related to quality of services. The five domains include:

Access to Services

Participation in Treatment

Cultural Sensitivity

Satisfaction with Services

Perceived Outcomes

Each item on the survey is rated by the respondent from “strongly disagree” to “strongly agree.” A total score and scores on each of the five domains are produced from a simple sum of scores of individual items.

Psychometrics: In a study of children receiving mental health services in one state agency, Cronbach’s alpha reliability estimates for the five domains ranged from .70 to .94 for the YSS and from .72 to .94 for the YSSF. In a study of concurrent validity, caregiver ratings of outcomes were found to correlate with caregiver reports of school absenteeism, arrests, and out of home placements.

Additional notes: The YSS and YSSF are available for use without cost. The instrument has been widely used in state mental health systems for assessing consumer satisfaction with services. Of 43 states and the District of Columbia reporting SAMSHA National Outcome Measures (NOMs) data, 24 utilized the YSS or YSSF to address perception of care.

OHIO Scales (Short Form) Youth Problem, Functioning, and Satisfaction Scales

Description: The purposes of the OHIO SCALES are to obtain feedback from children and adolescents, their parents or caregivers, and agency workers regarding the youth’s problems, level of functioning, and satisfaction with services. The scales provide a parallel form for each respondent with regard to problems and functioning, with the parent/caregiver and youth forms providing additional questions regarding hopefulness, satisfaction with services, and overall quality of life. The scales include 48 items on the youth self-report and caregiver report. The agency worker version of the scale includes 40 questions related to functioning and problems with an additional checklist of items concerning placement settings occupied during the last 90 days.

The areas of interest or domains include:

Problems experienced in the last 90 days

Level of functioning

Hopefulness/Overall well-being (parent and youth forms)

Satisfaction with services (parent and youth forms)

Placement settings in last 90 days (agency worker form)

Items in each of the areas of interest are rated differently. Problems are rated on a six-point scale from “not at all” to “all of the time.” Level of functioning items are rated on a five point scale from “extreme troubles” to “doing very well.” Hopefulness and satisfaction with services are Likert style questions with various response categories. A total score for problem severity is a simple sum of the twenty items problems experienced with a range of 0 to 100. A total score for level of functioning is a simple sum of the twenty items for that scale with a range of 0 to 80.

Psychometrics: Data provided in the Procedural Manual for the Ohio Mental Health Outcomes System reports Cronbach’s alpha reliability estimates for the problem severity and functioning domains of from .86 to .93 in small samples of parents and agency workers. In a study of concurrent validity, correlations were obtained between the Ohio Scale short form and the original Ohio Scales. For problem severity, correlations ranged from .89 to .95 while correlations for the functioning domain ranged from .86 to .96. As the short form of the instrument significantly overlaps the original longer scale, interrater reliabilities, sensitivity to change, and validity have been presumed to be similar to those reported for the longer form. For the long form, test-retest reliability ranged from .67 for Satisfaction to .88 for problem severity for parents, and from .67 to .72 for youth. Problem severity and Level of functioning scales significantly correlated with the CAFAS (.52 and .59)and CGAS (.31 and .32).

Additional notes: This instrument is currently used in the state mental health authority in Ohio.

General Recommendations:

Level of functioning and change over time: Each of the instruments described have been used by state mental health authorities for the assessment of child outcomes. Based on the criteria provided below, the CAFAS and the CANS-MH are recommended for consideration for assessing individual functioning as part of an outcomes oriented, management information system. The advantages of these two instruments include:

1) They are in wide and relatively successful use around the country with the result that the state may have the opportunity to consult with other state agency staff regarding their implementation, use, and interpretation. The long history of their use may help identify potential pitfalls and limitations in the implementation and use of either instrument. Sites may also share their methods, reporting formats, and any lessons learned.

2) The author of each instrument, Dr. Kay Hodges for the CAFAS, and Dr. John Lyons for the CANS-MH, have a history of working with state mental health authorities in the implementation and use of their instruments. Each provide consultation, manuals, and training protocols.

3) Both instruments have psychometric evidence to support their use in the assessment of child and adolescent outcomes. As a newer instrument the CANS-MH has somewhat less published psychometric evidence of its validity and reliability than the CAFAS.

4) Both instruments have been used for multiple purposes in state mental health settings. These two instruments have been variously used for assessing individual improvement, identification of service needs, determination of level of care, treatment planning, assessing the quality of services, and providing feedback regarding system functioning.

Potential limitations or drawbacks in the use of these instruments may include:

1) The CAFAS is a proprietary instrument and has been rejected for use by several state mental health authorities because of the ongoing cost of the instrument.

2) Both instruments, the CANS-MH because of the number of areas assessed, and the CAFAS because of the use of the interview guide (parent report), each require a significant investment of time for completion.

Consumer feedback: With regard to the assessment of outcomes from the family and child perspective, the Consumer Quality Initiative (CQI) that has been used in the state may be considered for Medicaid eligible children. In lieu of working through CQI, the state may also consider using the YSS and YSSF. This instrument is widely used around the nation for the assessment of consumer perceptions of the accessibility, satisfaction and outcome of services. In either case, a system that allows for regular feedback from children and their caregivers is an important piece of the performance accountability process.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download