RMC Basic Report & Proposal Template



Contents

Background of HYS 2014 1

History of Washington’s Youth Survey Efforts 1

Statewide Support - Multiagency Partnership 1

HYS 2014 Administration 2

Small School District Pilot 3

Survey Form Variations 5

Survey Form Content 6

Statistical Issues - Fundamentals for Understanding Your Results 7

What is Reliability? 7

What is Validity? 8

What is Generalizability? 9

What are Confidence Intervals? 10

Determining Statistical Significance 11

Comparing Data Over Time 16

What is Available on and Getting Access 17

Levels of access to AskHYS 17

Getting Accessing to District and Building Results on 18

2014 HYS Frequency Reports 20

Opening a Frequency Report 20

Introduction and Overview 22

Report Highlights 23

Selected Results by Gender 24

Understanding Your Report 26

Frequency Results 27

2014 HYS PowerPoint Slides 32

Powerpoint Slides 32

2014 HYS Fact Sheets 35

Fact Sheet Types and Topics 35

To Open Fact Sheets 36

2014 HYS QxQ Analysis 40

QxQ Online Data Query System 40

Generating Results with the QxQ 42

QxQ Results 46

Crafting a Communication Message Around HYS Results 53

Talking about Survey Results 54

Planning to Communicate 57

Obtaining More Information 65

Section 1:

Background of HYS 2014

History of Washington’s Youth Survey Efforts

School-based youth health and risk surveys have been conducted in Washington since 1988.

[pic]

U Student Alcohol and Drug Use Survey

W Washington State Adolescent Health Behaviors

Y Youth Risk Behavior Survey

H Healthy Youth Survey

HYS 2014 was the 14th statewide school-based survey of adolescent health behaviors in Washington. Surveys have been implemented in Washington over the past 26 years,

beginning in 1988.

The time between survey administrations has fluctuated over the years, though, on

average the surveys have been administered every other year in the in the fall or

spring.

The content of the surveys has also varied. Some surveys have focused on health risks

(1988, 1990, 1999), whereas others have focused on risk and protective factors (1992,

1998). Recent surveys (1995, 2000, 2002, 2004, 2006, 2008, 2010, 2012 and 2014) have focused on both health risks and risk and protective factors.

Statewide Support - Multiagency Partnership

The planning and implementation of the Healthy Youth Survey is the collaborative effort of many state agencies. The efforts are coordinated though the Healthy Youth Survey Planning Committee (formerly known as the Joint Survey Planning Committee JSPC).

Healthy Youth Survey Planning Committee Agencies

Office of the Superintendent of Public Instruction (OSPI)

Division of Behavioral Health and Recovery, Department of Social and Health Services (DBHR/DSHS)

Department of Health (DOH)

Liquor Control Board

Looking Glass Analytics, Inc. (survey contractor)

HYS 2014 Administration

A simple random sample of schools is drawn to produce state-level estimates.

County samples are drawn as appropriate for larger counties with 30 or more schools in a grade.

• In 2014 King, Pierce, Snohomish, Spokane (Grade 6 & 8), Thurston (Grade 6) and Clark (Grades 6) had county samples drawn. The responses from these schools were used to produce county-level estimates.

[pic]

Non-sampled schools were also invited to participate in the survey; participation allows

these schools to obtain their own school results and to contribute to district-, county-,

and ESD-level results.

.

Map of 2014 HYS Participation by School District

Small School District Pilot

For the 2014 HYS, the HYSPC decided to allow “small” districts to survey additional grade levels – grades 7, 9 and 11 (in addition to the traditional survey grade levels 6, 8, 10 and 12) – to increase the number of students taking the survey.

To be considered “small” the district needed to have less than 150 students in grades 6, 8, 10 or 12. Statewide, 186 school districts were eligible to participate in the Pilot. All schools within those districts were eligible to participate in the Pilot.

Additional Results for Pilot Schools and Districts

Pilot schools and districts that surveyed students in the additional grade levels 7, 9 and 11 received additional results if they met the minimum reporting requirements.

A school or district that surveyed at least one student in grades 6, 7 and 8 (and had a total of at least 15 valid respondents) received Combined Middle School results. These results added together all of the students that took the survey in those grades:

Grade 6 students + Grade 7 students + Grade 8 students

= Combined Middle School results

A school or district that surveyed at least one student in grades 9, 10, 11 and 12 (and had a total of at least 15 valid respondents) received Combined High School results:

Grade 9 students + Grade 10 students + Grade 11 students + Grade 12 students

= Combined High School results

By increasing the number of students contributing to the Combined Middle School and Combined High School results – the results were more stable and had smaller confidence intervals.

Pilot Participation

For the 2014 Small School District Pilot, 81 districts and 141 schools in those districts participated. 101 schools received combined middle and/or high school results

Map of 2014 HYS Participation by School District

More information on the 2014 Small School District Pilot is available on .

Survey Form Variations

Secondary Survey Forms (Grades 8 through 12)

For Grades 8, 10, and 12, two variations of the survey, Form A and Form B, ensure

that the survey is not too long.

Most of the items are unique to one version or the other, but a set of common items forms the survey “core”.

When the surveys are packed in shrink-wrapped bundles of 30, they are interleaved so that half of the students receive Form A and half receive Form B. This approach effectively results in randomized distribution.

Forms A and B both include a 1-page tear-off scannable answer sheet.

Both secondary survey forms A and B are available in enhanced versions that include optional questions.

Form A-enhanced includes one question about sexual orientation:

o Which of the following best describes you? a. Heterosexual (straight), b. Gay or lesbian, c. Bisexual, d. Not sure

Form B-enhanced includes two questions about sexual abuse and four questions about sexual behavior.

o Have you ever been in a situation where someone made you engage in kissing, sexual touch or intercourse when you did not want to? a. No, b. Yes

o In the past 12 months, have you been in a situation where someone made you engage in kissing, sexual touch or intercourse when you did not want to? a. No, b. Yes

o How old were you when you had sexual intercourse for the first time? a. I have never had sexual intercourse., b. 11 years old or younger, c. 12 years old, d. 13 years old, e. 14 years old, f. 15 years old, g. 16 years old, h. 17 years old or older

o During your life, with whom have you had sexual contact? a. I have never had sexual contact., b. Females, c. Males, d. Females and males

o With how many people have you ever had sexual intercourse? a. I have never had sexual intercourse., b. 1 person, c. 2 people, d. 3 people, e. 4 people, f. 5 people, g. 6 or more people

o The last time you had sexual intercourse, did you or your partner use a condom? a. I have never had sexual intercourse., b. Yes, c. No

When schools registered for the 2014 HYS, they had the option to request the enhanced survey forms.

Elementary Survey Form (Grades 6, and Grade 7 in Pilot Districts)

For Grade 6 the sole variation, Form C, is a shortened and simplified combination of

Forms A and B.

• Form C includes a 1-page tear-off scannable answer sheet.

• There are no optional questions for Grade 6

Survey Form Content

|HYS 2014 Form A | |HYS 2014 Form B |

|Development lead by DBHR/DSHS and OSPI | |Development led by DOH |

|Many questions come from the Communities that Care Survey and | |Many questions come from the Youth Risk Behavior Survey and the|

|Monitoring the Future | |Youth Tobacco Survey |

|Demographics | |Demographics |

|Alcohol, marijuana, other drugs | |Alcohol, tobacco, other drugs |

|School risk and protective factors | |Nutrition |

|Community risk and protective factors | |Physical activity |

|Peer-individual-family risk and protective factor | |Safety behaviors |

| | |Mental health/depression/suicide |

|Form C: For Grade 6 students | | |

|Core survey items and other mutually agreed-upon items. | | |

Section 2

Statistical Issues - Fundamentals for Understanding Your Results

To ensure the validity and reliability of the data, the Healthy Youth Survey is implemented in accordance with standard administration and data cleaning procedures. The purpose for covering this is so you can answer the question “How do you know we can believe these students?”

What is Reliability?

The extent to which a measure, procedure or instrument yields the same result on repeated trials. A survey item is reliable if it consistently produces the same results under the same circumstances.

How we assure reliability:

• Standardized administration procedures

Administration procedures

• Student and parental notification (students and parents can choose not to take the survey).

• Standardized administration procedures (e.g., coordinator training, teacher training, written instructions, teacher stays in room but at desk, single class period to avoid discussion, absent students do not make up).

• Students informed of importance of the survey.

• Students do not put their name or other identifying information on the survey.

• Students place own answer sheet in envelope.

• Students receive resource list with phone numbers if the survey questions bring up issues that they need to discuss.

The administration procedures ensure a safe and confidential survey environment in which students can answer questions honestly.

What is Validity?

The degree to which the results are likely to be true, believable, and free of bias and can be generalized to a larger population. A survey item is valid if it accurately measures the concept it is intended to measure.

How we assure validity:

• [pic] Items from established instruments

• [pic] Use validity checks – data cleaning procedures

Content and Sources of HYS Items

Nearly all of the HYS items are from established surveys that have been used for years

throughout the United States, including:

• Communities That Care (CTC), developed by Hawkins and Catalano and supported by the U.S. Department of Health and Human Services

• Monitoring the Future (MTF), administered by the University of Michigan and supported by the National Institute of Drug Abuse

• Youth Risk Behavior Survey (YRBS), supported by the Center for Disease Control

• Youth Tobacco Survey (YTS), supported by the Center for Disease Control

Data Cleaning Procedures

Data cleaning procedures remove surveys that appear to have been answered dishonestly:

• Multiple inconsistent answers (e.g., if a student reports never drinking alcohol in their life and reports drinking alcohol in the past 30 days and provides other responses that similarly inconsistent with each other).

• Evidence of faking a high level of substance use (e.g., reporting the use of all substances every day).

• Dishonesty (if a student responds negatively to the item about honesty).

• Wrong grade (e.g., if a student received the Grade 6 survey form but reported being in Grade 8; these responses are included in school-level reports only).

Consistently, about 4% of the surveys submitted are removed through data cleaning. Although the responses on the removed surveys are very different than the average responses, removing them makes only a very small difference to the total results.

What is Generalizability?

Generalizability is the extent to which research survey results can be applied to the larger population (e.g., applying the results of the state sample to the entire state or the results of the county sample to the entire county). For example, if 9,100 public school Grade 6 students provide valid surveys, if the results are generalizable they are representative of the approximately 11,700 Grade 6 students statewide.

To whom can results be generalized?

• State and county sample.

• Non-sampled counties, districts, ESDs, and schools.

Why is a 70% participation rate important?

A high participation rate helps ensure that most of the schools and students eligible to take the

survey is represented in the results. If, however, important groups of students are missing from the data, there may be limitations despite a high participation rate.

What are the challenges to generalizability?

How could the situations described in the sidebar affect generalizability?

• How might they affect school-, district-, county- or ESD-level data?

• How do these challenges affect data interpretation?

In each of the situations important groups of students did not take the survey or distractions

might have influenced their responses.

Do I have to generalize?

• Yes, if you want to apply the results to a larger population: “Eighth graders said . . . .”

• Yes, if you want to compare your results overtime, or compare your results to others.

• No, if you want to simply describe the students who took the survey “Eighth graders who took the survey said . . . .”

What are Confidence Intervals?

HYS results include a ± number after each item estimate—this number is a confidence interval. A confidence interval accounts for the fact that the reported value is probably a little different than the true value for all of the students.

• A 95% confidence interval, for example, means that we are 95% confident that the true value is within the ± range.

• Confidence intervals are important when you generalize results to a larger population.

Why do we need confidence intervals if data are valid?

Confidence intervals account for variability among students, NOT the validity of the data.

• Variability is inherent in any population worth studying. If variability were not a factor, administering a survey to answer questions would not be necessary.

• Variability causes uncertainty in the results.

• Confidence intervals allow for the comparison of results to others and to us over time.

• Confidence intervals can protect us from making exaggerated claims - or claims that we can’t have confidence in.

What do confidence intervals look like?

For example, your survey results say 18.0% (± 2.0%) Grade 10 grade students used marijuana:

18.0 – 2.0 = 16.0, 18.0 + 2.0 = 20.0

You can interpret this as between: 16.0% and 20.0% used marijuana

[pic]

What if your confidence interval is larger? Your survey results say 18.0% (± 5.0%) Grade 10 grade students used marijuana:

18.0 – 5.0 = 11.0, 18.0 + 5.0 = 23.0

You can interpret this as between: 11.0% and 23.0% used marijuana

Why are confidence intervals different sizes?

The size of a confidence interval is effected by:

• Number of students. In general, the more students surveyed, the smaller the confidence interval.

• Inherent variability. If most students answer a survey question in the same way, then there is less variability. The more variable the answers, the wider the confidence intervals.

• Level of confidence. HYS uses 95% confidence intervals. This percentage is commonly used, but results can be calculated for different percentages. If 80% confidence were desired, the confidence interval would be smaller. If 99% confidence were desired, the confidence interval would be larger.

• Sampling design.

Determining Statistical Significance

What is statistical significance?

Statistical significance means that the probability that differences in results are not due to chance alone. When using 95% confidence intervals, a difference between two groups is considered statistically significant if chance could explain it only 5% of the time or less.

Confidence intervals can help you quickly determine significant differences, but there are more precise ways to determine significance. We will be showing you a tool later that can help with this. Also, assistance determining statistical significance is available from many sources including the local health department, the local ESD, HYSPC agencies, or the Internet.

Example of a significant difference between state and local data:

• Local students 25% ±5% (so range is 20% to 30%)

• State students 36% ±3% (so range is 33% to 39%)

[pic]

Conclusion: The highest value for the local students (30.0%) and the lowest value for the

State (33.0%) does not overlap; thus the difference IS statistically significant.

Example of a non-significant difference between state and local data:

• Local students 25% ±5% (so range is 20% to 30%)

• State students 28% ±3% (so range is 25% to 31%)

[pic]

Conclusion: The local student confidence interval range (20% to 30%) overlaps the state point estimate 25%, so the different is NOT statistically significant.

Example of an inconclusive difference between state and local data:

• Local students 25% ±5 (so range is 20% to 30%)

• State students 32% ±3 (so range is 29% to 35%)

[pic]

Conclusion: Inconclusive, requires more testing. The confidence intervals overlap each other, but don’t include one of the point estimates (neither 25% local or 32% state), so we can’t tell if the difference is significant or not.

Note: We will show you a way around this soon.

Exercise 1

Practice Using Confidence Intervals

In the sample below, are the state and local rates significantly different?

Yes No (circle one)

Example: What do the statistics suggest are the lowest and highest percentages of students who said “Yes” to the question “Has a doctor or nurse ever told you that you have asthma?”

|Example local |28.0 (± 4.0) |Example state estimate: |21.5 (±1.5) |

|estimate: | | | |

|Your range |24.0% to 32.0% |State range: |20.0% to 23.0% |

| | (lowest) (highest) | | (lowest) (highest) |

Turn to Item 18 in your local report. What are the lowest and highest percentages of students who said “Yes” to the question “Have you ever, even once in your life: Used marijuana?”

|Your estimate: |_________ ± __________ |State estimate: |_________ ± __________ |

|Your range: |_________ to __________ |State range: |_________ to __________ |

| | (lowest) (highest) | | (lowest) (highest) |

Are the differences above, between your local students and the state, statistically significant?

Yes No (circle one)

What does this mean for your students?

Answer to Question 1

Yes the difference is statistically significant. The lowest end of the range for the local students was 24.0%, is lower than the highest end of the state range, 23.0%. The confidence intervals don’t overlap, so the local result is significantly higher than the state.

You could say something like “Lifetime asthma in our county is significantly higher that lifetime asthma statewide”. Or “student in our county are more likely to have asthma than students statewide”.

[pic]

What if I am in a small school that has large confidence intervals?

Having a confidence interval protects you (and your program) from appearing to be

ineffective when just a few students make big changes.

Consider the input of teachers and staff from small-school environments when

interpreting data—the data should be used to complement what they already know

about their students.

If you’re a small district, consider using your Small School District Pilot combined Middle School or combined High School results. Your confidence intervals will be smaller.

Tool: Using Excel and Confidence Intervals to Perform a Statistical Test

Available at:

This Excel spreadsheet, available online can be used to determine statistical significance. Simply enter the results from your local report and the spreadsheet calculates a p-value which indicates whether there is statistical significance.

How to Determine if Your Local Results is Different from a State Results

Compute z-test from 95% confidence intervals

This spreadsheet computes the p-value for a test of the difference between two points

estimates, given their 95% confidence intervals and sample sizes.

Use this test ONLY if there are 30 or more students for the local results, and 30

or more for the state results.

[pic]

You can also use this tool to compare other types of results:

• Comparing grade levels (e.g., comparing Grade 8 to Grade 10)

• Comparing gender

• Comparing year to year (e.g., comparing 2012 to 2014)

Comparing Data Over Time

Things to think about:

Was the survey the same as in previous survey administrations?

• Did the survey question wording change?

• Did the response options change?

• Did the risk and protective factors scales change?

• More information about survey questions over time, see the HYS Data Dictionary and Crosswalk at

• To see if your location participated and recived survey results overtime, see

Were the challenges to generalizability similar?

• Were the response rates similar?

• Did the groups of students taking the survey change?

Is there a reason to think that changes might have occurred?

• Was a prevention program implemented that might have caused results to change?

• Was a policy implemented that might have have caused results to change?

Trends

Fact sheets do provide data back to 2004 for some questions – but do not provide a test for overall change in trend (only year to year comparisons)

To measure trends you need statistical software like SAS or STATA. Or you can use free software from the National Cancer Institute – uses your point estimates and confidence intervals:

Section 3

What is Available on and Getting Access

AskHYS survey results and information

New information – Welcome

Brief overview – About

HYSPC contact information – Contact

Access to district and building results – Log On

Current and past training materials – Training

• Survey Results

o Frequency reports

o Fact sheets

o QxQ Analysis

o Additional reports

o Past participation

o Survey questionnaires

o Press releases

o Errata

Levels of access to AskHYS

County, ESD and State results are available to everyone.

School district and school building HYS results are available only with the permission of the superintendents’ offices.

Getting Accessing to District and Building Results on

Healthy Youth Survey results at the state, county and Educational Service District (ESD) levels are available to the public on .

To access district and/or school level results, you must be granted access by the school district’s administration.

School and District Employees

1. Check to see if you can log on to the website using your EDS login (your email) and password.

a. If you do not have an EDS account, please create one here:

b. If you do not remember your password, please contact Krissy Johnson (OSPI) at krissy.johnson@k12.wa.us or (360) 725-6045 or your District Data Security Manager.

2. If you do have a login and password, and are unable to access your district and/or school reports, you may not have the correct permissions/roles in EDS. Please contact your District Data Security Manager to request the proper access.

If you are a school staff or administrator, you can request the “HYS Building Level Access” role for your school which will allow you to see your school’s results. If you are a district staff or administrator, you can request the “HYS Building Level Access” role for your district. You do not need both the Building Level and the District Level roles for the same organization. If you continue to have trouble accessing your school and district reports, please contact Krissy Johnson (OSPI) at krissy.johnson@k12.wa.us or (360) 725-6045.

Community Members, Researchers and Parents

1. Check to see if results are available for the district or building you are interested in. To do this, go to the “Past Participation” web page and search for the districts and/or schools that are on the list.

2. Contact the District Superintendent to request access to the specific district and/or building results. Explain who you are, your purpose of accessing specific district and/or building results, and how the results will be used. A school district directory is available at:

3. If you receive approval from the District Superintendent, contact the District Security Manager to receive the required electronic permission. A list of District Security Managers is available at:

4. Once you receive permission, log on the website using your assigned EDS user id and password. If you log on correctly, a message will appear in fine print at the bottom of the main screen.

5. The names of the districts and/or school buildings that you received permission for will now appear in the "Geography" drop down menu, under Frequency Reports or Fact Sheets.

For school district and building level access, click on the Log On page.

[pic]

If you successfully log on, you’ll see the following screen.

[pic]

Section 4

2014 HYS Frequency Reports

Opening a Frequency Report

• Select Frequency Reports under the Survey Results dropdown menu.

• Select the year

• Select your location

• Then the reports available will pop up on the right side of the screen. Click on the report to open it.

Notice there are new Multi-grade reports. These reports include the results available for all grades in the location in one report. Schools and districts that participated in the 2014 Small School District Pilot can also have additional reports with Combined Middle School and Combined High School results.

Frequency report title and sections

Note the date on the title page to ensure that your results are the most recent version available.

[pic]

Introduction and Overview

[pic]

Report Highlights

• Tables and charts provided for quick refernce. Detailed infomartion for each item is provided in the Frequency Result section of the report

• All reports contain the same items

• All local report results are compared to the state, except for the new Multi-grade reports which only include local results

• Charts match items

• Results displayed have been consisten over time

[pic]

Selected Results by Gender

• Selected survey questions are presented by gender to highlight any differences between females and males.

• The p-values reported after each question, calculated from a chi-square test, can be used to examine whether differences between females and males are statistically significant.

• P-values less than 0.05 indicate a significant difference between females and males.

• Results are suppress to protect student anonymity.

[pic]

Why Asterisks Instead of Number for Gender?

To produce results by gender a minimum of 10 respondents must give each response option

The number in each cell is dependent on two factors:

8 [pic] The proportions of females and males

9 [pic] The proportion of students reporting each response

For example to produce results for current cigarette smoking:

• At least 10 girls have to report smoking & at least 10 girls have to report not smoking AND

• At least 10 boys have to report smoking & at least 10 boys have to report not smoking

[pic]

[pic]

Understanding Your Report

“N” is shorthand for “number” --- the number of students who took this survey or who answered a survey question.

There are multiple reasons why different questions have different “N’s”:

• Question is only on Form A or B

• Question is optional

• Question is near the end of the survey

• Question is difficult to answer, or too personal

Samples of n’s from different survey questions for a district with 400 10th graders:

[pic]

Frequency Results

Item Grouping

The results from each survey item are grouped in the following order:

← General information.

← Alcohol, tobacco, and other drug use.

← Other health concerns.

← School climate.

← Quality of life

← Risk and protective factor items.

Example of an item result

[pic]

Example of an item result as a chart, Current (past 30-day) Marijuana Use

[pic]

Example of a Pilot Combined Middle School item result

[pic]

Example of a Multi-Grade Pilot Middle School item result

[pic]

Why are there asterisks?

• No students responded (n = 0).

• Overweight (Item 64) was suppressed to protect anonymity.

• School chose not to ask optional question.

• Results by gender require a minimum number of respondents (10 per cell for local results).

Risk and Protective Factors

Included with the item results is information about the risk and protective factor scale.

• Each “factor” is a “scale” measured with two or more questions, so that the factor includes multiple dimensions of the risk or protection.

• Risk factors that predict youth substance use, violence and delinquent behaviors. Protective factors that can protect youth from the effects of those risks.

• Prevention strategies can be designed to lower risk and increase protection.

• Academic Failure is a school risk factor, it is made up of the following two questions:

o What grades do you usually get

o How far do you think you will get in school

Risk and Protective Factor framework and reporting schedule

Provides information about which scales were included in the survey from 2002 to 2014.

o Provides information about which scales are asked on the secondary, elementary or both versions of the forms.

[pic]

Risk and Protective Factor scale results

Tables and charts for each set of factors by domain – Community, Family, School and Peer-Individual

[pic]

Risk and Protective Factor individual questions

Provides a list of the individual items that were grouped together to form scales (a set of closely related behaviors or attitudes).

[pic]

Last Pages of the Report

All reports end with an Item index with page numbers for specific topics

The last page of the 8th, 10th and 12th grade reports contains a List of Core Items. This list can help interpret n’s.

Section 5

2014 HYS PowerPoint Slides

Powerpoint Slides

In 2014, PowerPoint slide sets include information about student participation and 51 slides that include charts of local and state sample results. Slide topics include:

• Tobacco, Alcohol, Marijuana & Other Drugs

• School & Alcohol/Tobacco/Other Drugs (ATOD)

• School Environment & Safety

• Community & Safety

• Nutrition & Physical Activity

• Health & Mental Health

• ATOD & Community Norms

• ATOD & Access

• ATOD & Perceived Risk

Opening PowerPoint Slides

PowerPoint slides are located under Frequency Reports.

[pic]

Example of a typical slide

[pic]

Example of a Small School Pilot High School slide

[pic]

Section 6

2014 HYS Fact Sheets

Fact Sheet Types and Topics

In 2014, AskHYS includes 20 single grade and five multi-grade or “all grade” topical fact sheet.

Single grade fact sheets topics: All grade fact sheet topics:

To Open Fact Sheets

• Select Fact Sheets under the Survey Results dropdown menu.

• Select the survey year

• Select your location. Remember school district and building level results will only be an option if you are logged in to AskHYS.

• Then Fact Sheets that are available to you will appear on the right.

• You can select one or up to a maximum of 5 Fact Sheets at one time. Then hit the Submit button.

• You can run Fact Sheets for by gender, select Male or Female. The same minimum requirements for that are in place to present results by gender in the Frequency Reports apply to the Fact Sheets. If you run a Fact Sheet by gender, your results may be suppressed.

Single grade Fact Sheets contain the following information:

• The chart title, year, grade, gender and the number of students in that location that had valid responses to the survey. The actual number of youth who answered a specific question varies depending on which survey form the question is on and the number of students who did not answer the question but completed most of the survey.

• Background and where to find more information on the fact sheet topic.

• A bar chart with selected results for the year and grade selected. Charts include the percent values and 95% confidence intervals (as black bar I) for each measure. Some charts also include a short sentence that interprets one of the measures for that year, grade and location.

• A trend chart with results for all of the years that the selected results were available. Under the chart is a table with the percent values and 95% confidence intervals (as ±). If there is a statistically significant change from one year to the next, an asterisks * indicates the change. Trend charts do not include an overall test for trend

• A state comparison chart for all grades that the selected result is asked of. Results for the location selected and the state are included. Under the chart is a table with the percent values and 95% confidence intervals (as ±). If there is a statistically significant from the state, an asterisks * indicates the difference.

• Some fact sheets also include a state level cross-tab of a selected result by academic achievement. It shows the percent of youth who reported the result and if they are “mostly getting C’s, D’s, or F’s” in school. These results are only state level, and will be same on ESD, county, district and school fact sheets

“All grade” Fact Sheets contain the following information:

• Include information similar to single grade Fact Sheets, but present results for multiple grades so it’s easy to look at the differences in a question by grade level.

• Some include state level cross-tab of a selected result with appropriate questions like substance use.

Example of a typical 2014 single grade Fact Sheet

Example of a typical 2014 single grade Fact Sheet

Example of a typical 2014 single grade Fact Sheet

Example of a typical 2014 single grade Fact Sheet

Example of a typical 2014 “all grade” Fact Sheet (page 1)

[pic]

Example of a typical 2014 “all grade” Fact Sheet (page 2)

Section 7

2014 HYS QxQ Analysis

QxQ Online Data Query System

Running frequencies and cross-tabs

There are two types of analysis you can run on the QxQ:

• Run frequencies: responses to individual questions. Just like the results in your Frequency Reports.

• Run crosstabs: looking at the relationship between two questions - crossing one question by the second question.

If you want to create results by crossing one variable by a second variable, then you are producing a crosstab. Any variable that is crossed with any other variable, even gender (selected in the menu above) is considered a crosstab.

There are a few basic rules that are required to produce crosstabs

• There is a minimum number of respondents per cell

o For state level analysis you must have 5 or more respondents in each cell.

o For sub-state level analysis you must have 10 or more respondents in each cell.

• These cell size requirements are necessary to protect the anonymity of the youth who participate in the survey, are a requirement of the Washington State Institutional Review Board.

• Questions must be on the same survey form or be a “core” survey question.

• It is not possible to cross one question that is only on the form B survey questionnaire with one that is only on the form A survey questionnaire. For example, you cannot cross current cigar smoking (only on form B) with current methamphetamine use (only on form A).

If these requirements are not met then you will receive the following error messages

• For state level analysis: At least one cell in the result table contained a count of less than 5. Output is suppressed.

For sub-state level analysis: At least one cell in the result table contained a count of less than 10. Output is suppressed.

• If questions are not on the same form, or core: No surveys contained responses to all the selected variables.

• If question is not asked of the grade level: At least one of the questions in this query was not asked of Xth grade students.

Before you start, questions to ask yourself

• What do you want to know? Be familiar with the HYS questions and think about the results you want to create.

• Do you have results to analyze? Which grades do you want to look at? Check the Past Participation page to see what results are available:

• Can the questions you’re interested in be crossed? Check the HYS Data Dictionary and Crosswalk on the QxQ Instructions tab.

Generating Results with the QxQ

• Select QxQ Analysis under the Survey Results dropdown menu.

• The QxQ opens with an Instructions tab.

• Note that there are training videos throughout the AskHYS website. If you are new to the QxQ, these training videos maybe helpful in setting up your analysis and understanding why some results can’t be run. Also, please not that these training videos have not be updated yet to reflect some of the recent formatting changes to AskHYS, but the general information in the videos is still relevant.

[pic]

To set up your analysis

• Click on the Analysis Parameters tab.

• Select the survey year.

• Select the survey grade. Note that you can select a single grade or you can select “All” grades.

• Select your location. Remember school district and building level results will only be an option if you are logged in to AskHYS.

• You can run the QxQ for just one gender by selecting it on the Analysis Parameters tab, or you can use gender as one of your crosstab questions in the Query Builder.

[pic]

To run a frequency result (a result for just one question)

• Click on the Query Builder tab.

• Click on the Select a data category dropdown menu. There is a long list of topics and subtopics. The organization of the topics somewhat follows the way results are presented in the Frequency Reports. Note that some questions are under multiple topics. For example the question about easy access to marijuana is under the Marijuana topic, but also under the Community Risk Factor topic.

[pic]

Selecting your question

When you click on a variable, the question text and response options appear in the Description box below. This way you can verify that you are selecting the question you want to analyze.

Notice that all of the response options are shown in the box, but that the “Collapsed” variable option is often selected as the default. This was done primarily to make it easier to interpret your analysis and to deal with response options that have a small number of respondents.

• Once you have your question selected, click on it and drag it down into the Row Variables box.

• Change the response options to Collapsed or Survey as needed.

• Hit the Submit button.

QxQ Results

Example of a frequency output

In this example, we set up our Analysis Parameters for 2014, State Sample, All Grades (only results for 6th and 8th grades are presented below, but if you ran this analysis you’d also get output for 10th and 12th grades too).

We selected Current Alcohol Drinking as our question for analysis (collapsed responses as no days and any days).

[pic]

Analysis Output

The output provides you with the percent value for both of the response options, the ± 95% confidence interval and then the number of respondents that answered each response.

The number of respondents is just the number of youth who answered this question for this grade, year and location. It is also called the number of respondents per cell. In this case, the first cell “no days” for the 6th grade output has 8,574 respondents and the second cell “any days” has 180 respondents.

The output also provides the variable question again, so you can be certain which HYS question the results are for. Running frequency results can be useful, especially if you want to double check your analysis against the results in your original reports.

Selecting questions to run a crosstab

• To run a crosstab, you need to select two variables. Drag your first question into the Row Variables box and your second question into the Column Variables box.

• Review your questions in the Description box below and determine if you want Collapsed or Surveyed response options (remember you have a better chance of not getting your output suppressed for not meeting cell number requirements if you keep your response options collapsed).

• Hit the Submit button

[pic]

Example of a crosstab output

In this example, we set up our Analysis Parameters for 2014, State Sample, 8th Grade.

We selected Current Alcohol Drinking as our Row Variable and Current Marijuana Use as our Column Variable (collapsed responses as no days and any days for both questions).

[pic]

Analysis output

When reviewing your crosstab output, it’s very important to verify that you’ve generated the correct results.

In this example, we selected Current Alcohol as our Row Variable, so the results produced should be read as – among current alcohol drinkers.

Looking at each cell, this is how we can interpret the results:

1. Among 8th graders who did not drink, 96.4% did not use marijuana

2. Among 8th graders who did not drink, 3.6% did use marijuana

3. Among 8th graders who drank, 50.6% did not use marijuana

4. Among 8th graders who drank, 49.4% did use marijuana

If we really wanted to know if marijuana users drank instead, then we could switch our Row Variable to be Current Marijuana Use and our Column Variable to be Current Alcohol Use.

Another crosstab example

In our first example, if we really wanted to know if marijuana users drank instead, then we could switch our Row Variable to be Current Marijuana Use and our Column Variable to be Current Alcohol Use.

For some crosstabs there is definitely a right way and a wrong order to selecting your Row and Column Variables.

Correct way example – Race/ethnicity and bullying

In this example our Analysis Parameters are 2014, State sample and 8th grade. Our Variable selection is:

• Row Variable = Race/Ethnicity [RACEETH]

• Colum Variable = Bulling

Wrong way example – Race/ethnicity and bullying

In this example our Analysis Parameters are still 2014, State sample and 8th grade. Our Variable selection is reversed:

• Row Variable = Bulling

• Colum Variable = Race/Ethnicity [RACEETH]

[pic]

Other tips for using the Q x Q

• To remove a variable(s) from your analysis, simply drag it back up into the Available Variables box or hit the Reset button.

• At the bottom of the page there is a section title “Table Variable”. Currently this isn’t working and will just give you an error message. We’ll notify you on the Welcome page if it becomes usable.

• If you are going to be presenting results you run in the QxQ, be sure to double check them.

o Try running a similar analysis by gender and comparing it to the Results by gender in your Frequency Reports.

o Ask another person to confirm your results. There is a list of local health assessment staff who can help you run and interpret your results at:

• Please note that not all statistical packages run survey results exactly the same way. Sometimes they produce results that are slightly different due to how the round numbers.

Section 8

Crafting a Communication Message around HYS Results

Data are an important piece of the puzzle, but must be considered in the context of your community.

Data are only one type of information. Other information that may also be important to consider, include knowledge of community issues, the political climate, and available resources.

Remember:

• Using your data might be difficult, but you can do it!

• Data cannot answer everything, but they can be an important part of program planning and developing communication strategies.

Why talk about HYS results?

There are a number of reasons, here are just a few…

• HYS results give you an opportunity to share important information about our youth.

• Having real data, may make it possible to try to change or open people’s minds.

• HYS can be used to help with planning or decision making.

• HYS results can be used to help evaluate the effectiveness of a program or policy that you are implementing.

• HYS results can provide the information you need to ask for help, advocate for a change, or to even request or apply for funding.

An important consideration when talking about HYS results – your audience

Talking about Survey Results

Simplify so that your audience can understand the numbers

When quoting Health Youth Survey data, don’t say something like:

“About 17.4% plus or minus 3.1% of students said…..”

Round percentages to whole numbers: If the decimal point is between 0 and 4, round down; it the decimal point is between 5 and 9, round up:

• 17.0% to 17.4% becomes 17%

• 17.5% to 17.9% becomes 18%

Survey results are estimates, so you can use language that acknowledges these percentages are estimates.

• Say “about 17%”

• Carefully include your confidence interval

o Say between “14 to 20%”

o Say “plus or minus 3%”

Use language to convey that these percentages represent a specific group of students only (if your response rates are low or if you do not want to generalize to a larger population).

• Say “about 17% of the students surveyed in our community said . . . ”

Consider different ways of presenting the data:

Instead of saying “75.3% ±5.1% of 8th graders said that they ‘mostly’ or ‘definitely’ felt safe at school”, you could say:

• “about 75% of our 8th grades feel safe in school”

• “about 3/4ths of 8th grades feel safe in school”

• “about 3 out of 4 of 8th grades feel safe in school”

And remember that you can present data in the other direction; if that’s the point you need to make:

• “about 25% of our 8th grades don’t feel safe at school ”

• “about one in four of our 8th grades don’t feel safe at school ”

You can turn percentage results into numbers of students. If you have 200 8th graders in your school and 25% of them don’t feel safe, then you can say “about 50 of our 8th graders don’t feel safe at school”.

• Use some caution when turning percentages into numbers of students. Remember these are estimates, so you should consider your confidence interval. In this example the confidence interval was ±5.1%, so you’re 95% sure that about 20% to 30% of your 8th graders don’t feel safe at school. Always round to whole numbers and include your confidence interval to say something like “between 40 and 60 of our 8th graders don’t feel safe at school”

When possible, use visuals

• You have charts and tables you can use from your Frequency Reports, PowerPoint slides and Fact Sheets.

• Create your own charts or other types of visuals

7 ½ out of 10 students…

[pic]

Consider what messages are appropriate for communicating with different people, and how data fit into your message:

• An administrator or your boss.

• Your coworkers.

• Parents.

• Reporters.

• Your neighbor at a dinner party.

• Your grandmother.

• A grant application review.

Be ready to back up your talk

Know your facts about HYS

4 How, when and to whom it was administered

5 Details about the survey questions

Do the numbers make sense?

Are you using your results in an appropriate context?

What else is needed to tell the whole story?

9 Additional data sources

10 Information from informed people

General talking tips

• Keep it short and simple

• Double check your numbers

• Run it past a few people, especially a critic

• Don’t speculate. Remember your limitations

• It’s ok to say “I don’t know, I’ll get back to you”

• Be compassionate - these numbers are actually kids

Planning to Communicate

Communicating about data is difficult. Before you share your HYS results, devise a

communication plan. There are a number of ways to do this, but one approach to ensuring that your message is clear is to identify a single overriding communication objective—a SOCO—and to use a message map and to organize your thoughts.

Single Overriding Communication Objective (SOCO):

• The “so what” of your message.

• The “big picture”—the meaning in its context.

• Meaningful to your target audience.

• Tells them what you want them to learn or do.

What does a SOCO look like?

• For broadcast media: 10- to 12-word “sound bite”.

• For print media: 1- to 3 line quotation.

• For real people: 3 or 4 ideas—most people will not remember more than that.

SOCO example

“Community leaders should speak out in support of Safe & Drug Free Schools Programs”

A SOCO needs to be supported by details: “Healthy kids learn better”

• Provide logical justification.

• These are the reasons why.

The details need to be supported by facts: “HYS data show strong associations between substance use and lower academic achievement”

• These are the evidence of your details.

• Use your data here.

Old Example of a SOCO and message map

The state Tobacco Prevention & Control program used the following SOCO for an old press release and legislative report. Notice that there aren’t really any HYS percentages, but that HYS results were used to calculate numbers like “45 kids start smoking every day in WA”.

[pic]

Stay on message. The answer to every question is your SOCO.

“That’s an interesting point, but what I’d

really like people to know is [my SOCO]”

Exercise 2

Using Your Data to Communicate

You are a team of “Healthy School” advocates who have been seeking opportunities to increase support for early prevention programs among youth.

Using data provided on the next page and data from your own report, practice talking about Healthy Youth Survey results in one of the following scenarios. Use the message maps on the following pages. Be prepared to share your messages.

The school board has invited your team to talk with them and make your case about the need for more early prevention programs among youth.

The son of a prominent city council member was arrested for driving under the influence of marijuana. He attends the local high school. A radio station calls and asks you to provide a sound bite about marijuana use among youth in your community.

A local agency is offering funds for programs that support youth. Support your grant application for a mentoring program.

Of if you already have message that you’re working on. Look up HYS data to support you and share your message.

Exercise 4 Data

|HYS Question |Grade |2014 State |2014 Local |

|Current marijuana use among students (percentage who reported any use |6th |1.3 ± 0.4 |2.0 ±1.0 |

|within the past 30 days). | | | |

|Item #33 in 2014 report. | | | |

| |8th |7.3 ±1.1 |12.0 ±2.0 |

| |10th |18.1 ±1.7 |25.0 ±3.0 |

| |12th |26.7 ±2.2 |30.0 ±3.0 |

|Recent riding in car with someone who was drinking alcohol (percentage who |6th |na |na |

|report any times in the past 30 days). Item #108 in 2014 report. | | | |

| |8th |9.6 ±1.4 |10.0 ±2.0 |

| |10th |19.0 ±1.7 |26.0 ±2.0 |

| |12th |25.9 ±2.5 |30.0 ±3.0 |

|Recent driving while drinking alcohol (percentage who reported any driving |6th |na |na |

|while drinking in the past 30 days). Item #110 in 2001 report. | | | |

| |8th |4.5 ±0.8 |4.0 ±1.0 |

| |10th |9.1 ±1.4 |15.0 ±3.0 |

| |12th |16.8 ±1.6 |25.0 ±3.0 |

|Perceived harm using marijuana regularly among youth (percentage who |6th |10.9 ±1.4 |18.0 ±3.0 |

|reported believing there is “no risk”). | | | |

|Item #219 in 2014 report. | | | |

| |8th |8.7 ±1.8 |18.0 ±3.0 |

| |10th |15.4 ±1.8 |20.0 ±2.0 |

| |12th |21.9 ±2.2 |22.0 ±2.0 |

|Parental discussions about why youth should not use marijuana (percentage |6th |na |na |

|who reported No). Item #57 in 2014 report. | | | |

| |8th |25.0 ±1.7 |28.0 ±3.0 |

| |10th |32.1 ±1.3 |32.0 ±2.0 |

| |12th |42.4 ±1.7 |42.0 ±2.0 |

|Marijuana use on school property in the past 30 days (percentage that |6th |na |na |

|reported any days). Item 59 in the 2014 report. | | | |

| | | | |

| |8th |3.3 ±0.8 |8.0 ±3.0 |

| |10th |7.1 ±1.2 |10.0 ±2.0 |

| |12th |7.8 ±1.2 |10.0 ±2.0 |

|Received information in classes about reasons not to use alcohol or drugs |6th |24.7 ±2.3 |33.0 ±3.0 |

|(percentage who reported Never). Item 62 in the 2014 report. | | | |

| |8th |12.8 ±2.2 |30.0 ±3.0 |

| |10th |17.9 ±2.7 |20.0 ±2.0 |

| |12th |30.9 ±3.1 |30.0 ±2.0 |

Exercise 4 Message Map

|Single Overriding Communications Objective (SOCO): |

|Detail 1 |Detail 2 |Detail 3 |

|Fact |Fact |Fact |

|Fact |Fact |Fact |

|Fact |Fact |Fact |

Some SOCOs that we came up with:

• “The school board needs to take action to prioritize marijuana prevention programs for our middle school children”

• “Every parent in our community should talk to their kids about marijuana and driving”

• “Funding this program will reduce the excess danger that children in our community face from marijuana”

Details (supported by facts in the data)

• Marijuana use among our 8th and 10th graders is significantly higher compared to the state.

• Both driving after using marijuana and riding with a driver who had been using marijuana are significantly higher for our 10th and 12th graders compared to the state.

• The perception that there is “no harm” from regular youth marijuana use is significantly higher among our 6th, 8th and 10th graders compared to the state.

• Our 8th, 10th and 12th graders are just as likely to report their parents did not talk to them about not using marijuana compared to the state.

• Our 8th graders are significantly more likely to have used marijuana at school compared to the state.

• Our 6th and 8th graders are significantly more likely to say they never received information in classes about reasons not to use drugs or alcohol compared to the state.

Taking a step back, where do you start?

What information do you have?

• HYS reports, slides, fact sheets.

• What/who else can help you:

o Are other data available? (e.g. school discipline data)

o Are there other people who have information?

What are your findings?

What do your 2014 HYS results say?

• What HYS questions address school safety?

• How do your results compare to the state results?

• Do you have HYS results from previous years?

Developing your talking points

• If you could only make one point – what would it be?

• Are their specific opinions you need to influence?

• What do you want the board to ultimately do?

Section 9

Obtaining More Information

|Web Sites |

| |

|Main HYS website: |

| |

|Additional HYS background and technical information: |

| |

|Sponsoring Agencies & and Primary Contacts |

| |

|Office of Superintendent of Public Instruction |

|Krissy Johnson |

|360-725-6045, Krissy.Johnson@k12.wa.us |

| |

|Department of Health |

|Kevin Beck |

|877-HYS-7111, kevin.beck@doh. |

| |

|Department of Social and Health Services |

|Steve Smoothers |

|360-438-8066, Smothsw@dshs. |

| |

|Liquor Control Board |

|Mary Segawa |

|360-664-1771, MBSE@liq. |

| |

|Looking Glass Analytics, Inc. |

|Susan Richardson |

|Susan.Richardson@ |

-----------------------

Using your

Healthy Youth Survey

Results…

Exercise Workbook

Spring 2015

Regional Workshops

Learning Objective

Develop understanding of the history and contents of the Healthy Youth Survey.

HYS 2014 Participation

HYS 2014 had the greatest participation of any Washington youth health and risk survey.

Overall Participation State Sample Participation

223,825 students 35,262 students

989 schools 192 schools

215 school districts 110 school districts

All 39 counties 32 counties

Form A

Form B

Survey

Core

Form C

Core

Learning Objective

Understand concepts including validity, reliability

and generalizability and how to use confidence

intervals.

Quotations from 2014 HYS Administrators

“Most of the students didn’t have time to finish the survey because we had a fire drill”

“I thought giving the survey after the SAT would be good timing, but everyone just got up and left”

1

2

3

4

20.0% to 23.0%

24.0% to 32.0%

P-value is less than 0.05, so there is a significant difference. The local results are significantly higher than the state.

Learning Objective

Develop familiarity with the contents of the AskHYS website.

Need permission to access district and building results. Click here for information.

Learning Objective

Develop familiarity with the contents of the 2014 Frequency Reports.

Updates and corrections to original reports release in March

See the notice in the bottom left corner. This notice will always let you know fi there have been any updates or corrections made to HYS results. Click on the word Errata for details.

There were considerable changes since the original release of HYS results in March. Please see the Errata and re-access your survey results as necessary.

Local Report Sections

• Introduction & Overview

o About the survey, the number of respondents, keys to the notes

• Highlights

o Graphs of selected 30-day use, bullying and school climate results

• Selected Results by Gender

o Results to key questions for males and females

• Understanding Your Report

• Frequency Results

o Results of survey items organized by topic.

• Risk & Protective Factor Scales and individual items

o Factor framework, reporting schedule, scale results in graphs and tables, lists of surveys items that contribute to scales

• Item Index

• Core Questions

Explains notes used throughout the report

Explains the importance of the participation rates

Students enrolled from fall 2014

Enrolled ÷ usable surveys = participation

Submitted a “usable” survey (valid)

Submitted a survey

If p-value is less than 0.05 then the difference between males and females is significantly different

Sample factor tables and charts for Multi-grade reports

Learning Objective

Develop familiarity with the 2014 PowerPoint slides.

Learning Objective

Develop familiarity with the contents of the 2014 Fact Sheets.

Bullying and Harassment for Any County

In this County

What Do Any County Youth Say About Marijuana in 2014?

Learning Objective

Develop familiarity with the QxQ Analysis system and interpret output

In the Description box you can select the response options for you analysis output. In this example, the default is collapsed – no days and any days. If you want all the response options, click on the “Surveyed” radio button.

Results:

Statewide in 2014, 8.1% (± 1.0%) of 8th graders drank alcohol on any days

Results:

Statewide in 2014, 2.1% (± 0.4%) of 6th graders drank alcohol on any days

Results:

Statewide in 2014, 29.1% (±2.0) of White non-Hispanic 8th graders were bullied

Results:

Statewide in 2014, 26.3% (±2.3) of Hispanic 8th graders were bullied

Looking at these results by race/ethnicity, they all seem pretty reasonable. Some results are a little higher/lower but all in a similar range or about 20% to 30%

Looking at these results by race/ethnicity this way doesn’t seem reasonable. There are really big differences in the results among the race/ethnic groups.

The problem is that when the crosstab was set up, it ran, among those who were bullied for each race ethnicity. This is highly influenced by the high proportion of White respondents compared to the lower proportion of minority group respondents.

Results:

Statewide in 2014, 16.5% (±5.1) of Hispanic 8th graders were bullied

Results:

Statewide in 2014, 51.3% (±5.0) of White non-Hispanic 8th graders were bullied

Learning Objective

Learn effective communication methods and be better able to craft messages around HYS data.

A1

B

C

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download