RMC Basic Report & Proposal Template
Contents
Background of HYS 2014 1
History of Washington’s Youth Survey Efforts 1
Statewide Support - Multiagency Partnership 1
HYS 2014 Administration 2
Small School District Pilot 3
Survey Form Variations 5
Survey Form Content 6
Statistical Issues - Fundamentals for Understanding Your Results 7
What is Reliability? 7
What is Validity? 8
What is Generalizability? 9
What are Confidence Intervals? 10
Determining Statistical Significance 11
Comparing Data Over Time 16
What is Available on and Getting Access 17
Levels of access to AskHYS 17
Getting Accessing to District and Building Results on 18
2014 HYS Frequency Reports 20
Opening a Frequency Report 20
Introduction and Overview 22
Report Highlights 23
Selected Results by Gender 24
Understanding Your Report 26
Frequency Results 27
2014 HYS PowerPoint Slides 32
Powerpoint Slides 32
2014 HYS Fact Sheets 35
Fact Sheet Types and Topics 35
To Open Fact Sheets 36
2014 HYS QxQ Analysis 40
QxQ Online Data Query System 40
Generating Results with the QxQ 42
QxQ Results 46
Crafting a Communication Message Around HYS Results 53
Talking about Survey Results 54
Planning to Communicate 57
Obtaining More Information 65
Section 1:
Background of HYS 2014
History of Washington’s Youth Survey Efforts
School-based youth health and risk surveys have been conducted in Washington since 1988.
[pic]
U Student Alcohol and Drug Use Survey
W Washington State Adolescent Health Behaviors
Y Youth Risk Behavior Survey
H Healthy Youth Survey
HYS 2014 was the 14th statewide school-based survey of adolescent health behaviors in Washington. Surveys have been implemented in Washington over the past 26 years,
beginning in 1988.
The time between survey administrations has fluctuated over the years, though, on
average the surveys have been administered every other year in the in the fall or
spring.
The content of the surveys has also varied. Some surveys have focused on health risks
(1988, 1990, 1999), whereas others have focused on risk and protective factors (1992,
1998). Recent surveys (1995, 2000, 2002, 2004, 2006, 2008, 2010, 2012 and 2014) have focused on both health risks and risk and protective factors.
Statewide Support - Multiagency Partnership
The planning and implementation of the Healthy Youth Survey is the collaborative effort of many state agencies. The efforts are coordinated though the Healthy Youth Survey Planning Committee (formerly known as the Joint Survey Planning Committee JSPC).
Healthy Youth Survey Planning Committee Agencies
Office of the Superintendent of Public Instruction (OSPI)
Division of Behavioral Health and Recovery, Department of Social and Health Services (DBHR/DSHS)
Department of Health (DOH)
Liquor Control Board
Looking Glass Analytics, Inc. (survey contractor)
HYS 2014 Administration
A simple random sample of schools is drawn to produce state-level estimates.
County samples are drawn as appropriate for larger counties with 30 or more schools in a grade.
• In 2014 King, Pierce, Snohomish, Spokane (Grade 6 & 8), Thurston (Grade 6) and Clark (Grades 6) had county samples drawn. The responses from these schools were used to produce county-level estimates.
[pic]
Non-sampled schools were also invited to participate in the survey; participation allows
these schools to obtain their own school results and to contribute to district-, county-,
and ESD-level results.
.
Map of 2014 HYS Participation by School District
Small School District Pilot
For the 2014 HYS, the HYSPC decided to allow “small” districts to survey additional grade levels – grades 7, 9 and 11 (in addition to the traditional survey grade levels 6, 8, 10 and 12) – to increase the number of students taking the survey.
To be considered “small” the district needed to have less than 150 students in grades 6, 8, 10 or 12. Statewide, 186 school districts were eligible to participate in the Pilot. All schools within those districts were eligible to participate in the Pilot.
Additional Results for Pilot Schools and Districts
Pilot schools and districts that surveyed students in the additional grade levels 7, 9 and 11 received additional results if they met the minimum reporting requirements.
A school or district that surveyed at least one student in grades 6, 7 and 8 (and had a total of at least 15 valid respondents) received Combined Middle School results. These results added together all of the students that took the survey in those grades:
Grade 6 students + Grade 7 students + Grade 8 students
= Combined Middle School results
A school or district that surveyed at least one student in grades 9, 10, 11 and 12 (and had a total of at least 15 valid respondents) received Combined High School results:
Grade 9 students + Grade 10 students + Grade 11 students + Grade 12 students
= Combined High School results
By increasing the number of students contributing to the Combined Middle School and Combined High School results – the results were more stable and had smaller confidence intervals.
Pilot Participation
For the 2014 Small School District Pilot, 81 districts and 141 schools in those districts participated. 101 schools received combined middle and/or high school results
Map of 2014 HYS Participation by School District
More information on the 2014 Small School District Pilot is available on .
Survey Form Variations
Secondary Survey Forms (Grades 8 through 12)
For Grades 8, 10, and 12, two variations of the survey, Form A and Form B, ensure
that the survey is not too long.
Most of the items are unique to one version or the other, but a set of common items forms the survey “core”.
When the surveys are packed in shrink-wrapped bundles of 30, they are interleaved so that half of the students receive Form A and half receive Form B. This approach effectively results in randomized distribution.
Forms A and B both include a 1-page tear-off scannable answer sheet.
Both secondary survey forms A and B are available in enhanced versions that include optional questions.
Form A-enhanced includes one question about sexual orientation:
o Which of the following best describes you? a. Heterosexual (straight), b. Gay or lesbian, c. Bisexual, d. Not sure
Form B-enhanced includes two questions about sexual abuse and four questions about sexual behavior.
o Have you ever been in a situation where someone made you engage in kissing, sexual touch or intercourse when you did not want to? a. No, b. Yes
o In the past 12 months, have you been in a situation where someone made you engage in kissing, sexual touch or intercourse when you did not want to? a. No, b. Yes
o How old were you when you had sexual intercourse for the first time? a. I have never had sexual intercourse., b. 11 years old or younger, c. 12 years old, d. 13 years old, e. 14 years old, f. 15 years old, g. 16 years old, h. 17 years old or older
o During your life, with whom have you had sexual contact? a. I have never had sexual contact., b. Females, c. Males, d. Females and males
o With how many people have you ever had sexual intercourse? a. I have never had sexual intercourse., b. 1 person, c. 2 people, d. 3 people, e. 4 people, f. 5 people, g. 6 or more people
o The last time you had sexual intercourse, did you or your partner use a condom? a. I have never had sexual intercourse., b. Yes, c. No
When schools registered for the 2014 HYS, they had the option to request the enhanced survey forms.
Elementary Survey Form (Grades 6, and Grade 7 in Pilot Districts)
For Grade 6 the sole variation, Form C, is a shortened and simplified combination of
Forms A and B.
• Form C includes a 1-page tear-off scannable answer sheet.
• There are no optional questions for Grade 6
Survey Form Content
|HYS 2014 Form A | |HYS 2014 Form B |
|Development lead by DBHR/DSHS and OSPI | |Development led by DOH |
|Many questions come from the Communities that Care Survey and | |Many questions come from the Youth Risk Behavior Survey and the|
|Monitoring the Future | |Youth Tobacco Survey |
|Demographics | |Demographics |
|Alcohol, marijuana, other drugs | |Alcohol, tobacco, other drugs |
|School risk and protective factors | |Nutrition |
|Community risk and protective factors | |Physical activity |
|Peer-individual-family risk and protective factor | |Safety behaviors |
| | |Mental health/depression/suicide |
|Form C: For Grade 6 students | | |
|Core survey items and other mutually agreed-upon items. | | |
Section 2
Statistical Issues - Fundamentals for Understanding Your Results
To ensure the validity and reliability of the data, the Healthy Youth Survey is implemented in accordance with standard administration and data cleaning procedures. The purpose for covering this is so you can answer the question “How do you know we can believe these students?”
What is Reliability?
The extent to which a measure, procedure or instrument yields the same result on repeated trials. A survey item is reliable if it consistently produces the same results under the same circumstances.
How we assure reliability:
• Standardized administration procedures
Administration procedures
• Student and parental notification (students and parents can choose not to take the survey).
• Standardized administration procedures (e.g., coordinator training, teacher training, written instructions, teacher stays in room but at desk, single class period to avoid discussion, absent students do not make up).
• Students informed of importance of the survey.
• Students do not put their name or other identifying information on the survey.
• Students place own answer sheet in envelope.
• Students receive resource list with phone numbers if the survey questions bring up issues that they need to discuss.
The administration procedures ensure a safe and confidential survey environment in which students can answer questions honestly.
What is Validity?
The degree to which the results are likely to be true, believable, and free of bias and can be generalized to a larger population. A survey item is valid if it accurately measures the concept it is intended to measure.
How we assure validity:
• [pic] Items from established instruments
• [pic] Use validity checks – data cleaning procedures
Content and Sources of HYS Items
Nearly all of the HYS items are from established surveys that have been used for years
throughout the United States, including:
• Communities That Care (CTC), developed by Hawkins and Catalano and supported by the U.S. Department of Health and Human Services
• Monitoring the Future (MTF), administered by the University of Michigan and supported by the National Institute of Drug Abuse
• Youth Risk Behavior Survey (YRBS), supported by the Center for Disease Control
• Youth Tobacco Survey (YTS), supported by the Center for Disease Control
Data Cleaning Procedures
Data cleaning procedures remove surveys that appear to have been answered dishonestly:
• Multiple inconsistent answers (e.g., if a student reports never drinking alcohol in their life and reports drinking alcohol in the past 30 days and provides other responses that similarly inconsistent with each other).
• Evidence of faking a high level of substance use (e.g., reporting the use of all substances every day).
• Dishonesty (if a student responds negatively to the item about honesty).
• Wrong grade (e.g., if a student received the Grade 6 survey form but reported being in Grade 8; these responses are included in school-level reports only).
Consistently, about 4% of the surveys submitted are removed through data cleaning. Although the responses on the removed surveys are very different than the average responses, removing them makes only a very small difference to the total results.
What is Generalizability?
Generalizability is the extent to which research survey results can be applied to the larger population (e.g., applying the results of the state sample to the entire state or the results of the county sample to the entire county). For example, if 9,100 public school Grade 6 students provide valid surveys, if the results are generalizable they are representative of the approximately 11,700 Grade 6 students statewide.
To whom can results be generalized?
• State and county sample.
• Non-sampled counties, districts, ESDs, and schools.
Why is a 70% participation rate important?
A high participation rate helps ensure that most of the schools and students eligible to take the
survey is represented in the results. If, however, important groups of students are missing from the data, there may be limitations despite a high participation rate.
What are the challenges to generalizability?
How could the situations described in the sidebar affect generalizability?
• How might they affect school-, district-, county- or ESD-level data?
• How do these challenges affect data interpretation?
In each of the situations important groups of students did not take the survey or distractions
might have influenced their responses.
Do I have to generalize?
• Yes, if you want to apply the results to a larger population: “Eighth graders said . . . .”
• Yes, if you want to compare your results overtime, or compare your results to others.
• No, if you want to simply describe the students who took the survey “Eighth graders who took the survey said . . . .”
What are Confidence Intervals?
HYS results include a ± number after each item estimate—this number is a confidence interval. A confidence interval accounts for the fact that the reported value is probably a little different than the true value for all of the students.
• A 95% confidence interval, for example, means that we are 95% confident that the true value is within the ± range.
• Confidence intervals are important when you generalize results to a larger population.
Why do we need confidence intervals if data are valid?
Confidence intervals account for variability among students, NOT the validity of the data.
• Variability is inherent in any population worth studying. If variability were not a factor, administering a survey to answer questions would not be necessary.
• Variability causes uncertainty in the results.
• Confidence intervals allow for the comparison of results to others and to us over time.
• Confidence intervals can protect us from making exaggerated claims - or claims that we can’t have confidence in.
What do confidence intervals look like?
For example, your survey results say 18.0% (± 2.0%) Grade 10 grade students used marijuana:
18.0 – 2.0 = 16.0, 18.0 + 2.0 = 20.0
You can interpret this as between: 16.0% and 20.0% used marijuana
[pic]
What if your confidence interval is larger? Your survey results say 18.0% (± 5.0%) Grade 10 grade students used marijuana:
18.0 – 5.0 = 11.0, 18.0 + 5.0 = 23.0
You can interpret this as between: 11.0% and 23.0% used marijuana
Why are confidence intervals different sizes?
The size of a confidence interval is effected by:
• Number of students. In general, the more students surveyed, the smaller the confidence interval.
• Inherent variability. If most students answer a survey question in the same way, then there is less variability. The more variable the answers, the wider the confidence intervals.
• Level of confidence. HYS uses 95% confidence intervals. This percentage is commonly used, but results can be calculated for different percentages. If 80% confidence were desired, the confidence interval would be smaller. If 99% confidence were desired, the confidence interval would be larger.
• Sampling design.
Determining Statistical Significance
What is statistical significance?
Statistical significance means that the probability that differences in results are not due to chance alone. When using 95% confidence intervals, a difference between two groups is considered statistically significant if chance could explain it only 5% of the time or less.
Confidence intervals can help you quickly determine significant differences, but there are more precise ways to determine significance. We will be showing you a tool later that can help with this. Also, assistance determining statistical significance is available from many sources including the local health department, the local ESD, HYSPC agencies, or the Internet.
Example of a significant difference between state and local data:
• Local students 25% ±5% (so range is 20% to 30%)
• State students 36% ±3% (so range is 33% to 39%)
[pic]
Conclusion: The highest value for the local students (30.0%) and the lowest value for the
State (33.0%) does not overlap; thus the difference IS statistically significant.
Example of a non-significant difference between state and local data:
• Local students 25% ±5% (so range is 20% to 30%)
• State students 28% ±3% (so range is 25% to 31%)
[pic]
Conclusion: The local student confidence interval range (20% to 30%) overlaps the state point estimate 25%, so the different is NOT statistically significant.
Example of an inconclusive difference between state and local data:
• Local students 25% ±5 (so range is 20% to 30%)
• State students 32% ±3 (so range is 29% to 35%)
[pic]
Conclusion: Inconclusive, requires more testing. The confidence intervals overlap each other, but don’t include one of the point estimates (neither 25% local or 32% state), so we can’t tell if the difference is significant or not.
Note: We will show you a way around this soon.
Exercise 1
Practice Using Confidence Intervals
In the sample below, are the state and local rates significantly different?
Yes No (circle one)
Example: What do the statistics suggest are the lowest and highest percentages of students who said “Yes” to the question “Has a doctor or nurse ever told you that you have asthma?”
|Example local |28.0 (± 4.0) |Example state estimate: |21.5 (±1.5) |
|estimate: | | | |
|Your range |24.0% to 32.0% |State range: |20.0% to 23.0% |
| | (lowest) (highest) | | (lowest) (highest) |
Turn to Item 18 in your local report. What are the lowest and highest percentages of students who said “Yes” to the question “Have you ever, even once in your life: Used marijuana?”
|Your estimate: |_________ ± __________ |State estimate: |_________ ± __________ |
|Your range: |_________ to __________ |State range: |_________ to __________ |
| | (lowest) (highest) | | (lowest) (highest) |
Are the differences above, between your local students and the state, statistically significant?
Yes No (circle one)
What does this mean for your students?
Answer to Question 1
Yes the difference is statistically significant. The lowest end of the range for the local students was 24.0%, is lower than the highest end of the state range, 23.0%. The confidence intervals don’t overlap, so the local result is significantly higher than the state.
You could say something like “Lifetime asthma in our county is significantly higher that lifetime asthma statewide”. Or “student in our county are more likely to have asthma than students statewide”.
[pic]
What if I am in a small school that has large confidence intervals?
Having a confidence interval protects you (and your program) from appearing to be
ineffective when just a few students make big changes.
Consider the input of teachers and staff from small-school environments when
interpreting data—the data should be used to complement what they already know
about their students.
If you’re a small district, consider using your Small School District Pilot combined Middle School or combined High School results. Your confidence intervals will be smaller.
Tool: Using Excel and Confidence Intervals to Perform a Statistical Test
Available at:
This Excel spreadsheet, available online can be used to determine statistical significance. Simply enter the results from your local report and the spreadsheet calculates a p-value which indicates whether there is statistical significance.
How to Determine if Your Local Results is Different from a State Results
Compute z-test from 95% confidence intervals
This spreadsheet computes the p-value for a test of the difference between two points
estimates, given their 95% confidence intervals and sample sizes.
Use this test ONLY if there are 30 or more students for the local results, and 30
or more for the state results.
[pic]
You can also use this tool to compare other types of results:
• Comparing grade levels (e.g., comparing Grade 8 to Grade 10)
• Comparing gender
• Comparing year to year (e.g., comparing 2012 to 2014)
Comparing Data Over Time
Things to think about:
Was the survey the same as in previous survey administrations?
• Did the survey question wording change?
• Did the response options change?
• Did the risk and protective factors scales change?
• More information about survey questions over time, see the HYS Data Dictionary and Crosswalk at
• To see if your location participated and recived survey results overtime, see
Were the challenges to generalizability similar?
• Were the response rates similar?
• Did the groups of students taking the survey change?
Is there a reason to think that changes might have occurred?
• Was a prevention program implemented that might have caused results to change?
• Was a policy implemented that might have have caused results to change?
Trends
Fact sheets do provide data back to 2004 for some questions – but do not provide a test for overall change in trend (only year to year comparisons)
To measure trends you need statistical software like SAS or STATA. Or you can use free software from the National Cancer Institute – uses your point estimates and confidence intervals:
Section 3
What is Available on and Getting Access
AskHYS survey results and information
New information – Welcome
Brief overview – About
HYSPC contact information – Contact
Access to district and building results – Log On
Current and past training materials – Training
• Survey Results
o Frequency reports
o Fact sheets
o QxQ Analysis
o Additional reports
o Past participation
o Survey questionnaires
o Press releases
o Errata
Levels of access to AskHYS
County, ESD and State results are available to everyone.
School district and school building HYS results are available only with the permission of the superintendents’ offices.
Getting Accessing to District and Building Results on
Healthy Youth Survey results at the state, county and Educational Service District (ESD) levels are available to the public on .
To access district and/or school level results, you must be granted access by the school district’s administration.
School and District Employees
1. Check to see if you can log on to the website using your EDS login (your email) and password.
a. If you do not have an EDS account, please create one here:
b. If you do not remember your password, please contact Krissy Johnson (OSPI) at krissy.johnson@k12.wa.us or (360) 725-6045 or your District Data Security Manager.
2. If you do have a login and password, and are unable to access your district and/or school reports, you may not have the correct permissions/roles in EDS. Please contact your District Data Security Manager to request the proper access.
If you are a school staff or administrator, you can request the “HYS Building Level Access” role for your school which will allow you to see your school’s results. If you are a district staff or administrator, you can request the “HYS Building Level Access” role for your district. You do not need both the Building Level and the District Level roles for the same organization. If you continue to have trouble accessing your school and district reports, please contact Krissy Johnson (OSPI) at krissy.johnson@k12.wa.us or (360) 725-6045.
Community Members, Researchers and Parents
1. Check to see if results are available for the district or building you are interested in. To do this, go to the “Past Participation” web page and search for the districts and/or schools that are on the list.
2. Contact the District Superintendent to request access to the specific district and/or building results. Explain who you are, your purpose of accessing specific district and/or building results, and how the results will be used. A school district directory is available at:
3. If you receive approval from the District Superintendent, contact the District Security Manager to receive the required electronic permission. A list of District Security Managers is available at:
4. Once you receive permission, log on the website using your assigned EDS user id and password. If you log on correctly, a message will appear in fine print at the bottom of the main screen.
5. The names of the districts and/or school buildings that you received permission for will now appear in the "Geography" drop down menu, under Frequency Reports or Fact Sheets.
For school district and building level access, click on the Log On page.
[pic]
If you successfully log on, you’ll see the following screen.
[pic]
Section 4
2014 HYS Frequency Reports
Opening a Frequency Report
• Select Frequency Reports under the Survey Results dropdown menu.
• Select the year
• Select your location
• Then the reports available will pop up on the right side of the screen. Click on the report to open it.
Notice there are new Multi-grade reports. These reports include the results available for all grades in the location in one report. Schools and districts that participated in the 2014 Small School District Pilot can also have additional reports with Combined Middle School and Combined High School results.
Frequency report title and sections
Note the date on the title page to ensure that your results are the most recent version available.
[pic]
Introduction and Overview
[pic]
Report Highlights
• Tables and charts provided for quick refernce. Detailed infomartion for each item is provided in the Frequency Result section of the report
• All reports contain the same items
• All local report results are compared to the state, except for the new Multi-grade reports which only include local results
• Charts match items
• Results displayed have been consisten over time
[pic]
Selected Results by Gender
• Selected survey questions are presented by gender to highlight any differences between females and males.
• The p-values reported after each question, calculated from a chi-square test, can be used to examine whether differences between females and males are statistically significant.
• P-values less than 0.05 indicate a significant difference between females and males.
• Results are suppress to protect student anonymity.
[pic]
Why Asterisks Instead of Number for Gender?
To produce results by gender a minimum of 10 respondents must give each response option
The number in each cell is dependent on two factors:
8 [pic] The proportions of females and males
9 [pic] The proportion of students reporting each response
For example to produce results for current cigarette smoking:
• At least 10 girls have to report smoking & at least 10 girls have to report not smoking AND
• At least 10 boys have to report smoking & at least 10 boys have to report not smoking
[pic]
[pic]
Understanding Your Report
“N” is shorthand for “number” --- the number of students who took this survey or who answered a survey question.
There are multiple reasons why different questions have different “N’s”:
• Question is only on Form A or B
• Question is optional
• Question is near the end of the survey
• Question is difficult to answer, or too personal
Samples of n’s from different survey questions for a district with 400 10th graders:
[pic]
Frequency Results
Item Grouping
The results from each survey item are grouped in the following order:
← General information.
← Alcohol, tobacco, and other drug use.
← Other health concerns.
← School climate.
← Quality of life
← Risk and protective factor items.
Example of an item result
[pic]
Example of an item result as a chart, Current (past 30-day) Marijuana Use
[pic]
Example of a Pilot Combined Middle School item result
[pic]
Example of a Multi-Grade Pilot Middle School item result
[pic]
Why are there asterisks?
• No students responded (n = 0).
• Overweight (Item 64) was suppressed to protect anonymity.
• School chose not to ask optional question.
• Results by gender require a minimum number of respondents (10 per cell for local results).
Risk and Protective Factors
Included with the item results is information about the risk and protective factor scale.
• Each “factor” is a “scale” measured with two or more questions, so that the factor includes multiple dimensions of the risk or protection.
• Risk factors that predict youth substance use, violence and delinquent behaviors. Protective factors that can protect youth from the effects of those risks.
• Prevention strategies can be designed to lower risk and increase protection.
• Academic Failure is a school risk factor, it is made up of the following two questions:
o What grades do you usually get
o How far do you think you will get in school
Risk and Protective Factor framework and reporting schedule
Provides information about which scales were included in the survey from 2002 to 2014.
o Provides information about which scales are asked on the secondary, elementary or both versions of the forms.
[pic]
Risk and Protective Factor scale results
Tables and charts for each set of factors by domain – Community, Family, School and Peer-Individual
[pic]
Risk and Protective Factor individual questions
Provides a list of the individual items that were grouped together to form scales (a set of closely related behaviors or attitudes).
[pic]
Last Pages of the Report
All reports end with an Item index with page numbers for specific topics
The last page of the 8th, 10th and 12th grade reports contains a List of Core Items. This list can help interpret n’s.
Section 5
2014 HYS PowerPoint Slides
Powerpoint Slides
In 2014, PowerPoint slide sets include information about student participation and 51 slides that include charts of local and state sample results. Slide topics include:
• Tobacco, Alcohol, Marijuana & Other Drugs
• School & Alcohol/Tobacco/Other Drugs (ATOD)
• School Environment & Safety
• Community & Safety
• Nutrition & Physical Activity
• Health & Mental Health
• ATOD & Community Norms
• ATOD & Access
• ATOD & Perceived Risk
Opening PowerPoint Slides
PowerPoint slides are located under Frequency Reports.
[pic]
Example of a typical slide
[pic]
Example of a Small School Pilot High School slide
[pic]
Section 6
2014 HYS Fact Sheets
Fact Sheet Types and Topics
In 2014, AskHYS includes 20 single grade and five multi-grade or “all grade” topical fact sheet.
Single grade fact sheets topics: All grade fact sheet topics:
To Open Fact Sheets
• Select Fact Sheets under the Survey Results dropdown menu.
• Select the survey year
• Select your location. Remember school district and building level results will only be an option if you are logged in to AskHYS.
• Then Fact Sheets that are available to you will appear on the right.
• You can select one or up to a maximum of 5 Fact Sheets at one time. Then hit the Submit button.
• You can run Fact Sheets for by gender, select Male or Female. The same minimum requirements for that are in place to present results by gender in the Frequency Reports apply to the Fact Sheets. If you run a Fact Sheet by gender, your results may be suppressed.
Single grade Fact Sheets contain the following information:
• The chart title, year, grade, gender and the number of students in that location that had valid responses to the survey. The actual number of youth who answered a specific question varies depending on which survey form the question is on and the number of students who did not answer the question but completed most of the survey.
• Background and where to find more information on the fact sheet topic.
• A bar chart with selected results for the year and grade selected. Charts include the percent values and 95% confidence intervals (as black bar I) for each measure. Some charts also include a short sentence that interprets one of the measures for that year, grade and location.
• A trend chart with results for all of the years that the selected results were available. Under the chart is a table with the percent values and 95% confidence intervals (as ±). If there is a statistically significant change from one year to the next, an asterisks * indicates the change. Trend charts do not include an overall test for trend
• A state comparison chart for all grades that the selected result is asked of. Results for the location selected and the state are included. Under the chart is a table with the percent values and 95% confidence intervals (as ±). If there is a statistically significant from the state, an asterisks * indicates the difference.
• Some fact sheets also include a state level cross-tab of a selected result by academic achievement. It shows the percent of youth who reported the result and if they are “mostly getting C’s, D’s, or F’s” in school. These results are only state level, and will be same on ESD, county, district and school fact sheets
“All grade” Fact Sheets contain the following information:
• Include information similar to single grade Fact Sheets, but present results for multiple grades so it’s easy to look at the differences in a question by grade level.
• Some include state level cross-tab of a selected result with appropriate questions like substance use.
Example of a typical 2014 single grade Fact Sheet
Example of a typical 2014 single grade Fact Sheet
Example of a typical 2014 single grade Fact Sheet
Example of a typical 2014 single grade Fact Sheet
Example of a typical 2014 “all grade” Fact Sheet (page 1)
[pic]
Example of a typical 2014 “all grade” Fact Sheet (page 2)
Section 7
2014 HYS QxQ Analysis
QxQ Online Data Query System
Running frequencies and cross-tabs
There are two types of analysis you can run on the QxQ:
• Run frequencies: responses to individual questions. Just like the results in your Frequency Reports.
• Run crosstabs: looking at the relationship between two questions - crossing one question by the second question.
If you want to create results by crossing one variable by a second variable, then you are producing a crosstab. Any variable that is crossed with any other variable, even gender (selected in the menu above) is considered a crosstab.
There are a few basic rules that are required to produce crosstabs
• There is a minimum number of respondents per cell
o For state level analysis you must have 5 or more respondents in each cell.
o For sub-state level analysis you must have 10 or more respondents in each cell.
• These cell size requirements are necessary to protect the anonymity of the youth who participate in the survey, are a requirement of the Washington State Institutional Review Board.
• Questions must be on the same survey form or be a “core” survey question.
• It is not possible to cross one question that is only on the form B survey questionnaire with one that is only on the form A survey questionnaire. For example, you cannot cross current cigar smoking (only on form B) with current methamphetamine use (only on form A).
If these requirements are not met then you will receive the following error messages
• For state level analysis: At least one cell in the result table contained a count of less than 5. Output is suppressed.
For sub-state level analysis: At least one cell in the result table contained a count of less than 10. Output is suppressed.
• If questions are not on the same form, or core: No surveys contained responses to all the selected variables.
• If question is not asked of the grade level: At least one of the questions in this query was not asked of Xth grade students.
Before you start, questions to ask yourself
• What do you want to know? Be familiar with the HYS questions and think about the results you want to create.
• Do you have results to analyze? Which grades do you want to look at? Check the Past Participation page to see what results are available:
• Can the questions you’re interested in be crossed? Check the HYS Data Dictionary and Crosswalk on the QxQ Instructions tab.
Generating Results with the QxQ
• Select QxQ Analysis under the Survey Results dropdown menu.
• The QxQ opens with an Instructions tab.
• Note that there are training videos throughout the AskHYS website. If you are new to the QxQ, these training videos maybe helpful in setting up your analysis and understanding why some results can’t be run. Also, please not that these training videos have not be updated yet to reflect some of the recent formatting changes to AskHYS, but the general information in the videos is still relevant.
[pic]
To set up your analysis
• Click on the Analysis Parameters tab.
• Select the survey year.
• Select the survey grade. Note that you can select a single grade or you can select “All” grades.
• Select your location. Remember school district and building level results will only be an option if you are logged in to AskHYS.
• You can run the QxQ for just one gender by selecting it on the Analysis Parameters tab, or you can use gender as one of your crosstab questions in the Query Builder.
[pic]
To run a frequency result (a result for just one question)
• Click on the Query Builder tab.
• Click on the Select a data category dropdown menu. There is a long list of topics and subtopics. The organization of the topics somewhat follows the way results are presented in the Frequency Reports. Note that some questions are under multiple topics. For example the question about easy access to marijuana is under the Marijuana topic, but also under the Community Risk Factor topic.
[pic]
Selecting your question
When you click on a variable, the question text and response options appear in the Description box below. This way you can verify that you are selecting the question you want to analyze.
Notice that all of the response options are shown in the box, but that the “Collapsed” variable option is often selected as the default. This was done primarily to make it easier to interpret your analysis and to deal with response options that have a small number of respondents.
• Once you have your question selected, click on it and drag it down into the Row Variables box.
• Change the response options to Collapsed or Survey as needed.
• Hit the Submit button.
QxQ Results
Example of a frequency output
In this example, we set up our Analysis Parameters for 2014, State Sample, All Grades (only results for 6th and 8th grades are presented below, but if you ran this analysis you’d also get output for 10th and 12th grades too).
We selected Current Alcohol Drinking as our question for analysis (collapsed responses as no days and any days).
[pic]
Analysis Output
The output provides you with the percent value for both of the response options, the ± 95% confidence interval and then the number of respondents that answered each response.
The number of respondents is just the number of youth who answered this question for this grade, year and location. It is also called the number of respondents per cell. In this case, the first cell “no days” for the 6th grade output has 8,574 respondents and the second cell “any days” has 180 respondents.
The output also provides the variable question again, so you can be certain which HYS question the results are for. Running frequency results can be useful, especially if you want to double check your analysis against the results in your original reports.
Selecting questions to run a crosstab
• To run a crosstab, you need to select two variables. Drag your first question into the Row Variables box and your second question into the Column Variables box.
• Review your questions in the Description box below and determine if you want Collapsed or Surveyed response options (remember you have a better chance of not getting your output suppressed for not meeting cell number requirements if you keep your response options collapsed).
• Hit the Submit button
[pic]
Example of a crosstab output
In this example, we set up our Analysis Parameters for 2014, State Sample, 8th Grade.
We selected Current Alcohol Drinking as our Row Variable and Current Marijuana Use as our Column Variable (collapsed responses as no days and any days for both questions).
[pic]
Analysis output
When reviewing your crosstab output, it’s very important to verify that you’ve generated the correct results.
In this example, we selected Current Alcohol as our Row Variable, so the results produced should be read as – among current alcohol drinkers.
Looking at each cell, this is how we can interpret the results:
1. Among 8th graders who did not drink, 96.4% did not use marijuana
2. Among 8th graders who did not drink, 3.6% did use marijuana
3. Among 8th graders who drank, 50.6% did not use marijuana
4. Among 8th graders who drank, 49.4% did use marijuana
If we really wanted to know if marijuana users drank instead, then we could switch our Row Variable to be Current Marijuana Use and our Column Variable to be Current Alcohol Use.
Another crosstab example
In our first example, if we really wanted to know if marijuana users drank instead, then we could switch our Row Variable to be Current Marijuana Use and our Column Variable to be Current Alcohol Use.
For some crosstabs there is definitely a right way and a wrong order to selecting your Row and Column Variables.
Correct way example – Race/ethnicity and bullying
In this example our Analysis Parameters are 2014, State sample and 8th grade. Our Variable selection is:
• Row Variable = Race/Ethnicity [RACEETH]
• Colum Variable = Bulling
Wrong way example – Race/ethnicity and bullying
In this example our Analysis Parameters are still 2014, State sample and 8th grade. Our Variable selection is reversed:
• Row Variable = Bulling
• Colum Variable = Race/Ethnicity [RACEETH]
[pic]
Other tips for using the Q x Q
• To remove a variable(s) from your analysis, simply drag it back up into the Available Variables box or hit the Reset button.
• At the bottom of the page there is a section title “Table Variable”. Currently this isn’t working and will just give you an error message. We’ll notify you on the Welcome page if it becomes usable.
• If you are going to be presenting results you run in the QxQ, be sure to double check them.
o Try running a similar analysis by gender and comparing it to the Results by gender in your Frequency Reports.
o Ask another person to confirm your results. There is a list of local health assessment staff who can help you run and interpret your results at:
• Please note that not all statistical packages run survey results exactly the same way. Sometimes they produce results that are slightly different due to how the round numbers.
Section 8
Crafting a Communication Message around HYS Results
Data are an important piece of the puzzle, but must be considered in the context of your community.
Data are only one type of information. Other information that may also be important to consider, include knowledge of community issues, the political climate, and available resources.
Remember:
• Using your data might be difficult, but you can do it!
• Data cannot answer everything, but they can be an important part of program planning and developing communication strategies.
Why talk about HYS results?
There are a number of reasons, here are just a few…
• HYS results give you an opportunity to share important information about our youth.
• Having real data, may make it possible to try to change or open people’s minds.
• HYS can be used to help with planning or decision making.
• HYS results can be used to help evaluate the effectiveness of a program or policy that you are implementing.
• HYS results can provide the information you need to ask for help, advocate for a change, or to even request or apply for funding.
An important consideration when talking about HYS results – your audience
Talking about Survey Results
Simplify so that your audience can understand the numbers
When quoting Health Youth Survey data, don’t say something like:
“About 17.4% plus or minus 3.1% of students said…..”
Round percentages to whole numbers: If the decimal point is between 0 and 4, round down; it the decimal point is between 5 and 9, round up:
• 17.0% to 17.4% becomes 17%
• 17.5% to 17.9% becomes 18%
Survey results are estimates, so you can use language that acknowledges these percentages are estimates.
• Say “about 17%”
• Carefully include your confidence interval
o Say between “14 to 20%”
o Say “plus or minus 3%”
Use language to convey that these percentages represent a specific group of students only (if your response rates are low or if you do not want to generalize to a larger population).
• Say “about 17% of the students surveyed in our community said . . . ”
Consider different ways of presenting the data:
Instead of saying “75.3% ±5.1% of 8th graders said that they ‘mostly’ or ‘definitely’ felt safe at school”, you could say:
• “about 75% of our 8th grades feel safe in school”
• “about 3/4ths of 8th grades feel safe in school”
• “about 3 out of 4 of 8th grades feel safe in school”
And remember that you can present data in the other direction; if that’s the point you need to make:
• “about 25% of our 8th grades don’t feel safe at school ”
• “about one in four of our 8th grades don’t feel safe at school ”
You can turn percentage results into numbers of students. If you have 200 8th graders in your school and 25% of them don’t feel safe, then you can say “about 50 of our 8th graders don’t feel safe at school”.
• Use some caution when turning percentages into numbers of students. Remember these are estimates, so you should consider your confidence interval. In this example the confidence interval was ±5.1%, so you’re 95% sure that about 20% to 30% of your 8th graders don’t feel safe at school. Always round to whole numbers and include your confidence interval to say something like “between 40 and 60 of our 8th graders don’t feel safe at school”
When possible, use visuals
• You have charts and tables you can use from your Frequency Reports, PowerPoint slides and Fact Sheets.
• Create your own charts or other types of visuals
7 ½ out of 10 students…
[pic]
Consider what messages are appropriate for communicating with different people, and how data fit into your message:
• An administrator or your boss.
• Your coworkers.
• Parents.
• Reporters.
• Your neighbor at a dinner party.
• Your grandmother.
• A grant application review.
Be ready to back up your talk
Know your facts about HYS
4 How, when and to whom it was administered
5 Details about the survey questions
Do the numbers make sense?
Are you using your results in an appropriate context?
What else is needed to tell the whole story?
9 Additional data sources
10 Information from informed people
General talking tips
• Keep it short and simple
• Double check your numbers
• Run it past a few people, especially a critic
• Don’t speculate. Remember your limitations
• It’s ok to say “I don’t know, I’ll get back to you”
• Be compassionate - these numbers are actually kids
Planning to Communicate
Communicating about data is difficult. Before you share your HYS results, devise a
communication plan. There are a number of ways to do this, but one approach to ensuring that your message is clear is to identify a single overriding communication objective—a SOCO—and to use a message map and to organize your thoughts.
Single Overriding Communication Objective (SOCO):
• The “so what” of your message.
• The “big picture”—the meaning in its context.
• Meaningful to your target audience.
• Tells them what you want them to learn or do.
What does a SOCO look like?
• For broadcast media: 10- to 12-word “sound bite”.
• For print media: 1- to 3 line quotation.
• For real people: 3 or 4 ideas—most people will not remember more than that.
SOCO example
“Community leaders should speak out in support of Safe & Drug Free Schools Programs”
A SOCO needs to be supported by details: “Healthy kids learn better”
• Provide logical justification.
• These are the reasons why.
The details need to be supported by facts: “HYS data show strong associations between substance use and lower academic achievement”
• These are the evidence of your details.
• Use your data here.
Old Example of a SOCO and message map
The state Tobacco Prevention & Control program used the following SOCO for an old press release and legislative report. Notice that there aren’t really any HYS percentages, but that HYS results were used to calculate numbers like “45 kids start smoking every day in WA”.
[pic]
Stay on message. The answer to every question is your SOCO.
“That’s an interesting point, but what I’d
really like people to know is [my SOCO]”
Exercise 2
Using Your Data to Communicate
You are a team of “Healthy School” advocates who have been seeking opportunities to increase support for early prevention programs among youth.
Using data provided on the next page and data from your own report, practice talking about Healthy Youth Survey results in one of the following scenarios. Use the message maps on the following pages. Be prepared to share your messages.
The school board has invited your team to talk with them and make your case about the need for more early prevention programs among youth.
The son of a prominent city council member was arrested for driving under the influence of marijuana. He attends the local high school. A radio station calls and asks you to provide a sound bite about marijuana use among youth in your community.
A local agency is offering funds for programs that support youth. Support your grant application for a mentoring program.
Of if you already have message that you’re working on. Look up HYS data to support you and share your message.
Exercise 4 Data
|HYS Question |Grade |2014 State |2014 Local |
|Current marijuana use among students (percentage who reported any use |6th |1.3 ± 0.4 |2.0 ±1.0 |
|within the past 30 days). | | | |
|Item #33 in 2014 report. | | | |
| |8th |7.3 ±1.1 |12.0 ±2.0 |
| |10th |18.1 ±1.7 |25.0 ±3.0 |
| |12th |26.7 ±2.2 |30.0 ±3.0 |
|Recent riding in car with someone who was drinking alcohol (percentage who |6th |na |na |
|report any times in the past 30 days). Item #108 in 2014 report. | | | |
| |8th |9.6 ±1.4 |10.0 ±2.0 |
| |10th |19.0 ±1.7 |26.0 ±2.0 |
| |12th |25.9 ±2.5 |30.0 ±3.0 |
|Recent driving while drinking alcohol (percentage who reported any driving |6th |na |na |
|while drinking in the past 30 days). Item #110 in 2001 report. | | | |
| |8th |4.5 ±0.8 |4.0 ±1.0 |
| |10th |9.1 ±1.4 |15.0 ±3.0 |
| |12th |16.8 ±1.6 |25.0 ±3.0 |
|Perceived harm using marijuana regularly among youth (percentage who |6th |10.9 ±1.4 |18.0 ±3.0 |
|reported believing there is “no risk”). | | | |
|Item #219 in 2014 report. | | | |
| |8th |8.7 ±1.8 |18.0 ±3.0 |
| |10th |15.4 ±1.8 |20.0 ±2.0 |
| |12th |21.9 ±2.2 |22.0 ±2.0 |
|Parental discussions about why youth should not use marijuana (percentage |6th |na |na |
|who reported No). Item #57 in 2014 report. | | | |
| |8th |25.0 ±1.7 |28.0 ±3.0 |
| |10th |32.1 ±1.3 |32.0 ±2.0 |
| |12th |42.4 ±1.7 |42.0 ±2.0 |
|Marijuana use on school property in the past 30 days (percentage that |6th |na |na |
|reported any days). Item 59 in the 2014 report. | | | |
| | | | |
| |8th |3.3 ±0.8 |8.0 ±3.0 |
| |10th |7.1 ±1.2 |10.0 ±2.0 |
| |12th |7.8 ±1.2 |10.0 ±2.0 |
|Received information in classes about reasons not to use alcohol or drugs |6th |24.7 ±2.3 |33.0 ±3.0 |
|(percentage who reported Never). Item 62 in the 2014 report. | | | |
| |8th |12.8 ±2.2 |30.0 ±3.0 |
| |10th |17.9 ±2.7 |20.0 ±2.0 |
| |12th |30.9 ±3.1 |30.0 ±2.0 |
Exercise 4 Message Map
|Single Overriding Communications Objective (SOCO): |
|Detail 1 |Detail 2 |Detail 3 |
|Fact |Fact |Fact |
|Fact |Fact |Fact |
|Fact |Fact |Fact |
Some SOCOs that we came up with:
• “The school board needs to take action to prioritize marijuana prevention programs for our middle school children”
• “Every parent in our community should talk to their kids about marijuana and driving”
• “Funding this program will reduce the excess danger that children in our community face from marijuana”
Details (supported by facts in the data)
• Marijuana use among our 8th and 10th graders is significantly higher compared to the state.
• Both driving after using marijuana and riding with a driver who had been using marijuana are significantly higher for our 10th and 12th graders compared to the state.
• The perception that there is “no harm” from regular youth marijuana use is significantly higher among our 6th, 8th and 10th graders compared to the state.
• Our 8th, 10th and 12th graders are just as likely to report their parents did not talk to them about not using marijuana compared to the state.
• Our 8th graders are significantly more likely to have used marijuana at school compared to the state.
• Our 6th and 8th graders are significantly more likely to say they never received information in classes about reasons not to use drugs or alcohol compared to the state.
Taking a step back, where do you start?
What information do you have?
• HYS reports, slides, fact sheets.
• What/who else can help you:
o Are other data available? (e.g. school discipline data)
o Are there other people who have information?
What are your findings?
What do your 2014 HYS results say?
• What HYS questions address school safety?
• How do your results compare to the state results?
• Do you have HYS results from previous years?
Developing your talking points
• If you could only make one point – what would it be?
• Are their specific opinions you need to influence?
• What do you want the board to ultimately do?
Section 9
Obtaining More Information
|Web Sites |
| |
|Main HYS website: |
| |
|Additional HYS background and technical information: |
| |
|Sponsoring Agencies & and Primary Contacts |
| |
|Office of Superintendent of Public Instruction |
|Krissy Johnson |
|360-725-6045, Krissy.Johnson@k12.wa.us |
| |
|Department of Health |
|Kevin Beck |
|877-HYS-7111, kevin.beck@doh. |
| |
|Department of Social and Health Services |
|Steve Smoothers |
|360-438-8066, Smothsw@dshs. |
| |
|Liquor Control Board |
|Mary Segawa |
|360-664-1771, MBSE@liq. |
| |
|Looking Glass Analytics, Inc. |
|Susan Richardson |
|Susan.Richardson@ |
-----------------------
Using your
Healthy Youth Survey
Results…
Exercise Workbook
Spring 2015
Regional Workshops
Learning Objective
Develop understanding of the history and contents of the Healthy Youth Survey.
HYS 2014 Participation
HYS 2014 had the greatest participation of any Washington youth health and risk survey.
Overall Participation State Sample Participation
223,825 students 35,262 students
989 schools 192 schools
215 school districts 110 school districts
All 39 counties 32 counties
Form A
Form B
Survey
Core
Form C
Core
Learning Objective
Understand concepts including validity, reliability
and generalizability and how to use confidence
intervals.
Quotations from 2014 HYS Administrators
“Most of the students didn’t have time to finish the survey because we had a fire drill”
“I thought giving the survey after the SAT would be good timing, but everyone just got up and left”
1
2
3
4
20.0% to 23.0%
24.0% to 32.0%
P-value is less than 0.05, so there is a significant difference. The local results are significantly higher than the state.
Learning Objective
Develop familiarity with the contents of the AskHYS website.
Need permission to access district and building results. Click here for information.
Learning Objective
Develop familiarity with the contents of the 2014 Frequency Reports.
Updates and corrections to original reports release in March
See the notice in the bottom left corner. This notice will always let you know fi there have been any updates or corrections made to HYS results. Click on the word Errata for details.
There were considerable changes since the original release of HYS results in March. Please see the Errata and re-access your survey results as necessary.
Local Report Sections
• Introduction & Overview
o About the survey, the number of respondents, keys to the notes
• Highlights
o Graphs of selected 30-day use, bullying and school climate results
• Selected Results by Gender
o Results to key questions for males and females
• Understanding Your Report
• Frequency Results
o Results of survey items organized by topic.
• Risk & Protective Factor Scales and individual items
o Factor framework, reporting schedule, scale results in graphs and tables, lists of surveys items that contribute to scales
• Item Index
• Core Questions
Explains notes used throughout the report
Explains the importance of the participation rates
Students enrolled from fall 2014
Enrolled ÷ usable surveys = participation
Submitted a “usable” survey (valid)
Submitted a survey
If p-value is less than 0.05 then the difference between males and females is significantly different
Sample factor tables and charts for Multi-grade reports
Learning Objective
Develop familiarity with the 2014 PowerPoint slides.
Learning Objective
Develop familiarity with the contents of the 2014 Fact Sheets.
Bullying and Harassment for Any County
In this County
What Do Any County Youth Say About Marijuana in 2014?
Learning Objective
Develop familiarity with the QxQ Analysis system and interpret output
In the Description box you can select the response options for you analysis output. In this example, the default is collapsed – no days and any days. If you want all the response options, click on the “Surveyed” radio button.
Results:
Statewide in 2014, 8.1% (± 1.0%) of 8th graders drank alcohol on any days
Results:
Statewide in 2014, 2.1% (± 0.4%) of 6th graders drank alcohol on any days
Results:
Statewide in 2014, 29.1% (±2.0) of White non-Hispanic 8th graders were bullied
Results:
Statewide in 2014, 26.3% (±2.3) of Hispanic 8th graders were bullied
Looking at these results by race/ethnicity, they all seem pretty reasonable. Some results are a little higher/lower but all in a similar range or about 20% to 30%
Looking at these results by race/ethnicity this way doesn’t seem reasonable. There are really big differences in the results among the race/ethnic groups.
The problem is that when the crosstab was set up, it ran, among those who were bullied for each race ethnicity. This is highly influenced by the high proportion of White respondents compared to the lower proportion of minority group respondents.
Results:
Statewide in 2014, 16.5% (±5.1) of Hispanic 8th graders were bullied
Results:
Statewide in 2014, 51.3% (±5.0) of White non-Hispanic 8th graders were bullied
Learning Objective
Learn effective communication methods and be better able to craft messages around HYS data.
A1
B
C
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- free business proposal template pdf
- business proposal template word
- business proposal template free
- blank proposal template word
- simple business proposal template word
- free proposal template microsoft word
- business proposal template pdf
- free construction proposal template word
- blank construction proposal template free
- request for proposal template microsoft word
- free business proposal template word
- free marketing proposal template word