Protocol for the NM SPF SIG Community Survey



TABLE OF CONTENTS

General Background Information on the NMCS 3

Timeline 4

Data Collection Methodology/Protocol Overview 5

Qualtrics App Download 7

Local Data Collection Strategy 8

“Paper and Pencil” versus “iPad App” versus “Online Browser” versions of the survey 9

Key issues for implementing the survey using the qualtrics app 10

Key issues for implementing the survey on-line using the URL or QR code 10

Adding Questions 11

Time and Venue-Based Data Collection Concerns 11

Timing and community culture 12

Incentives 12

Protecting Human Subjects 13

Data Collection Logs 13

NMCS 2020 Protocol Guidance for Budget Planning 14

Summary of Responsibilities for Prevention Staff and Community Partners 17

Summary of PIRE’s Responsibilities 18

A Note about Budgets 20

Community Data Collection Protocol 21

Provider use of OSAP iPads 26

Using the NMCS app on a private device 26

NMCS App Protocol 26

PIRE Contact Information 29

General Background Information on the NMCS

The New Mexico Community Survey is sponsored by the NM Office of Substance Abuse Prevention (OSAP). In fiscal year 2020, two methodologies will be used to collect data: 1) in person recruitment and 2) on-line recruitment through provided links and PIRE’s social media efforts. This document is specifically intended for use by communities to collect community-level data, using a time and venue-based sampling methodology. While not a true random sample, using a consistent data collection methodology over time can add integrity to the data collection process and quality to your data, in turn contributing to the reliability and validity of the conclusions drawn from the data. Here, times and venues/places where data are collected are selected based on the likelihood of collecting a reasonably representative sample of the intended population. This is an approach frequently used for ‘hard-to-reach” populations, however, in the case of the community survey, it is being used to add credibility and scientific rigor to the convenience samples collected by communities for the purposes of increasing their sample sizes and potentially to target critical subpopulations.

There is a Core module that everyone must implement and then optional modules based on what prevention strategies your community is addressing and/or what data your stakeholders would like to collect. You and your evaluator will decide on the modules you will include as well as any additional questions you may want to ask, but not already included in the modules. All modules will be available in English and in Spanish.

These are the FY20 modules:

1- Core Module- Includes all demographic, perception of ATOD within the community, alcohol consumption & access, and prescription painkiller use & access. This module is required by all programs to implement on and off-line. (42 questions)

2- Methamphetamine -Includes new questions related to access and use. This module will be standard for all online submissions that are not community-administered using a program site ID (7 questions)

3- Opioid Module - Includes additional questions about Rx opioid and heroin use among family and friends. (9 questions)

4- Community Module- Includes questions about community concern and awareness of alcohol problems in the community and how they should be addressed. For those of you working on Social Host Ordinances, or the alcohol tax, this module will be required to measure outcomes. This survey parallels the College Community module, so you will likely do one or the other in your community. We recommend using every other year. (1 question)

5- College Community Module- Includes questions about community concern and awareness of alcohol problems in the community related to the college/university/students and how they should be addressed. Other communities with college campuses can also implement the college community module, and we are happy to discuss with you and your local evaluator(s) how this might work best. We recommend colleges use every other year. (1 question)

6- Marijuana-New questions related to access and use. (16 questions)

7- Adverse Childhood Experiences (ACEs) Module – Includes questions about adverse childhood events that may have occurred during the adult respondent’s youth. (12 questions)

8- Mental Health- Includes current mental and behavioral health status and access to services questions (6 questions)

9- Tobacco Module- Includes tobacco and vapor product questions (7 questions)

10- Gambling Module - Includes questions about involvement in various gambling activities and a 3-item gambling severity screening tool. (5 questions)

Like last year, we will still allow each community to add specific questions that are not included in these modules assuming we have the time and person power to get it programmed into Qualtrics. Please be aware of overall survey length when putting together content of your final survey. The longer the survey, the less likely respondents are to complete it.

College & Community Grantees in San Juan, Bernalillo, Doña Ana, and Socorro counties: Before writing and submitting your data collection protocol for review, you must confer and organize with the other grantee in the county (there is both a county and a college/university grantee in these counties) and determine how you will collaborate to implement the community modules. Our recommendation is that the college/university grantees implement the college community module and agree to cover locations near the campus, where students live and where residents are directly affected by campus/student drinking behavior. The county grantee will implement the community module with residents further removed from direct university influence. Locations, times, and other pertinent information should be mutually agreed upon.

Timeline

Please put these dates on your calendar so you can plan accordingly.

• January 22 - First draft of data collection protocol due to PIRE

• Feb 5– Budget due to OSAP

• January 23-31- PIRE internal review of data collection protocols. PIRE will work with sites individually to prepare protocol for SEOW review

• February 3-7 - SEOW reviews

• February 10-14 - PIRE works with communities to finalize protocols

• February 24 - Data collection may begin with approved protocol and online survey goes live

• Early March TBD - Data Entry Training

• April 27 - All data collection must end

• May 15 - Data files (for paper surveys) are due

• June 1-15 - PIRE returns aggregated data files to LEs

• Late June - PIRE returns R-reports (if resources available)

• July 15 - All EOY reporting documents submitted to OSAP

Data Collection Methodology/Protocol Overview

[pic]

You have three ways to collect data in person.

1) Through a Qualtrics app installed on an OSAP iPad, PIRE tablet, or your own IOS or Android device. This allows you to collect data while you are offline and connect later to upload the data. PIRE recommends using the Qualtrics app offline if you have a weak internet connection. If you have a strong internet connection, we recommend using option 2, rather than the app.

2) Through a browser to access the survey while you are online, from any internet connected device via or QR Code

[pic]

3) Using paper surveys.

When collecting data, all three methods require that you track the locations, dates & times and number of surveys collected in order to use for next year’s data collection planning.

Qualtrics App Download

If installing the app on your own device, follow these steps:

• Find “Qualtrics” app on your iOS or Android device. Check for app updates and update the app before downloading the 2020 NMCS surveys.

• Download app and to upload survey, when it enters, you will be told to enter a

Username: nmcs@ and password is PIRE2020!

• In the set-up, do not allow for geo-tracking and do not remove the ‘close’ button.

• Download the survey.

• When survey requires updating, you will be informed to “refresh”

• You will need your site ID for ALL data collection.

• Please test the survey by taking the survey before beginning data collection; BUT You must let us know so that we can delete any test data from the data file.

• Be ready for alerts from PIRE about need to refresh survey.

• Let PIRE know if issues arise immediately.

The age range of the sample targeted for this Community Survey is 18 and over. No one under the age of 18 may be recruited. Depending on your goals and scope of work, you may want to over-sample those 18 to 25 as well as any other subgroups that you feel should be represented (e.g. seniors, military, LatinX, LGBT folks). This is something you will need to consider as you complete your community-specific data collection protocol.

Local Data Collection Strategy

• Only adults 18 and older may complete the questionnaire.

• Only adult (over 18) trained community partners, volunteers and program staff will survey community members.

• You will provide pencils or pens, clip boards, and a box for collecting completed surveys anonymously.

• You may also use an OSAP-provided iPad with a PIRE/Qualtrics app installed that allows you to take the survey with or without being connected to the web. (You are not required to use iPads to collect data, but you may find this helpful with younger adults and their use relieves you of data entry burden.)

• Personal or prevention program devices can also be used to access the survey via an internet-connected web browser (not required).

• The app can also be installed on your personal iOS/Apple or Android device. This can be handy if you wish to have more devices with which to collect data when offline.

• You must use the entire survey module/s as they currently exist without any changes or modifications. You may add additional questions at the end with prior SEOW approval.

• During data collection, you will explain to every person the purpose of the survey, about the content, that participating in the survey is voluntary, and why the survey is being conducted (located on the front consent page).

• In addition, you must give everyone who completes the questionnaire the “take home information” about whom to call if they have further questions about the survey or want to talk with a mental health, substance abuse professional, gambling, or other problem.

The following locations are possible recruitment sites. The selection of each site depends upon where you will be most likely to gather data from a representative sample of your community, or subpopulation.

• The MVD*

• The local library

• The community college, university

• Local employers

• Post office

• Sporting events & hardware stores

• Target, markets, coffee shops, Wal-Mart, restaurants and other local businesses

• Farmer’s Markets

*Large communities with MVD office are required to at least try to collect at the local MVD office.

Areas where you generally are not allowed to gather data include healthcare settings, health centers or health fairs, local programming for high-risk populations, treatment facilities, AA, or substance use-related clinics. You cannot collect data in bars or public events where participants can be inebriated. You are also generally not allowed to collect data from incarcerated or homeless populations. Exceptions to this must be approved by the SEOW and you must provide justification in your protocol proposal.

If you need data on specific subpopulations, or if you have another site in mind that is very specific to your program’s experience, or your community’s specific culture, geography and economy, please communicate with PIRE to determine feasibility of including this site in order to ease data collection protocol review process. Always provide a justification or explanation in your protocol for any “questionable” data collection locations. If an additional site comes to mind once you are already collecting data, immediately communicate this additional information to PIRE before collecting data at a new site. This is a requirement as per PIRE’s Institutional Review Board approval of this data collection methodology. Violations of one’s approved protocol could have consequences for everyone collecting data.

Each program must decide what will work in your community. We ask that you carefully consider if a location or event may bias the responses you are getting. For example, by targeting a health fair, will you unfairly bias your responses to those who engage in “more healthy” behaviors? Or if you go to an event where there is a beer garden, will that influence the responses you get unfairly? In this example, bias seems possible to us, but you know your community best and you must make that decision after careful consideration. Ideally, you don’t want your survey to reflect the opinions only of those who are most law-abiding and alternatively, you don’t want your survey results to reflect the opinion of those who are most likely to have higher rates of ATOD. That is why you should strive for getting as random and representative sample as possible.

“Paper and Pencil” versus “iPad App” versus “Online Browser” versions of the survey

The community-level data collection can entail any combination of the paper and pencil, iPad or online browser versions of the survey, as long as it is through an approved data collection venue.

• The advantage of using the app or online version is that you do not have to enter data yourself. Some participants may prefer the ‘quicker’ version of the app, while others will feel reticent to commit personal responses to the internet. Therefore, always be prepared to administer the paper version as well.

• All versions of the survey are available in Spanish and English.

• When administering any version of the survey, the administrator should enter the site ID prior to the respondent completing the survey. In this way PIRE can quickly tally surveys collected from your site.

• If you use either the app while you are on-line or browser version on-line, participants will be eligible for the online cash gift drawing. If you are not connected to the internet you will be unable to access the gift drawing website. NOTE: PIRE recommends using the Qualtrics app offline and using the URL or QR link to the survey if you have a strong internet connection.

• You may choose to send a link to the survey out to a listserv that you feel would be an appropriate group of potential respondents. This is allowed but it should be well documented in your protocol (e.g., describe the listserv and the number of people on the listserv or the population that the listserv serves (e.g., patrons of a local public library) should be noted.)

• A QR code for the survey is available above. This can be posted on a sign at recruiting locations so that people with Smartphones may take the survey at their leisure. Please note if you will do this in your protocol and at which locations you will use this.

Key issues for implementing the survey using the qualtrics app

• Please note that PIRE recommends using the Qualtrics app offline when your internet connection is problematic or non-existent. If you have a strong and consistent internet connection, consider using the URL and QR for the survey instead.

• If participants complete the survey using the Qualtrics app offline, they will not be able to enter the gift-card drawing. Please let them know this upfront and consider providing a different incentive.

• Fully charge your device and test the app before beginning data collection. Also, store the password “PIRE2020!” and login information separately from the device; if the device is stolen, the data will be protected.

• When administering the survey in person, you will select for the respondent that you are “administering the survey to someone else.” You will then go to the next screen, where you will input the site ID, an abbreviation for the location of data collection, and the data collector’s initials. After entering all the preparatory information into Qualtrics, you will give the iPad or other device to the participant to complete and submit the questionnaire.

• Explain to participants that nobody can go back into the survey once the final screen is reached, and even though you cannot go back, instruct participants to close this final screen when they are done with survey, so they feel secure that their data will not be shared.

• Upon completion of the survey, the person completing the survey reaches an ‘end’ screen and the survey will be sent to the app’s database. If a participant decides to stop taking the survey before reaching the end, the surveyor should instruct the participant to simply click ‘next’ until the app reaches the ‘end’ screen. If the participant decides that s/he wishes to withdraw participation altogether, then they can choose to close the survey (“X” in the upper right corner).

• Please follow instructions on pages 25-26 for uploading survey data at the end of each data collection day or event.

• If you wish to install the app on your own iOS or Android device, follow directions above and contact PIRE if you have any questions or problems.

• OSAP iPads do not come equipped with wireless internet access. If you wish to have access to the internet for data collection you must purchase this separately and ensure that the cost is not prohibitive to purchase a separate hotspot or to use the hotspot on your own phone. Make sure to consider the cost of a data plan to allow you to use the internet if using the tablets.

• If experiencing problems with the app, contact PIRE. If experiencing problems with the tablet or hardware, contact Coop Consulting.

• If any device with the app installed is lost or stolen, inform PIRE immediately. If any OSAP iPad is lost or stolen, inform Coop Consulting as well.

• When collecting data in this fashion, you must use a data collection log to track time and venue information separately (date, start and end time of data collection, location, total hours onsite, number of data collectors, # of surveys completed) as you do with paper surveys.

Key issues for implementing the survey on-line using the URL or QR code

• Please note that PIRE recommends completing the survey on-line using the tinyURL or QR code rather than the Qualtrics app because no upload is required. If you have strong and consistent internet access (not intermittent) we strongly recommend that you use the URL and/or QR code to link respondents to the survey.

• When administering the survey in person, you will select for the respondent that you are “administering the survey to someone else”. You will then go to the next screen, where you will input the site ID, an abbreviation for the location of data collection, and the data collector’s initials. After entering all the preparatory information into Qualtrics, you will give the tablet or laptop to the participant to complete and submit the questionnaire.

• Explain to participants that nobody can go back into the survey once the final screen is reached, and even though you cannot go back, instruct participants to close this final screen when they are done with survey, so they feel secure that their data will not be shared.

• Upon completion of the survey, the person completing the survey reaches an ‘end’ screen and the survey will be immediately uploaded to PIRE’s database. If a participant decides to stop taking the survey before reaching the end, the surveyor should instruct the participant to simply click ‘next’ until the app reaches the ‘end’ screen. If the participant decides that s/he wishes to withdraw participation altogether, then they can choose to close the survey (“X” in the upper right corner).

• If participating on the survey while online, participants will be able to go to a separate website to enter their contact information to enter a lottery for a $100 or $500 check when they have completed the survey. Participant contact information for entering the raffle/lottery is NOT associated with survey data.

Adding Questions

In previous years, some communities have added additional questions at the end of the survey that are unique to the community. These questions MUST be approved by the SEOW prior to inclusion so they should be included in your submitted protocol. Typically, these have been questions related to alcohol tax increases or issues very specific to your community that might not be relevant to other communities and therefore, are not included on the general survey. You may again include some additional questions on the survey at the end of the survey (after core module and other selected/approved modules). You need to identify these as questions specifically requested by the prevention provider on the paper survey. For example, we will add these questions to the Qualtrics app and we will preface with the questions with a statement like:

“The last five questions are asked by the X County Partnership and are specific to your county. Your answers help us plan and improve services.”

On the paper survey, you will need to add an identical “preface’ to the additional questions. Respondents will be “screened into” the additional questions based on their site ID or county so that only those living in the relevant community will be offered the additional questions.

Most importantly, the addition of any questions must first be reviewed and approved by the SEOW when you submit your data collection protocol for review. PIRE reserves the right to not include additional questions to the Qualtrics instruments if adding them becomes too great a burden on staff and resources.

Time and Venue-Based Data Collection Concerns

As you have rigorously tracked your data collection by time, date and site last year, you will have a basis upon which to collect data this year. In this way, your sample will ideally reflect last year’s sample closely with respect to age, gender, and race/ethnicity. Begin your planning by looking at your sample from last year. Where are the gaps? Who is over-represented or under-represented when you compare your sample to that of the census or other sub-population measures? How can you improve on the process this year? Then return to your data collection log and determine with your local evaluator where and when will be the ideal times and places to collect data. Do you need to change what you did last year in any way?

We are aware that it will be impossible to exactly duplicate your process each year, and that there will be circumstances that are beyond your control. There may be a new MVD manager who doesn’t want you to collect surveys there, or Walmart may only allow you to collect data on limited days, when before you were offered any day you chose. Document these issues as you go along, so to always improve your process.

Timing and community culture: Cultural issues and events may, or may not, affect your target community and the timing of your data collection. These particular events can affect your data collection, or bias your responses, making substance use and mental health appear better or worse for your population. Do people in your community practice Lent and do many abstain from alcohol during that time? For college communities, when is Spring Break and when are mid-terms, finals and graduation? Is there a youth sports season that you can take advantage of in a small rural community to collect data? Feast days or festivals? How might this change from year to year? How might your data collection and data quality be affected by these events? These are important considerations to plan for in advance but also to consider when interpreting your data findings.

Incentives: The most typical incentives are a small snack (candy bar, snack bar, or fruit), bottled water, or small amounts of cash that demonstrate novelty like Sacajawea $1 coin. We have found that $1 bills are good incentives for younger adults. (Be aware that using cash incentives will require additional tracking, but PIRE can help you with this). Please also be aware that as of December 2019, using OSAP funds for cash incentives over $1 and gift card amounts over $2.50 (OSAP prefers less when possible) is no longer allowed by OSAP. Obtain something that is culturally appropriate and won’t get you into trouble as a public health promoter. Some have used a random incentive, where every 10th or 20th participant receives a larger prize such as a gift card. For example, you can place a Post-it on every 10th paper survey that says the participant is a winner. Others have used coupons from local businesses as incentives. This works nicely if you are collecting data in that venue. If an incentive has worked well for you in the past, use it and let PIRE know so we can share your success.

DO NOT use public lotteries or other methods that that will reveal participants’ identities.

A cash check lottery is available only to those who participate in the online survey. Those using the app when it is not online cannot participate in the on-line lottery. These participants can only receive an incentive from you.

If you need assistance with your agency in obtaining approval for incentives, please contact PIRE for help.

Protecting Human Subjects

It is of utmost importance that you protect the anonymity of the respondents’ data. You should provide a collection system that will allow people to provide their completed surveys to you without you, your coalition/agency or anyone else being able to identify it. Typically, if a lot of people are completing the survey, this is less of an issue. However, if you only have a few completed surveys at a given location, then you may well be able to identify the respondents, particularly in small communities. In that case, you must explain this technique in detail in your protocol so that reviewers will understand the process you are putting in place to protect the anonymity of respondents.

When using the online or app versions of the NMCS, make sure that the app and the web page are closed at the end of each survey, which is best done by the participant her or himself. Please ask the respondent to close the app and webpage at the end of the survey. If the respondent needs assistance, you may help them close it. This adds another level of security and protection to the respondent’s data.

Collecting data with a mobile device may require greater vigilance to protect against theft. Make sure that the person using the device is being observed by someone on your data collection team, but at a safe distance so they do not feel that their privacy is compromised.

Data Collection Logs

Data collection logs are your means to document where and when you collected data and how much time it typically takes to collect your surveys at different community locations (data collection rates). Make sure that at the end of each data collection event, that the data collector(s) document and record the place, time, data collectors name or initials, and number of each kind of survey collected (e.g., paper vs app, Spanish vs English language), plus any issues they had with collecting data. The app tracks the number of surveys collected and uploaded, but to be sure of your progress, you should track separately as well. The more information you provide on these logs the better. Think of the sustainability of your program here, as you may not be around to collect data next year. Each year we will ask that you submit this log to PIRE with your final data report. And each year, we will also ask that you consider it when developing your protocol next year. During the data collection time frame, PIRE will provide you with on-going updates on the number of on-line surveys by county and site ID on approximately a weekly basis.

|NMCS 2020 Protocol Guidance for Budget Planning |

|This document is for community planning purposes only. This was written by PIRE to guide the protocol development process and is not intended to be a direct message |

|from OSAP. |

|Direct (In-Person) Data Collection Decision Tree |

|note: assume 300 surveys to collect |

|NM Community Survey participant recruitment occurs in-person in local communities and through targeted social media advertising. This decision tree is designed for |

|in-person, community-based recruitment efforts. |

| |

|App / Online Collection Focus | |Considerations | |Paper Collection Focus |

|  |  |  |  |  |

|College population | |Population | |Older participants |

|Young adults | | | |Technology resistant |

|Generally works best when not provided other | | | |Can feel more private |

|options | | | |Less concern about theft |

|  |  |  |  |  |

|$150-$750 (.5 - $2.50 per survey); incentive | |Incentives: Small Cash Incentive to take the survey (max of | |$150-$750 (.5 - $2.50 per survey) |

|for online participants = option to enter | |$1 cash or $2.50 gift card from OSAP or other resources) vs | | |

|weekly $100 drawing entry); extra monthly cost | |Other Incentive (snack or water or token item). Cash | | |

|if hot spot used for online data collection | |incentive requires extra tracking on additional form. | | |

|  |  |  |  |  |

|30-50 hours total ONSITE (assume at least 3 | |Data collector TEAMS to sites (look at your data collection | |30-45 hours ONSITE x 2 people (more people can |

|devices) x 2 people | |log from 2019 to determine your data collection rate) | |complete at same time w/o limited devices) |

|  |  |  |  |  |

|2-5 hours (laptop battery time consideration!);| |Length of time Onsite each data collection time | |2-5 hours depending on stamina and people flow |

|possible extra expense for hot spots | | | | |

|  |  |  |  |  |

|count on 30-60 minutes round-trip each time out| |Travel time to sites | |count on 30-60 minutes round-trip each time out |

|for data collection | | | |for data collection |

|  |  |  |  |  |

|count on .5 - 1.5 hours each data collection | |Administrative time (prep for data collection, wrap up after | |count on .5 - 1 hour each data collection shift |

|shift (if using tablet or Ipad includes data | |each data collection shift | | |

|upload, login and out of app) | | | | |

|  |  |  |  |  |

|None | |Data Entry | |many hours! requires strong supervision / |

| | | | |guidance / oversight (est 20 hours) |

|  |  |  |  |  |

|Minor | |Data Cleaning | |yes, evaluator to estimate |

| | | | |(est 1-5 hours) |

Regarding Expenses: Assessment: Data Collection Expenses & Time Estimates

OSAP requires submission and approval of your budget work pages. Please send to heather.burnham@state.nm.us and antonette.jose@state.nm.us and karen.cheman@state.nm.us.

Data Collection Training for collectors (from Evaluator): Includes time to train data collectors (ie: paper and/or online procedures, onsite prep, tracking and securing paper surveys, uploading app surveys).

o Time estimate for this task: Budget approximately 1.5 hours per training plus any travel for data collectors

OSAP Conversations/correspondence regarding billing & Budget: Incentive and stipend approval. You must submit your budget to OSAP for review and approval.

o Time estimate for this task: Budget approximately 1-3 hours

Survey Administration Expenses: this varies depending on data collection site (rural vs urban community), skill of data collectors, day of week, time of day, and whether collecting on paper surveys or with Ipads / laptops.

o Time estimate for this task: Average of 10 surveys collected per hour for team of 2 (range 6-13/per hour for teams of 2 data collectors). So, budget at least 30 hours ONSITE for a team of 2 people (60 people-hours) to collect 300 surveys at different sites. Note: this could be 10 3-hour shifts for a team of 2 or 15 2-hour shifts for team of 2, 7.5 4-hour shifts for team of 2, etc. See excel table on page 4 for examples of summary data collection log data.

Data Entry Expenses: number of hours required to enter 300 surveys; less if using Ipads

o Time estimate for this task: Budget approximately 20 hours for skilled data entry. If training someone in data entry, can cost additional local evaluator time in oversight and correction.

Data Cleaning Expenses:

o Time estimate for this task: Budget approximately 1-5 hours

Other Data Administration Prep Time: time coordinating data collection sites, time recruiting data collectors, making copies for paper surveys & take-home cards, leveraging or purchasing incentives (gift cards, snacks, waters) putting up signage in the data collection site, purchase of materials (secure boxes, envelopes, pens, tables, clipboards, tape),

o Time estimate for this task: Budget approximately 10 hours

Post Data Collection: coordinating with Local Evaluator to pick-up, deliver paper surveys, time with local evaluator (ie: questions about surveys, completion of tracking sheets), obtaining paper surveys from LE; Small item to show appreciation to supporting entity (bouquet, staff fruit basket, card, newspaper ad acknowledging site, certificate or letter of appreciation).

o Time estimate for this task: Budget approximately 2-3 hours if travel is involved

Data Collection Capacity and other expenses:

The NMCS was designed in part to build local capacity in data collection. Ideally, participants from data protocol developers, data collectors, data entry staff learn through this process so they can build their own professional and individual capacity with data.

However you need to consider additional variables.

□ Additional compensation for bilingual data collectors

□ Additional compensation for seasoned data collectors that can help support a data collection site in lieu of staff or can help train others

□ Opportunities to obtain data collectors from other entities? Neighboring OSAP coalitions? Other colleagues from your same organization? (who can also use this data for their own programming)?

□ Additional training and oversight costs? Volunteers are wonderful, but they must be willing to be trained and follow the protocols with rigor; be sure that ALL your data collectors are well trained and well supervised, using the instructions and template provided by PIRE. (Also, consider that bad data entry can sabotage careful data collection efforts and statewide results).

Materials

• Cost of printing: takeaway document, hand-outs or posters or flyers with tinyurl encouraging survey participation, paper surveys (+ modules) as needed (all in English and Spanish)

• Purchase of Pens, clipboards, and box to secure surveys

• Brightly colored vests, similar attire or other means to identify surveyors from larger population.

• Table / Ironing Board, colored tape or other means to demarcate space, chair, Ipad stands (check with OSAP if can be purchased)

• Ipads or laptops for data collection (look for in-kind, donations or contributions; cannot be purchased with OSAP funds).

• Small item to show appreciation to supporting entity (bouquet, staff fruit basket, card, newspaper ad acknowledging site, certificate or letter of appreciation).

Other Considerations Regarding Expenses and/or Topics to Discuss IN ADVANCE with your OSAP program manager. Submit your budget to OSAP for Review & Approval

• OSAP allowable cash incentive or gift certificate for community member time to participate in survey – maximum of $1 cash or $2.50 gift certificate per person.

• OSAP allowable small incentive such as…water, granola bars, candy, etc for community members who complete a survey – typically costs less than $1/person

• Note: A stipend can be used to compensate a trained data collector for their time assisting in data collection under program direction and following approved community protocol

• Hot spot or cell data monthly fee – approximately $30/month per hot spot, approx. 2 months of data collection = $60 - $120

• Always seek to do this work with volunteer or in-kind support first. Document your efforts and decision-making process in your protocol.

• Seek donations for your incentives. This is often challenging but worthwhile as you obtain more community visibility or support. For example, if you offer a coupon to use at a local business or your college food court, if the local business cannot provide the full value of the coupon, consider offering part.

• You can pay for staff time to collect data through the OSAP grant, but do so to the degree possible through your or your local evaluator’s regular compensation structure. If you are unable to do so, check with your OSAP program manager about how to appropriately compensate.

• Think about optics and ethics: how could it be perceived for a government-based program to offer a cash stipend in a community that is undergoing severe economic turmoil? Could using certain staff, stipends or incentives appear to benefit your organization, someone in political leadership, or specific economic interests in ways that could seem unethical?

Sample Data Collection Rates – from 2018 Data Collection Logs

|Community |Total Surveys |Total Hours |Rate: surveys collected per hour |Type |

|Roosevelt |383 |48 |7.98 |paper |

|Grant |245 |20 |12.25 |paper & app (54) |

|Luna |151 |26 |5.81 |paper & app (7) |

|Taos |355 |42 |8.45 |paper |

|UNM |306 |24 |12.75 |app |

|Bernalillo |462 |34.5 |13.39 |paper & app (2) |

|SJC |114 |12 |9.50 |paper rate only! |

| | | | | |

| |2016 |206.5 |9.76 | |

Summary of Responsibilities for Prevention Staff and Community Partners

• Attend annual data collection (webinar) training in December 2019 or watch recorded presentation on .

• Complete local data collection protocol and submit by deadline to PIRE for SEOW review and approval before or by January 22, 2020. Respond to revision requests from SEOW, and initiate data collection only upon SEOW approval.

• Complete and submit budget form to OSAP for approval; and submit all budget revisions for approval to OSAP.

• Identify a goal for the number of surveys required: PIRE can provide support if power analysis needed. Typically, between 300 and 400 is ideal for most communities.

• Contact all targeted MVD supervisors, grocery store managers, school administrators, etc. getting permission to recruit and/or provide paper surveys at those locations and determining dates and times for this to occur. If an IRB review of the protocol is required (especially for campus or Tribal based data collection), you are responsible for following their required process.

• Print surveys and the (personalized to your community) take-home information in English and Spanish.

• If using an OSAP-iPad, you are responsible for taking care of the hardware and returning it to Coop Consulting. All hardware issues with OSAP iPads are the responsibility of the program and Coop Consulting, not PIRE.

• If using the internet version for the survey, assure that internet access is available for your program tablet, mobile device, or laptop.

• If using advertising for the survey, submit all drafts to PIRE for SEOW approval and review.

• Visit locations ahead of time to meet with managers and determine where & how to set up the “recruitment stand.”

• Print your site ID on your paper surveys before copying (if possible). Assure that site ID, surveyor name or ID, date and survey number are recorded on each survey. This includes the app and online browser versions for the survey. Use and update your data collection log every time you go out to collect data in-person. This helps you keep track of number and location of surveys collected.

• Provide a box for collection of paper surveys with a lid to protect anonymity of responses and respondents.

• Consider a small “up front” incentive to those completing the paper survey and the off-line version of the survey using the Qualtrics app. Use extra tracking form for any cash incentives.

• In areas where Spanish is required, provide sufficient Spanish language materials and an individual who can collect surveys who is able to communicate fluently in Spanish. (Same for any other languages such as Navajo, etc.)

• Attend survey data entry webinar training.

• Enter data and submit clean data in spreadsheet template provided.

• Maintain contact with PIRE evaluators and program managers as to the progress, obstacles, etc. of the data collection process.

• Take the initiative to communicate with PIRE about shifting approaches if goals are not being reached. COMMUNICATE WITH PIRE ABOUT ANY NEW SITES PRIOR TO COLLECTION.

• If an outside IRB (e.g., Tribal, University) requires changes to an already PIRE IRB-approved protocol, program staff must report these changes immediately, before changes are implemented, to PIRE staff, who are then required to inform the PIRE IRB about these changes.

• DOCUMENT ALL SITES WHERE DATA ARE COLLECTED and submit Data Collection tracking log at EOY reporting.

• Work to reach goal # and desired demographic representation. Consider tracking gender or age or other demographics periodically during data collection to monitor progress towards representative sample or desired over-sampling goal/s.

• Provide weekly updates to PIRE regarding progress.

• Safely store all collected surveys paper or electronic data for 5 years following data collection as mandated by Federal law.

• Analyze data for community/program needs once PIRE has returned cleaned data.

• Submit findings sheets to OSAP at EOY reporting.

Summary of PIRE’s Responsibilities

• Supervise SEOW review of convenience sample protocols and advertising content and provide feedback and final approval to local programs

• Provide power analysis to identify required number of surveys for each subpopulation for strong evaluation results.

• Monitor data collection efforts and respond immediately to changes in data collection protocol requests.

• Apply for IRB approval and review annually. Monitor and report human subjects’ violations IMMEDIATELY to PIRE IRB.

• Share PIRE IRB application and approval materials for those programs requiring additional documentation for other IRBs. Any local data collection that requires another IRB review and approval must proceed according to that IRB’s recommendations. PIRE will work with programs to help provide any necessary information and assurances and will consult with PIRE IRB if questions of IRB conflict arise. If an outside IRB (e.g. Tribal, University) requires changes to an already PIRE IRB-approved protocol, PIRE is required to inform the PIRE IRB about these changes.

• Provide Qualtrics app and browser-ready versions of all surveys & accompanying materials (take home card templates) in English and Spanish, data entry templates and syntax.

• Provide training on data collection protocol & recruitment strategies to all prevention providers and local evaluators

• Provide technical assistance and support

• Collect data online via social media advertising.

• Provide complete dataset (that includes online and locally-collected data) back to programs. These data include those collected at another site, where the participant indicates residency in another NM location, and data collected online.

• Provide findings sheets to those communities where PIRE is the local evaluator (local evaluation contracts).

A Note about Budgets

EACH COMMUNITY MUST SUBMIT BUDGET TO OSAP

FOR REVIEW & APPROVAL

You are not required to turn your budget in with your protocol to PIRE, but OSAP requires submission and approval before data collection can begin in your community.

Heather sends out the necessary forms annually and PIRE posts them to the NM Prevention Website here:

For FY20, Budgets are due to Heather no later than 2/5/2020 at heather.burnham@state.nm.us

Community Data Collection Protocol

Turn in to PIRE for SEOW Review by January 22, 2020

Program Name: Date:

Original or Resubmission:

Name of program representative(s) who attended online data collection training:

1) Please provide a general description of the geographic area & population to be affected by the interventions and therefore, surveyed. Provide any information that might influence where & how data are collected in your community (e.g., any sub-populations you will specifically try to over sample; native lands with few locations where tribal members gather regularly; native and non-native lands that will require separate data collection strategies to reach.)

2) How many completed surveys are needed for your program? (You should aim for between 300 & 400 in most cases. If less than 300, please explain.)

3) You must seek to repeat your previous year’s data collection to the degree possible. At the same time, we ask you to correct issues that in the previous year could have made your sample less representative of your community. What issues, if any will you address this year to correct for last year, and what areas may be difficult to collect data as you had experienced last year? (Make sure to attach last year’s data tracking log to this year’s submission!!!!)

4) Please list all those who will be involved in the data collection/recruiting process. Start with the primary contact person for data collection (PIRE will be calling the Primary Contact Person on a regular basis to check in.) Only those 18 years or older can collect surveys.

|Name of Staff Person or Volunteer |Email Address |Phone # |Role (e.g. supervisor, |18 years old or|

| | | |trainer, data collector, |over? |

| | | |data entry, etc.) |Y/N |

| | | | | |

| | | | | |

| | | | | |

| | | | | |

5) Who will train those responsible for collecting surveys about the data collection protocol and approved data collection sites and approaches? When will training take place?

6) Do you plan to provide small up-front incentives? If yes, please tell us about the small incentives (e.g., type of food, amount of money.) Remember the OSAP guidance regarding maximum allowable incentives ($1 cash or $2.50 gift card) Please see budget form and decision tree document for additional details. Will these be incentives you purchase or provide, or will you seek donation from local businesses? If you intend to seek donation, please elaborate on who, what, where, etc., you will seek donations?

7) Do you need to advertise or promote the survey? If no, why not? If yes, what are your plans for promoting the survey? Attach copies of advertising for review. For human subjects’ protection, no advertising can be used without SEOW approval.

8) Please explain how you intend to adequately protect respondent’s anonymity/confidentiality while taking the survey and returning the completed survey. If you prefer manila envelopes for paper surveys, please let PIRE know how many and we can bring some to the Recipient Meeting. After sealing the completed survey, respondents will then place their survey into a large box with a cover.

9) Do you need to have data collectors that can communicate in languages other than English? What is your plan for approaching these populations?

10) Are there sub-populations (examples: young adults age 18-25, Latinx, elders, LGBTQ) for whom you plan to collect data? Who are they and what is your plan?

11) It likely will be your evaluator who will be conducting or heading up the data entry for the paper and pencil surveys. Does anyone need to attend data entry training? Who? Who will keep the completed surveys locked and safe for 5 years?

12) Would you like for PIRE to provide additional assistance and/or training for your program prior to data collection?

13) You are required to include all the core survey questions as they currently exist, but you may add additional questions at the end if you choose. If you intend to add additional question, please provide a list of those questions and responses and the introductory statement to be used. Please also select/check the sub-modules you intend to use:

( methamphetamine ( opioids ( community ( college community

( marijuana ( adverse childhood experiences (ACEs) ( tobacco

( gambling ( mental health

14) You may wish to check out an iPad from Coop Consulting in order to use the Qualtrics app to collect data offline or to complete the survey online. You can also upload the survey app onto your own iOS or Android device or use your own internet connected device to complete the survey online. Finally, you may use the QR code to take people to the survey. The benefit of these approaches is that there is no need for data entry. Briefly describe your plans for using the app or online data collection. (Note that a detailed management protocol for iPad & app use is provided below.)

15) Do you need special permission to collect data anywhere? Besides obtaining permission from private businesses, typically you need to address the IRBs of universities and Tribes. PIRE can help you with this process, so please inform us if this is your intent.

16) How will paper surveys be stored following data collection and in preparation for data entry? Describe a secure process of transport and storage.

17) Is there anything else the SEOW should know when reviewing this protocol? Is this a sampling approach that is familiar to your program staff, coalition and community? Are there significant changes from previous approaches that enhance your data collection approach?

18) We have worked hard to design a protocol that is as protective as possible for the research volunteers and, consistent with Federal regulations, we have received the approval of PIRE's Institutional Review Board (IRB) for this protocol (and possibly other IRBs). We are relying on you to ensure that you adhere to this protocol. If any deviations from the IRB-approved protocol occur, we require that you notify us immediately so that we can notify our IRB (or the involved IRBs) as required by the regulations.

Please assent that you have read and understand this statement. ______________.

19) I have sent my budget to OSAP for approval ______________. (initial here).

DATA COLLECTION ONLY CAN BEGIN AFTER PROTOCOL IS APPROVED

DATA COLLECTION DATES ARE FEBRUARY 24, 2020-APRIL 27, 2020

IF USING THE NM COMMUNITY SURVEY APP TO COLLECT DATA YOU MUST COMPLETE THE PROTOCOL FOLLOWING THE TABLE ON THE NEXT PAGE

Don’t forget to attach your last year’s data collection tracking form and COMPLETE the NEXT PAGE!

Instructions: Using your data collection log from the previous year, choose your sites. If a new site, be sure to mention in question 3 above. ATTACH LAST YEAR’s LOG TO YOUR SUBMISSION. We acknowledge that your data collection log will not match this exactly, especially in numbers, dates, and times, but any NEW SITES MUST BE APPROVED BY PIRE.

19) List population or subpopulation targeted on separate rows

(e.g., young adults age 18-25)20) List locations you propose to recruit adult community members to complete the survey. Provide any additional information that may be needed to understand the location if it is not clear how you collect data in that situation. 21) List times you propose to recruit residents (having planned time intervals will help increase the representativeness of your sample). 22) Using what you learned from your data collection log from last year, list your projected numbers for this site or event. This will help you be prepared with the appropriate number of paper surveys when you travel to your site. 23) Plan to bring an iPad with the survey app or an internet connected device? Mention which type of device.

(If using an internet connected device like a laptop, be sure you have access to the internet.)24) List the dates when recruitment & data collection will take place. (Base dates upon previous years’ protocol to the extent possible). 25) List who or how many people will be on site to recruit/collect data.26) Do you need permission to collect date in the location? Who will seek that permission? What is the “back up” plan if you cannot get permission? (If you elaborate above in Q 15 please just mention “see Q 15”)27) Who will be in charge of completed surveys and devices?

Goal # of English surveys collected# of Spanish surveys

Internet or app users                                                                            

Don’t forget to complete the next page if collecting data using the Qualtrics app!

COMMUNITY PROCEDURES FOR USE OF THE NMCS DATA COLLECTION QUALTRICS APP

Keep these two pages with you while collecting data with the NMCS data collection app.

Provider use of OSAP iPads

Each program checks out iPads from Coop Consulting, who manages their distribution and return on behalf of OSAP.

Only trained data collectors can use OSAP iPads or the installed app on a private device.

OSAP iPads can only be used for data collection purposes. Do not install any additional apps.

Do not leave devices in sight in a locked or unlocked vehicle.

Return devices to the prevention office as soon as feasible, preferably immediately following any data collection.

When data collection is completed, you are responsible for returning the iPads to Coop Consulting in a timely manner and in good condition.

If any OSAP iPad is lost or stolen, the provider must immediately inform Coop Consulting (505- 424-4365) and PIRE (Dave Currey at 505- 765- 2330 or 919-265-2622).

Using the NMCS app on a private device

If you wish to install the NMCS app on your own device, contact PIRE so we can provide you the most up to date information. Even if using a private device, you must follow the procedures for the Qualtrics app described below.

NMCS App Protocol

KEEP WITH YOU AT ALL TIMES AND REVIEW PRIOR TO COLLECTING DATA USING THE APP

When possible, always collect data while connected to the internet using a browser rather than the Qualtrics app. When connected to the internet (and using a laptop or computer or tablet that is accessing the survey via the website location), the participants’ data can be immediately submitted, and participants can opt to participate in the online incentive, which means the provider may not need to provide an incentive. This is the most secure way to collect the data.

When collecting data using the app, data are stored on the tablet itself. Please note the data will need to be uploaded when you complete your data collection for the day, as soon as you are able to connect to the internet. Once the data are uploaded, they will continue to be stored on the tablet until you “log out” from the app (please see detailed instructions). We recommend logging out immediately after uploading your data. When using the tablet to collect data off-line, your program may want to provide a small upfront incentive since participants will not be eligible for the on-line drawing.

When using the app to collect data:

Instruct the respondent on how to navigate the survey instrument and how to close the survey at the end. Upon completion of the survey in both the online and app version, the survey participant will be sent to an ‘end’ screen and the survey will be sent to the app’s database. Explain to participants that nobody can go back into the survey once this final screen is reached, and even through you cannot go back, instruct participants to close this final screen when they are done, so they feel secure that their data will not be shared.

If a participant decides to stop taking the survey before reaching the end, the surveyor should instruct the participant to simply click ‘next’ until the app reaches the ‘end’ screen. If the participant wishes to withdraw their participation altogether, the ‘close’ button will allow them to withdraw.

Keep an eye on the individual using the device (at a respectful distance to maintain privacy) to deter theft. You may let them know that you will be nearby to make sure they are safe and to answer any questions they may have but will be respecting their privacy.

When your data collection is completed for the day and you have wireless access to the internet, immediately upload the surveys from inside the app. You may do this at home or at the office. But do not leave the data on the tablet for long.

After uploading data, you should see the number of pending responses reduce and the number of uploaded responses increase. Please note that these numbers may be different than the total number of surveys you believe have been completed/collected.

From the main app screen after uploading survey responses, go to the settings icon in top right corner of screen. Select “Log Out”. If you have already uploaded all the completed surveys, ignore the warning that says that logging out will delete all responses and proceed. You want survey responses to be deleted from your device after they are uploaded.

All devices using the app will be set to log out within 3 minutes of inactivity. Do not share the passwords for devices except with those trained to use them for data collection.

If theft occurs, do not put yourself at risk in order to retrieve it. Report to authorities, Coop Consulting, and Dave Currey @ PIRE immediately.

Use the device only in locations where you can maintain control over the interactions and environment and feel safe allowing people to use them.

Do not leave devices unattended at any time.

Here write your own procedures for use of the app, answering the following questions:

How many IOS (Apple) or Android devices will have the app installed? ___

How many of these devices are an OSAP iPad? ___

Who will be in charge of tracking the use and storage of the devices? _______

Who will be trained to use the devices and who will do the training? _______

How will devices be securely stored when not in use? _______

Will any data be collected using the app offline? ___

Based on your data collection protocol, where are the locations this likely to take place?

Where will data collectors go to upload the data collected offline?

For SEOW USE only:

Date of review (include this text in subsequent reviews):

SEOW REVIEW CHECKLIST

When reviewing the data collection protocols submitted by local prevention providers, please consider how well the following topics are addressed.

Does the provider seem to have a good understanding of the population in his/her community? Have s/he described any idiosyncrasies of their community that might influence the data collection process?

Has the provider identified any subpopulations s/he wishes to focus on during data collection? What are they? Does the provider intend to over-sample young adults aged 18-25? Based upon your knowledge of the program, are there any subpopulations they should consider?

Is the number of surveys the provider intends to collect realistic given their person resources? Why not?

Are sufficient details provided in the table on who will be involved, the roles that each will play and where and how data will be collected so that you feel you could replicate their strategy easily? What is missing?

Has the provider demonstrated adequately how s/he will protect respondents’ anonymity/ confidentiality? Please pay particular attention to this if the program is indicating they will administer the ACEs module.

Do you have any suggested changes to improve privacy and anonymity of those participating, the representativeness of sample, or to increase number of targeted respondents?

Are the incentives offered appropriate?

Any additional comments or concerns?

Do these changes require resubmission?

Has the program indicated that they have read question 18 and signed it?

If using the Qualtrics app to collect data, have they completed the protocol form sufficiently?

If the program is using additional questions, are they sound?

Is this program “good to go?”

PIRE Contact Information

Dave Currey 919-265-2622

dcurrey@

Ashley Simons-Rudolph - 919-265-2620

asimons-rudolph@

Marissa Elias – 575-202-2232

MElias@

Kim Zamarin 505-907-0801

KZamarin@

-----------------------

Protocol for the NM Community Survey

Data Collection Protocol and Planning Template

This document is intended to be used by NM OSAP grantees expected to collect community-level data using the NM Community Survey. PIRE recommends using a rigorous time and venue-based sampling methodology to provide as much rigor as possible to convenience-based sampling. The document provides recommendations for venues to conduct data collection and a template for planning purposes.

[pic]

2020

Written by: Pacific Institute for Research and Evaluation

for the NM Office of Substance Abuse Prevention

You may locate the NM Community Survey and corresponding documents here: .

Please make sure you have the 2020 documents. Note that the tiny url and QR code change each year!

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download