TABLE OF CONTENTS - Ten to Men | Ten to Men



2013 Major Report Laying the Foundation: Testing, Evaluation and Refinement of the Wave 1 Study InstrumentsRachel Koelmeyer, Dianne Currier, Marisa Schlichthorst, Jane Pirkis and Dallas English2013 Major Report Laying the Foundation: Testing, Evaluation and Refinement of the Wave 1 Study InstrumentsRachel Koelmeyer, Dianne Currier, Marisa Schlichthorst, Jane Pirkis and Dallas EnglishTABLE OF CONTENTS TOC \o "1-3" \h \z \u LIST OF TABLES PAGEREF _Toc424909238 \h 3LIST OF FIGURES PAGEREF _Toc424909239 \h 3ABBREVIATIONS PAGEREF _Toc424909240 \h 4ACKNOWLEDGEMENTS PAGEREF _Toc424909241 \h 4EXECUTIVE SUMMARY PAGEREF _Toc424909242 \h 51INTRODUCTION PAGEREF _Toc424909243 \h 61.1Ten to Men Overview PAGEREF _Toc424909244 \h 61.2Wave 1 Study Instruments PAGEREF _Toc424909245 \h 61.2.1Content Development Process PAGEREF _Toc424909246 \h 61.2.2Content Overview PAGEREF _Toc424909247 \h 82METHODS PAGEREF _Toc424909248 \h 102.1Pilot Testing Overview PAGEREF _Toc424909249 \h 102.2Evaluation of the Performance of the Study Instruments PAGEREF _Toc424909250 \h 162.3Refinement of the Study Instruments PAGEREF _Toc424909251 \h 173EVALUATION OF INSTRUMENT PERFORMANCE PAGEREF _Toc424909252 \h 183.1Missing Data Analysis PAGEREF _Toc424909253 \h 183.1.1Raw Data Elements PAGEREF _Toc424909254 \h 183.1.2Derived Variables PAGEREF _Toc424909255 \h 193.2Item Response Fractions by Page Number PAGEREF _Toc424909256 \h 213.3Item Response Fractions by Topic PAGEREF _Toc424909257 \h 233.4Implausible Value Checks PAGEREF _Toc424909258 \h 263.5Performance of Questionnaire Skip Logic PAGEREF _Toc424909259 \h 283.6Inconsistent Response Checks PAGEREF _Toc424909260 \h 293.6.1Within Study Instrument PAGEREF _Toc424909261 \h 293.6.2Between Study Instruments PAGEREF _Toc424909262 \h 313.7Evaluation of ‘Other, Specify’ Field Responses PAGEREF _Toc424909263 \h 323.8Item Format Checks PAGEREF _Toc424909264 \h 333.8.1Matrix-Style Questions PAGEREF _Toc424909265 \h 333.8.2Single Response Items Captured as Multiple Response Items PAGEREF _Toc424909266 \h 353.9Comparison between Pilot Phases PAGEREF _Toc424909267 \h 354REFINEMENT OF STUDY INSTRUMENTS PAGEREF _Toc424909268 \h 385IMPLICATIONS FOR WAVE 1 PAGEREF _Toc424909269 \h 516REFERENCES PAGEREF _Toc424909270 \h 527APPENDICES PAGEREF _Toc424909271 \h 537.1Lists of Specific Questionnaire Items PAGEREF _Toc424909272 \h 537.2Additional Documents PAGEREF _Toc424909273 \h 62LIST OF TABLES TOC \h \z \t "Table,1" Table 1.Domains and Constructs Covered in the Wave 1 Study Instruments PAGEREF _Toc424909274 \h 8Table 2.Summary of Data Items By Study Instrument and Field Type PAGEREF _Toc424909275 \h 9Table 3.Household-level response fractions for the drop and collect pilot PAGEREF _Toc424909276 \h 11Table 4.Individual-level response fractions for the drop and collect pilot PAGEREF _Toc424909277 \h 12Table 5.Number of Participants Recruited in Wave 1 Pilot Studies1 PAGEREF _Toc424909278 \h 13Table 6.Socio-demographic Characteristics of Adult Respondents PAGEREF _Toc424909279 \h 14Table 7.95% Confidence Intervals for Various Proportions By Sample Size PAGEREF _Toc424909280 \h 16Table 8.Summary of Missing Data Analysis By Study Instrument1 PAGEREF _Toc424909281 \h 18Table 9.Summary of Missing Data Analysis for Variables Derived from Composite Data Elements1 PAGEREF _Toc424909282 \h 19Table 10.Proportion of Missing Data for Anthropometric Measurements PAGEREF _Toc424909283 \h 20Table 11.Missing Data Characteristics of Variables Derived from Validated Measures1 PAGEREF _Toc424909284 \h 21Table 12.Adult Questionnaire - Variable Response Fractions By Topic PAGEREF _Toc424909285 \h 23Table 13.Parent Questionnaire - Variable Response Fractions By Topic PAGEREF _Toc424909286 \h 24Table 14.Young Men Questionnaire - Variable Response Fractions By Topic PAGEREF _Toc424909287 \h 25Table 15.Implausible Values for Time Goes to Sleep and Wakes Up Fields PAGEREF _Toc424909288 \h 27Table 16.Summary of Missing Data Analysis By Study Instrument# PAGEREF _Toc424909289 \h 35Table 17.Summary of Recommendations and Actions Taken for Each Study Instrument PAGEREF _Toc424909290 \h 38Table 18.Boys CAPI Refinement - Recommendations and Actions Taken PAGEREF _Toc424909291 \h 39Table 19.Parent Questionnaire Refinement - Recommendations and Actions Taken PAGEREF _Toc424909292 \h 41Table 20.Young Man Questionnaire Refinement - Recommendations and Actions Taken PAGEREF _Toc424909293 \h 43Table 21.Adult Questionnaire Refinement - Recommendations and Actions Taken PAGEREF _Toc424909294 \h 46Table 22.Items Composed of Composite Data Elements PAGEREF _Toc424909295 \h 54Table 23.Validated Measures in the piloted Wave 1 Study Instruments PAGEREF _Toc424909296 \h 55Table 24.Conditional Sections in the Self-Complete Questionnaires PAGEREF _Toc424909297 \h 56Table 25.Numeric Write-in Fields with Multiple Response Options PAGEREF _Toc424909298 \h 57Table 26.‘Other, specify’ fields in the piloted Wave 1 Study Instruments PAGEREF _Toc424909299 \h 59Table 27.Single Response Variables Captured as Multiple Response Variables PAGEREF _Toc424909300 \h 61LIST OF FIGURES TOC \h \z \t "Figure,1" Figure 1.Study Instrument Development Process PAGEREF _Toc357755698 \h 7Figure 2.Median Variable Response Fraction by Study Instrument and Page Number PAGEREF _Toc357755699 \h 22Figure 3.Layout of Time Goes to Bed and Wakes Up Fields in the Parent Questionnaire PAGEREF _Toc357755700 \h 28Figure 4.Examples of Questions with Multiple Response Options PAGEREF _Toc357755701 \h 30Figure 5.Examples of Matrix-style Questions with Yes/No Response Options PAGEREF _Toc357755702 \h 34Figure 6.Adult Questionnaire Median Variable Response Fraction by Page Number PAGEREF _Toc357755703 \h 37ABBREVIATIONSAbbreviationsCAPIComputer-Assisted Personal InterviewOTCOver-the-Counter MedicationACKNOWLEDGEMENTSWe thank the Ten to Men Study Team, The Study Steering Committee, the External Scientific Advisory Group and the Department of Health and Ageing for their contributions to the development of the Ten to Men Wave 1 instruments.The following members of the study team contributed to the development of the Ten to Men Dress Rehearsal Analysis Plan and Questionnaire Refinement Process: Rachel Koelmeyer, Dianne Currier, Marisa Schlichthorst, Matt Spittal, Jane Pirkis and Dallas English.Rachel Koelmeyer drafted the analysis plan and conducted the analysis with support from Caroline Sumpton and Kieran Knight. Roy Morgan Research conducted the Ten to Men pilot studies, on which the evaluation and refinement of the Ten to Men study instruments is based.Ten to Men is funded by the Commonwealth Department of Health and Ageing. EXECUTIVE SUMMARYTen to Men: The Australian Longitudinal Study on Male Health will be the first longitudinal study of male health of its kind in Australia. The study is funded by the Commonwealth Department of Health and Ageing under the 5th priority area for action in the 2010 National Male Health Policy: building a strong evidence base on male health. The study aims to recruit a large cohort of Australian males between the ages of 10 and 55 years, and, subject to ongoing funding, will follow participants every three years to gather key information about the determinants of male health.In 2012/3 a pilot study was conducted in preparation for Wave 1 of the study. The pilot study was conducted in two phases and aimed to test: Different contact sources for sampling: the Medicare Australia enrolment database, the Australian Electoral Commission electoral roll database, Australian schools and a household sampling frame;Different recruitment strategies (various pre-notification and reminder mail-out protocols; recruitment via households using a drop and collect methodology); and,The study instruments, data collection logistics and the consenting process.This report provides an overview of analyses conducted to evaluate the performance of the study instruments developed for the baseline wave of the Ten to Men study and pilot tested in the 2012/3 pilot study. This included the evaluation of four separate study instruments, designed specifically for particular age groups within the study cohort:A computer-assisted personal interview (CAPI) for boys aged 10 to 14 years (boys CAPI)A scannable self-complete questionnaire for parents of boys aged 10 to 14 years (parent questionnaire)A scannable self-complete questionnaire for young men aged 15 to 17 years(young men questionnaire)A scannable self-complete questionnaire for adult males aged 18 to 55 years(adult questionnaire)The report also provides an overview of recommendations made and actions taken in relation to refinement of the study instruments for use in Wave 1 of the study. Chapter 1 provides an overview of the Ten to Men study and the development of the Wave 1 study instruments. Chapter 2 provides an overview of the methodology used to conduct the 2012/3 pilot study, evaluate the performance of the study instruments and make decisions about refinement of the study instruments. Chapter 3 provides a detailed summary of the results of analyses conducted by the Ten to Men study team to evaluate the performance of the Wave 1 study instruments. Chapter 4 provides an overview of recommendations for study instrument refinement arising from the pilot study and the actions taken in relation to those recommendations. The report concludes with a general discussion of the implications of the testing, evaluation and refinement of the study instruments for Wave 1 of the study in Chapter 5. More detailed information about the study instruments and the analysis plan on which the evaluation of the study instruments was based is provided in the Appendices to this report. Overall, the analyses conducted to evaluate the performance of the four study instruments generally found that the instruments were performing well and capturing data as intended. Only a small number of items were identified to have some performance issues. As a result, only minor changes have been made to the study instruments for Wave 1 data collection as a result of performance issues. Additional modifications to the instruments were undertaken to reduce participant burden.INTRODUCTIONTen to Men OverviewTen to Men (the Australian Longitudinal Study on Male Health) will be the first longitudinal study of male health of its kind in Australia. The study is funded by the Commonwealth Department of Health and Ageing and aims to:Examine male health and its key determinants including social, economic, environmental and behavioural factors that affect the length and quality of life of Australian males.Address key research gaps about the health of Australian males such as men’s health and risk behaviour in life, while accounting for social, economic and environmental changes.Identify policy opportunities for improving the health and wellbeing of males and providing support for males at key life stages, particularly those at risk of poor health.The study falls under the 5th priority area for action in the 2010 National Male Health Policy: Building a strong evidence base on male health ADDIN EN.CITE <EndNote><Cite><Author>Department of Health and Ageing</Author><Year>2010</Year><RecNum>7</RecNum><DisplayText>[1]</DisplayText><record><rec-number>7</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">7</key></foreign-keys><ref-type name="Government Document">46</ref-type><contributors><authors><author>Department of Health and Ageing,</author></authors><secondary-authors><author>Department of Health and Ageing,</author></secondary-authors></contributors><titles><title>National Male Health Policy: Building on the Strengths of Australian Males</title></titles><dates><year>2010</year></dates><pub-location>Barton ACT</pub-location><publisher>Commonwealth of Australia</publisher><urls></urls></record></Cite></EndNote>[1]. The 2010 National Male Health Policy has the broader aim of improving the health of all males and achieving equal health outcomes for population groups of males at risk of poor health, such as Aboriginal and Torres Strait Islander males, migrant males, males living in rural and remote areas and socially disadvantaged males ADDIN EN.CITE <EndNote><Cite><Author>Department of Health and Ageing</Author><Year>2010</Year><RecNum>7</RecNum><DisplayText>[1]</DisplayText><record><rec-number>7</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">7</key></foreign-keys><ref-type name="Government Document">46</ref-type><contributors><authors><author>Department of Health and Ageing,</author></authors><secondary-authors><author>Department of Health and Ageing,</author></secondary-authors></contributors><titles><title>National Male Health Policy: Building on the Strengths of Australian Males</title></titles><dates><year>2010</year></dates><pub-location>Barton ACT</pub-location><publisher>Commonwealth of Australia</publisher><urls></urls></record></Cite></EndNote>[1].The study aims to recruit a large cohort of Australian males between the ages of 10 and 55 years, and, subject to ongoing funding, will follow participants every three years to gather key information about the determinants of male health. Given the broad age range of participants, three age-based subgroups have been defined for Wave 1 data collection, each of which has their own specific data collection instrument(s) that reflects the appropriate mode of data collection and relevant constructs for respondents of this age group: Adult males aged 18 to 55 yearsYoung Men aged 15 to 17 yearsBoys aged 10 to 14 yearsParents of boys aged 10 to 14 years will also be surveyed to provide data to augment that provided by the boys themselves.Wave 1 Study InstrumentsContent Development ProcessIn line with the framework set out in the 2010 National Male Health Policy ADDIN EN.CITE <EndNote><Cite><Author>Department of Health and Ageing</Author><Year>2010</Year><RecNum>7</RecNum><DisplayText>[1]</DisplayText><record><rec-number>7</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">7</key></foreign-keys><ref-type name="Government Document">46</ref-type><contributors><authors><author>Department of Health and Ageing,</author></authors><secondary-authors><author>Department of Health and Ageing,</author></secondary-authors></contributors><titles><title>National Male Health Policy: Building on the Strengths of Australian Males</title></titles><dates><year>2010</year></dates><pub-location>Barton ACT</pub-location><publisher>Commonwealth of Australia</publisher><urls></urls></record></Cite></EndNote>[1], the study instruments for Wave 1 of Ten to Men were developed based on an holistic model of male health that takes into account the broad range of social, environmental and individual level factors that may impact on male health and quality of life ADDIN EN.CITE <EndNote><Cite><Author>Schlichthorst</Author><Year>2012</Year><RecNum>17</RecNum><DisplayText>[2]</DisplayText><record><rec-number>17</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">17</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Schlichthorst, M.</author><author>Currier, D.</author><author>Brinkman, M.</author><author>Pirkis, J.</author><author>English, E.</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: An Introduction to the Study Design and Plan</title></titles><dates><year>2012</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne </publisher><urls></urls></record></Cite></EndNote>[2]. The model considers both population-level and individual-level factors that impact on health and wellbeing. In developing the Wave 1 study instruments, considerable thought was given to ensuring they captured information that was critical to the baseline wave of a longitudinal study and evaluation of the determinants of male health and wellbeing under a holistic model of male health. At the same time, consideration was also given to the burden placed upon participants to capture such data. To minimise participant burden and ensure that data on the full range of constructs needed, the data captured by the Wave 1 study instruments will be augmented by data linkage with health-related datasets, such as the Medicare Benefits Schedule and Pharmaceutical Benefits Scheme databases, and geospatial information sources. Priority was therefore given in the Wave 1 study instruments to capturing constructs that would not be readily available via data linkage. Development of the Wave 1 study instrumentsDevelopment of the Wave 1 study instruments occurred in five key stages (see Figure 1). Following articulation of the key priority areas for action in the 2010 National Male Health Policy ADDIN EN.CITE <EndNote><Cite><Author>Department of Health and Ageing</Author><Year>2010</Year><RecNum>7</RecNum><DisplayText>[1]</DisplayText><record><rec-number>7</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">7</key></foreign-keys><ref-type name="Government Document">46</ref-type><contributors><authors><author>Department of Health and Ageing,</author></authors><secondary-authors><author>Department of Health and Ageing,</author></secondary-authors></contributors><titles><title>National Male Health Policy: Building on the Strengths of Australian Males</title></titles><dates><year>2010</year></dates><pub-location>Barton ACT</pub-location><publisher>Commonwealth of Australia</publisher><urls></urls></record></Cite></EndNote>[1], the Ten to Men study team worked with Content Development Working Groups to define key constructs within each priority area (stage 1) and source questions for each construct and age group (stage 2). A preliminary version of the study instruments was then developed and reviewed by the Ten to Men study team, Content Development Working Groups and other external stakeholders (stage 3). Following review and refinement, an initial version of the study instruments underwent cognitive testing and another round of refinement based on the result of cognitive testing (stage 4). This version of the study instruments was pilot tested in the 2012/3 pilot study described in this report (stage 5). In the following pages we present the results of analyses conducted to evaluate the performance of the pilot-tested study instruments, an overview of recommendations and the final refinement of the study instruments for Wave 1 data collection. Study Instrument Development ProcessStartStage 1Stage 2Stage 4Key priority areas in the 2010 National Male Health PolicyDefine health constructs within the key priority areasInternal and external review of study instrumentsCognitive testing and refinement of study instrumentsFinalise study instruments for Wave 1 of data collectionStage 5FinalStage 3Source questions for each construct and age group Pilot testing of revised study instrumentsStartStage 1Stage 2Stage 4Key priority areas in the 2010 National Male Health PolicyDefine health constructs within the key priority areasInternal and external review of study instrumentsCognitive testing and refinement of study instrumentsFinalise study instruments for Wave 1 of data collectionStage 5FinalStage 3Source questions for each construct and age group Pilot testing of revised study instrumentsFurther information about the initial phases of development of the study instruments and the results of cognitive testing of the study instruments is available in the following reports:2012 Major Report. Ten to Men – The Australian Longitudinal Study on Male Health: An Introduction to the Study Design and Plan ADDIN EN.CITE <EndNote><Cite><Author>Schlichthorst</Author><Year>2012</Year><RecNum>17</RecNum><DisplayText>[2]</DisplayText><record><rec-number>17</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">17</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Schlichthorst, M.</author><author>Currier, D.</author><author>Brinkman, M.</author><author>Pirkis, J.</author><author>English, E.</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: An Introduction to the Study Design and Plan</title></titles><dates><year>2012</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne </publisher><urls></urls></record></Cite></EndNote>[2]Ten to Men – The Australian Longitudinal Study on Male Health: Technical Report #2 ADDIN EN.CITE <EndNote><Cite><Author>Ten to Men Study Team</Author><Year>2013</Year><RecNum>18</RecNum><DisplayText>[3]</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">18</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Ten to Men Study Team,</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: Technical Report #2</title></titles><dates><year>2013</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite></EndNote>[3]Content OverviewThe study instruments piloted in the 2012/3 pilot study contained constructs from six key domains related to male health and wellbeing:Social and environmental determinants of health and wellbeingHealth-related behavioursHealth literacy / knowledgePhysical health statusMental health and well-beingHealth service use REF _Ref354065018 \r \h Table 1 provides an overview of the types of constructs included from each domain.Domains and Constructs Covered in the Wave 1 Study InstrumentsSocial and Environment Determinants of Health:Socio-economic statusFamily and household structureGender roles, relationships and sexualityHousing and tenureBroader interpersonal dynamics (including social networks, social support, violence and bullying)Life eventsTransportWork satisfaction, control and securityPhysical Health Status:Self-rated health statusHealth condition diagnosesInjury and disabilityPubertal development Sexual functionProstate functionSleepHealth-Related Behaviours:Alcohol useTobacco useDrug use – illicit and prescription drugsPeer behaviours - drugs & alcoholDiet/nutritionPhysical activityWeightSun exposureRisky / antisocial behaviourSexual behaviourMental Health and Wellbeing:Depression and anxiety Life satisfactionPositive mental healthSelf-injury: suicidal and non-suicidalHealth Literacy / Knowledge: Health information sourcesTrust in health information sourcesHealth Service Use:Health services useAccess to health servicesHealth service preferencesGiven the broad age range of in-scope males, four specific study instruments were developed to capture the data from the entire study cohort:A computer-assisted personal interview (CAPI) for boys aged 10 to 14 years (hereafter referred to as the boys CAPI)A scannable self-complete questionnaire for parents of boys aged 10 to 14 years(hereafter referred to as the parent questionnaire)A scannable self-complete questionnaire for young men aged 15 to 17 years(hereafter referred to as the young men questionnaire)A scannable self-complete questionnaire for adult males aged 18 to 55 years(hereafter referred to as the adult questionnaire)The study instrument for each age group contained age-group specific items as well as items common across age-groups. Naming conventions for variables within the dataset were designed to provide an indication of the age group of respondents to whom each data element within the dataset applies (see section 8.1 of Technical Report #2 for further information ADDIN EN.CITE <EndNote><Cite><Author>Ten to Men Study Team</Author><Year>2013</Year><RecNum>18</RecNum><DisplayText>[3]</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">18</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Ten to Men Study Team,</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: Technical Report #2</title></titles><dates><year>2013</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite></EndNote>[3]). REF _Ref346282881 \r \h Table 2 provides an overview of the number of questions in each study instrument and the number of items by field type. The final pilot study instruments are available in Appendix REF _Ref356915142 \r \h 7.2.Summary of Data Items By Study Instrument and Field TypeBoys CAPIParent QuestionnaireYoung Men QuestionnaireAdult QuestionnaireNumber of PagesNA3122032Number of Questions6557101144Number of Data Elements1Total232237382566Categorical171170306436Numeric505565110Character10111019Date/Time1111Number of Derived Variables2Total22123036Categorical116910Numeric721726Character0000Date/Time44401 Data elements – raw data items included in the study instruments.2 Derived variables – variables computed from one or more data elements. The number of derived variables presented in the Table above represents the minimum set of derived variables that need to be derived from composite data elements, where the construct is captured using more than one data element for ease of reporting, and scale scores for validated measures.3 NA = Not applicableMETHODSPilot Testing OverviewThe study instruments were pilot-tested in two phases: a mail-out pilot and a household recruitment (drop and collect) pilot.Phase 1: Mail-out pilotFrom October to December 2012 the first phase of pilot testing was conducted to trial:Different contact sources for sampling: the Medicare Australia database, the Australian Electoral Commission (AEC) electoral roll and Australian schools;Different mail-out recruitment strategies (pre-notification letter, type of reminder); and,The study instruments, data collection logistics and the consenting process.Recruitment via the AEC and Australian schools encountered feasibility issues during pre-field work and did not progress to field testing. Thus, all phase 1 respondents were recruited via the Medicare Australia enrolment database. The sampling frame was based on postcodes in the Medicare Australia enrolment database and stratified by age and Australian Standard Geographical Classification regions (capital city/metro, inner regional and outer regional areas) in order to oversample young males and males living in regional areas. All potential participants (1250 adults, 500 young men and 500 boys and parents) were mailed study materials, which included the self-complete questionnaire for young men and adult participants. Young men and adult respondents were asked to complete and return the completed questionnaire and associated documents (consent documentation and contact details sheet) in the reply-paid envelopes provided. Young men were also provided with information for their parents, so that their parents could consider the study and provide parental consent. Boys were sent an information pack containing an overview of the study and their parents were sent a pack containing an overview of the study and a contact information sheet. Interested parents and boys were asked to contact the fieldwork organisation by telephone or via return of the contact information sheet to request an interview. Parents of boys completed their self-complete questionnaire at the time of the boys’ interviews, which were generally conducted in the boys’ home; documents completed by boys and their parents were then returned by the interviewer to the fieldwork organisation. Various pre-notification and reminder combinations were also trialled.The response fraction for the phase 1 pilot was lower than anticipated across all three streams of respondents (adult males: 5.8%; young men males: 5.4%; boys: 7.7%), raising concerns about the viability of this sampling strategy. As a result, a second pilot (the phase 2 pilot) was conducted to trial a household sampling strategy. Further details about the phase 1 pilot can be found in the Ten to Men – The Australian Longitudinal Study on Male Health: Technical Report #2 ADDIN EN.CITE <EndNote><Cite><Author>Ten to Men Study Team</Author><Year>2013</Year><RecNum>18</RecNum><DisplayText>[3]</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">18</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Ten to Men Study Team,</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: Technical Report #2</title></titles><dates><year>2013</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite></EndNote>[3].Phase 2: Drop and collect pilotThe phase 2 pilot was conducted from February to March 2013 and involved a fieldwork interviewer making contact with private dwellings in selected postcode regions (different postcode regions to the mailout pilot) and:Attempting to determine whether any in-scope males resided in the household;Where at least one in-scope male was found, explaining the study;Dropping off and collecting a study pack for young men and adult males who agreed to participate; and,Conducting a CAPI interview with boys who agreed to participate (their parent was asked to complete their questionnaire during the interview period as with the phase 1 pilot). Up to two adults (one 18-24 and one 25-55 year old), one young man and one boy were invited to participate if more than one in-scope male resided in the household. In general, the study materials used in the drop and collect pilot were identical or very similar to the mailout pilot. No changes were made to the study instruments. The drop and collect pilot study was powered to be able to detect a difference in the response fraction for the study amongst adult respondents. If the response fraction was 15% or higher in the adult group, the study had at least 85% power to detect a difference in the response fraction between the two pilot studies. REF _Ref356903524 \r \h Table 3 below provides an overview of the various household-level response fractions for the drop and collect pilot. Overall, 240 households were approached. Of these households, contact was made with 84% of households and the status (whether the household did / did not contain an in-scope male) of 78% of households was determined. A total of 36% of households approached were confirmed to contain at least one in-scope male. Among confirmed in-scope households, 43% of households provided usable data for at least one in-scope male (15% of all households approached overall).Household-level response fractions for the drop and collect pilotHousehold Response ParameterProportion of Households1Number (%)Number Approached240 (100%)Number Contacted201 (84%)Status ConfirmedNo in-scope males In-scope male(s) present186 (78%)99 (41%)87 (36%)Provided usable dataOf households approachedOf households contactedOf confirmed in-scope households37 (15%)37 (18%)37 (43%)1 Of households approached unless otherwise stated. REF _Ref356903527 \r \h Table 4 provides the individual-level response fractions for each age group of respondents approached. Overall, response fractions of 35% among confirmed in-scope under 18-year olds and 44% among confirmed in-scope adult males were achieved, demonstrating an improved response fraction with this recruitment methodology. However, the small numbers in the younger age groups indicates caution should be taken interpreting the data.Individual-level response fractions for the drop and collect pilotIndividual Level Response ParameterProportion By Age Group1 Number (%)BoysYoung Men(15-17 years)Adults (18 – 24 years)Adults(25 – 55 years)Confirmed in-scope12 (100%)8 (100%)20 (100%)71 (100%)Agreed to participate6 (50%)3 (38%)15 (75%)50 (70%)Provided usable data6 (50%)1 (13%)7 (35%)33 (46%)1 Of number in each age group confirmed to be in-scopeRespondent Characteristics REF _Ref355788387 \r \h Table 5 provides the total numbers of respondents who were recruited in each phase of the pilot study.Number of Participants Recruited in Wave 1 Pilot Studies1Study GroupNumber of Participants:MailoutNumber of Participants:Drop & Collect PilotBoys (10 – 14 year olds)376Young Men (15 – 17 year olds)2721Adults (18 – 55 years)70401 Participants with valid consent and questionnaire data. 2 Seven additional participants who returned documents late or from whom valid consent was subsequently obtained were also recruited (making a total of 34 young men recruited via the mailout pilot). Data from these respondents was not available at the time of analysis and so is not included in analyses presented in this report. Socio-demographic characteristics of adult respondents from each phase of the pilot were compared to assess potential socio-demographic differences in the characteristics of respondents recruited using mail-out compared to household sampling (see REF _Ref355790088 \r \h Table 6). Socio-demographic characteristics of respondents from both pilot phases were also compared to the characteristics of the Australian general population of adult in-scope males (males aged 18-55 years) based on data from the 2011 Census. Socio-demographic Characteristics of Adult Respondents Socio-demographic CharacteristicDescriptive StatisticsNumber (%)Mailout pilot(n = 70)Descriptive StatisticsNumber (%)Drop and collect pilot(n = 40)Age (years)18-2425-55OtherMissing14 (20.0%)54 (77.1%)0 (0.0%)2 (2.8%)7 (17.5%) 33 (82.5%)0 (0.0%)0 (0.0%)Marital StatusNever marriedWidowedDivorcedSeparated but not divorcedMarried/de factoMissing16 (22.9%)0 (0.0%)4 (5.7%)1 (1.4%)47 (67.1%)2 (2.8%)10 (25.0%)1 (2.5%)0 (0.0%)1 (2.5%)28 (70.0%)0 (0.0%)Country of birth1AustraliaElsewhereMissing53 (75.7%)16 (22.9%)1 (1.4%)26 (65.0%)14 (35.0%)0 (0.0%)Main language spoken at home1 EnglishOther languageMissing64 (91.4%)5 (7.1%)1 (1.4%)31 (77.50%)9 (22.50%)0 (0.0%)Highest education levelYear 12 or lessTrade certificate/apprenticeshipDiplomaUniversity degreeOtherMissing20 (28.6%)12 (17.1%)7 (10.0%)23 (32.9%)4 (5.7%)4 (5.7%)10 (25.0%)10 (25.0%)2 (5.0%)15 (37.5%)1 (2.5%)2 (5.0%)Employment statusEmployed (%)Unemployed, looking for work (%)Unemployed, not looking for work (%)Missing (%)55 (78.6%)8 (11.4%)7 (10.0%)0 (0.0%)32 (80.0%)5 (12.5%)1 (2.5%)2 (5.0%)Before Tax Household Income(2011/12 Financial Year) $200,000 or more (%)$150,000 - $199,999 (%)$125,000 - $149,999 (%)$100,000 - $124,999 (%)$80,000 - $99,999 (%)$60,000 - $79,999 (%)$50,000 - $59,999 (%)$40,000 - $49,999 (%)$30,000 - $39,999 (%)$20,000 - $29,999 (%)$10,000 - $19,999 (%)$1 - $9,999 (%)Nil income (%)Negative income (%)Don’t know (%)Missing (%)1 (1.4%)9 (12.9%)8 (11.4%)10 (14.3%)9 (12.9%)8 (11.4%)4 (5.7%)6 (8.6%)6 (8.6%)2 (2.8%)2 (2.8%)0 (0.0%)1 (1.4%)0 (0.0%)3 (4.2%)1 (1.4%)2 (5.0%)1 (2.5%)6 (15.0%)4 (10.0%)6 (15.0%)5 (12.5%)5 (12.5%)1 (2.5%)2 (5.0%)2 (5.0%)1 (2.5%)0 (0.0%)0 (0.0%)0 (0.0%)4 (10.0%)1 (2.5%)The mail-out sample was highly skewed towards a higher socioeconomic status group. While adult respondents from the drop and collect pilot were still more highly educated than the general population of Australian adult in-scope males and males who had never been married were under-represented amongst the study respondents, the sample of adult males recruited via the drop and collect pilot were more similar to the general population of in-scope males in terms of the proportion born in Australia versus overseas, the proportion who spoke English as their main language at home and in terms of their household income. In general, the sample of respondents recruited via the drop and collect pilot appeared to be more diverse and more representative of the general population of in-scope males than the sample recruited via the mail-out pilot.Socio-demographic characteristics of young men and boy respondents were not compared due to the small sample size of these respondents in the drop and collect pilot. Socio-demographic characteristics of young men and boy respondents from the mail-out pilot are provided elsewhere ADDIN EN.CITE <EndNote><Cite><Author>Ten to Men Study Team</Author><Year>2013</Year><RecNum>18</RecNum><DisplayText>[3]</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">18</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Ten to Men Study Team,</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: Technical Report #2</title></titles><dates><year>2013</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite></EndNote>[3]. In general, young men and boy respondents from the mail-out pilot also tended of higher socioeconomic status.Evaluation of the Performance of the Study InstrumentsAnalyses were conducted in accordance with Ten to Men – The Australian Longitudinal Study on Male Health: Dress Rehearsal Analysis Plan Version 1 (17 December 2012) (included as Appendix REF _Ref356917070 \r \h 7.2) ADDIN EN.CITE <EndNote><Cite><Author>Ten to Men Study Team</Author><Year>2012</Year><RecNum>23</RecNum><DisplayText>[6]</DisplayText><record><rec-number>23</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">23</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Ten to Men Study Team,</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: Dress Rehearsal Analysis Plan Version 1 (17 December 2012)</title></titles><dates><year>2012</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite></EndNote>[4]. StataSE 12 and Microsoft Excel 2007 were used to conduct the analyses. In most cases, descriptive statistics (mean, median, standard deviation, range, response frequencies) are used to summarise results of analyses outlined in the analysis plan.Analyses conducted to evaluate the performance of the Ten to Men instruments detailed in the analysis plan and reported here included an assessment of:the proportion of missing data for individual survey items;trends in item response fractions by page number and topic;the performance of questionnaire skip logic;implausible values and inconsistent responses; the utility of ‘other, specify’ responses provided; and,the performance of specific item formats.All of the above analyses were conducted following phase 1 of the pilot study and, except where indicated, the following results pertain to that sample. The analyses were conducted primarily using data from phase 1 of the pilot study as that phase was specifically designed to evaluate the performance of the study instruments, while the phase 2 pilot was specifically designed to assess the feasibility of an alternative sampling frame. Additional analyses, aimed at assessing potential differences in instrument performance related to change in the sampling frame (mail-out via the Medicare database versus household sampling using the drop and collect method) are reported in section REF _Ref355864598 \r \h 3.9. A number of the analyses involved assessing the frequency of particular questionnaire performance issues—for example, missing data, implausible values, inconsistent responses and so on. Given that we aim to make inferences about how the questionnaires will perform in Wave 1 data collection based on performance in the phase 1 pilot, it is important to consider the expected confidence interval for the proportions reported in this report (see REF _Ref341168914 \r \h Table 7 for a guide to the expected confidence interval width for proportions of different magnitude and for studies with varies sample sizes). Given that the final sample sizes of the phase 1 pilot fell short of the target sample sizes, results presented in this report should be interpreted with some caution as sampling variation may mean that the pilot results may differ from those obtained with a larger sample. 95% Confidence Intervals for Various Proportions By Sample Size95% Confidence Interval By Sample Size1Proportion (%)501001502002501n/c>0 – 5n/c>0 - 4n/c2>0 - 11>0 – 7>0 - 61 - 51 - 55n/c2 - 11n/c2 – 9n/c103 - 225 - 186 - 166 - 157 - 142010 - 3413 - 2914 - 2715 - 2615 - 265036 - 6540 - 6041 - 5843 - 5744 - 561 n/c = not calculatedRefinement of the Study InstrumentsFollowing completion of the study instrument evaluation process, a set of recommendations to guide revision of the study instruments for Wave 1 data collection were produced. These recommendations were based on:Analysis of the pilot datasets conducted by the Ten to Men study team (summarised in section REF _Ref355945275 \r \h 3 of this report);Qualitative and quantitative data provided by Roy Morgan Research in their analysis of the pilot studies;Research priorities for Ten to Men; and,Benchmark criteria for questionnaire items:Brief, reliable and valid measure of underlying constructImportance of the questionnaire item outweighs any concerns regarding participant burdenQuestionnaire item can be used across groups of study participants (boys, young men and adult males) wherever relevant. Input from the Questionnaire Development Working Groups (see ADDIN EN.CITE <EndNote><Cite><Author>Schlichthorst</Author><Year>2012</Year><RecNum>17</RecNum><DisplayText>[2]</DisplayText><record><rec-number>17</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">17</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Schlichthorst, M.</author><author>Currier, D.</author><author>Brinkman, M.</author><author>Pirkis, J.</author><author>English, E.</author></authors></contributors><titles><title>Ten to Men - The Australian Longitudinal Study on Male Health: An Introduction to the Study Design and Plan</title></titles><dates><year>2012</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne </publisher><urls></urls></record></Cite></EndNote>[2] for a list of these) regarding possible modifications to improve the items. The recommendations were reviewed by the Ten to Men study team and the appropriate actions in relation to the recommendations were determined. In general, actions taken by the study team fell into one of three categories: Application of data cleaning rules to ameliorate the effects of issues identified with the questionnaire itemQuestionnaire item modificationRemoval of the item from the questionnaireThe proposed modifications to the questionnaires were approved by the Commonwealth Department of Health and Ageing prior to commencement of Wave 1 of data collection.EVALUATION OF INSTRUMENT PERFORMANCEIn the interests of brevity, this chapter provides a summary of analyses conducted to evaluate the performance of the four piloted Wave 1 study instruments. The findings of these analyses are described in more detail in two longer reports ADDIN EN.CITE <EndNote><Cite><Author>Koelmeyer</Author><Year>2013</Year><RecNum>24</RecNum><DisplayText>[7, 8]</DisplayText><record><rec-number>24</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">24</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Koelmeyer, R.</author></authors></contributors><titles><title>Ten to Men 2012 Dress Rehearsal: Study Instrument Evaluation Report 1</title></titles><dates><year>2013</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite><Cite><Author>Sumpton</Author><Year>2013</Year><RecNum>25</RecNum><record><rec-number>25</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">25</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Sumpton, C.</author></authors></contributors><titles><title>Ten to Men 2012 Dress Rehearsal: Study Instrument Evaluation Report 2</title></titles><dates><year>2013</year></dates><pub-location>Melbourne</pub-location><publisher>The University of Melbourne</publisher><urls></urls></record></Cite></EndNote>[5,6]. These reports can be made available upon request. Items which were identified to have some performance issues are described in the text below. Actions taken in relation to items with performance issues are described in section REF _Ref357605700 \r \h 4.Missing Data AnalysisRaw Data ElementsFor each of the four study instruments, the proportion of missing data was assessed for each variable in the phase 1 pilot dataset. The proportion of missing data was defined as the proportion of respondents to whom the variable applied who did not provide a response. REF _Ref349309181 \r \h Table 8 provides an overview of the findings of the analysis. Overall, the proportion of missing data for each variable was generally low. Missing data was particularly infrequent for the boys CAPI (median proportion of missing data per variable: 0%; mean proportion of missing data per variable: 0.6%), which is expected given that the collection of this data was assisted by a trained interviewer and computer programming administered the skip logic. The young men questionnaire also performed well overall in relation to missing data, with the median proportion of missing data per variable being 0% and the mean proportion being 2.0%. The proportion of missing data was generally similar for the adult and parent self-complete questionnaires (adult questionnaire – median: 1.43%, mean 5.1%; parent questionnaire – median: 2.9%, mean: 5.7%). Summary of Missing Data Analysis By Study Instrument1Study InstrumentCharacteristicBoys CAPIYoung Men QuestionnaireAdult QuestionnaireParents QuestionnaireTotal Number of Respondents37277035Total Number of Variables221369538231Number (%) of variables with >0% Missing Data12 (5.4)98 (26.7)296 (55.0)124 (53.7)Number (%) of variables with >5% Missing Data5 (2.3)47 (12.8)86 (16.0)33 (14.3)Number (%) of variables with >10% Missing Data5 (2.3)25 (6.8)54 (10.0)21 (9.1)Number (%) of variables with >20% Missing Data2 (0.9)2 (0.5)38 (7.1)15 (6.5)Median % Missing Data (Range)0 (0 - 33.3)0 (0 – 50)1.43 (0 - 100)2.9 (0 - 100)Mean % Missing Data (Standard Deviation)0.6 (3.7)2.0 (4.7)5.1 (13.8)5.7 (15.6)1 The proportion of missing data for each variable was calculated as the proportion of respondents to whom the item applied who did not provide a response.Of interest is the similarity in the proportions of missing data for the adult and parent questionnaires, given that the parent questionnaire was less than half the length of the adult questionnaire and parents and adult males were of similar age and socioeconomic status. This finding suggests that the overall length of the questionnaire may have had little impact on the likelihood of data being missing. Overall, most of the variables in the boys CAPI (94.6%), almost three-quarters of the variables in the young men questionnaire (73.3%) and just under half of the variables in the adult (45.0%) and parent (46.3%) questionnaires had no missing data. Only a minority of variables had more than 5% missing data (boys: 2.3%; young men: 12.8%; adults: 16.0%; parents: 14.3%). In general, variables with more than 5% missing data tended to be:Variables that applied to a small number of respondents, such that the precision of the estimated proportion of missing data is likely to be quite low. Variables for which respondents may have had difficulty with calculating or recalling the behaviour in question (for example, number of standard drinks consumed on a typical drinking day).Variables for which sensible data cleaning rules are likely to remove a large proportion of missing data (for example, rows of variables requiring yes/no responses where respondents only answer the items that apply to them or questions where respondents are required to provide a null response of ‘0’ if the item doesn’t apply to them). The vast majority of variables with more than 5% missing data fell into this category.Derived VariablesItems with Composite Data ElementsFor ease of data collection and to cover multiple response options efficiently, some constructs in the questionnaires were captured as composite data elements (see REF _Ref342400228 \r \h Table 22 in Appendix REF _Ref356830777 \r \h 7.1 for a list of these) and the final variable was computed from the underlying data elements. The proportion of missing data for the final derived items was assessed. Overall, most derived variables had a low level of missing data, and those with a higher level of missing data generally applied to a small number of respondents, such that the estimates are fairly imprecise. REF _Ref349740015 \r \h Table 9 below provides a summary of the results of the derived variable missing data analysis.Summary of Missing Data Analysis for Variables Derived from Composite Data Elements1Study InstrumentCharacteristicBoys CAPI(n = 37)Young Men Questionnaire(n = 27)Adult Questionnaire(n = 70)Parents Questionnaire(n = 35)Total Number of Variables1316218Number (%) of variables with >10% Missing Data2 (15.4)4 (25.0)6 (28.6)1 (7.7)Median % Missing Data Per Variable (Range)0 (0 - 17)0 (0 – 40.7)2.9 (0 - 100)3 (0 - 25)1 Proportion of missing data calculated for each derived variable as the proportion of respondents to whom the item applied for whom a response could not be calculated.The main variables where level of missing data was cause for concern were the anthropometric measures which were derived variables with consolidated data for weight, height and waist circumference. REF _Ref349740245 \r \h Table 10 details the results for these anthropomorphic variables. Overall, the level of missing data for all three items in the young men subgroup was high (22.2% for weight, 11.1% for height and 40.7% for waist measurement). The level of missing data for the waist measurement for adult males was 27.1%. The proportion of missing data for waist circumference was much lower for the boys CAPI, which is expected, given that boys and their parents were asked to measure these parameters at the start of the boy’s interview. These findings suggest that additional emphasis may be needed on requesting that young men and adult respondents make an effort to determine their weight, height and waist measurements; alternatively the value of these variables may need reconsideration. Proportion of Missing Data for Anthropometric MeasurementsProportion of Missing Data By Respondent Age-group (%)Derived Variable(Variable Name)(don’t need)Boys (n = 37)Young Men (n = 27)Adults (n = 70)Weight in kilograms2.722.24.3Height in centimetres011.15.7Waist measurement in centimetres5.440.727.1Summary Scores for Validated Scales A number of validated measures were included in the Ten to Men instruments (see REF _Ref352145814 \r \h Table 23 in Appendix REF _Ref356830777 \r \h 7.1 for a list of these). The majority were validated scales that required a summary score to be computed from the component data elements in the scale. REF _Ref349741295 \r \h Table 11 provides a summary of the results of missing data analysis on the summary items and includes a comparison to the missing data statistics for the Ten to Men instruments overall. Overall, the validated measures performed well with a low level of missing data and respondents answering the vast majority of items in the scales. Only one summary measure (from the Pubertal Development Scale in the boys CAPI) had more than 10% missing data. The formula for calculating this variable contains some gaps (see Carskadon et al 1993 ADDIN EN.CITE <EndNote><Cite><Author>Carskadon</Author><Year>1993</Year><RecNum>21</RecNum><DisplayText>[9]</DisplayText><record><rec-number>21</rec-number><foreign-keys><key app="EN" db-id="pr25d0ef5exe5bees5yps9vsr55ftvedew9r">21</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Carskadon, M. A.</author><author>Acebo, C.</author></authors></contributors><titles><title>A Self-Administered Rating Scale for Pubertal Development</title><secondary-title>Journal of Adolescent Health</secondary-title></titles><periodical><full-title>Journal of Adolescent Health</full-title></periodical><pages>190-195</pages><volume>14</volume><dates><year>1993</year></dates><urls></urls></record></Cite></EndNote>[7]); such that it is likely that the level of missing data could be heavily reduced by clarifying where to classify respondents that fall in those gaps.Missing Data Characteristics of Variables Derived from Validated Measures1Study InstrumentCharacteristicBoys CAPI(n = 37)Young Man Questionnaire(n = 27)Adult Questionnaire(n = 70)Parents Questionnaire(n = 35)Raw Data Elements OverallTotal Number of Variables221367538231Number (%) of variables with >0% Missing Data12 (5.4)98 (26.7)296 (55.0)124 (53.7)Number (%) of variables with >5% Missing Data5 (2.3)47 (12.8)86 (16.0)33 (14.3)Median % Missing Data Per Variable (Range)0 (0 – 33.3)0 (0 – 50)1.43 (0 - 100)2.9 (0 - 100)Variables Derived from Validated MeasuresTotal Number of Variables915176Number (%) of variables with >0% Missing Data1 (11.1)0 (0)5 (29.4)0 (0)Number (%) of variables with >5% Missing Data1 (11.1)0 (0)0 (0)0 (0)Median % Missing Data Per Variable (Range)0 (0 - 16)0 (NA2)0 (0 – 4.3)0 (NA2)Median % of Items in Scale Answered (Range)100 (99 - 100)98 (96 - 100)100 (98 - 100)97 (0 - 100)1 Proportion of missing data calculated for each variable as the proportion of respondents to whom the item applied who did not provide a response to the item or for whom a response could not be calculated.2 NA = Not ApplicableItem Response Fractions by Page NumberFor the three self-complete questionnaires (parent, young men, adult), the proportion of applicable respondents who completed each variable was calculated and summarised by page number to assess if there were trends in the variable response fraction by page number. REF _Ref356831161 \r \h Figure 2 shows that the median per page response fractions were generally equal to or greater than 95% and there was no evidence of decay in response fraction by increasing page number. Where the response fraction was below 95%, it tended to be for pages containing items that were identified as problematic in other analyses described elsewhere in this report. Hence, it does not appear that the page position of items in the questionnaires were a deterrent to completion of the items.Median Variable Response Fraction by Study Instrument and Page NumberA) B)C)Vertical Axis – Median Variable Response Fraction (%); Horizontal Axis – Page Number. A) Adult Questionnaire B) Young Men Questionnaire C) Parent QuestionnaireItem Response Fractions by TopicVariable response fractions were also summarised by topic (the domain and construct for each variable) to assess if certain topics were less likely to be answered. REF _Ref356810488 \r \h Table 12 to REF _Ref356810490 \r \h Table 14 provide the results of this analysis. Similar to the summary of response fractions by page number, most topics in the questionnaires had low levels of non-response and where the mean or median response fraction fell below 95% this was typically for topics containing variables that had been identified as problematic in other analyses.Adult Questionnaire - Variable Response Fractions By Topic Descriptive Statistics of Response FractionTopicNumber of VariablesMean(Standard Deviation)Median(Range)Behaviours - alcohol1498.5 (1.5)98.4 (95.1 - 100.0)Behaviours - drugs4275.3 (33.2)97.1 (6.9 - 100.0)Behaviours - nutrition499.3 (0.8)99.3 (98.6 - 100.0)Behaviours - physical activity2289.2 (6.4)88.7 (76.5 - 98.6)Behaviours - Risky and antisocial behaviour1100.0 (NA)100.0 (100.0 - 100.0)Behaviours - sexual behaviour1276.1 (31.0)85.9 (0.0 - 98.6)Behaviours - sun exposure596.6 (1.3)95.7 (95.7 - 98.6)Behaviours - tobacco1099.7 (0.9)100.0 (97.2 - 100.0)Behaviours - Weight494.6 (4.3)95.7 (88.6 - 98.6)Data Collection5100.0 (0.0)100.0 (100.0 - 100.0)Health status - diagnoses4696.4 (2.2)97.1 (92.9 - 100.0)Health status - health status1299.5 (0.7)100.0 (98.6 - 100.0)Health status - injury and disability1199.7 (0.6)100.0 (98.6 - 100.0)Health status - prostate function7100.0 (0.0)100.0 (100.0 - 100.0)Health status - sexual function896.9 (0.0)96.9 (96.9 - 96.9)Knowledge - health literacy4100.0 (0.0)100.0 (100.0 - 100.0)Social determinants - family and household structure6792.8 (16.2)100.0 (0.0 - 100.0)Social determinants - gender roles and sexuality2499.4 (0.9)100.0 (97.1 - 100.0)Social determinants - housing and tenure1897.8 (3.1)98.6 (86.4 - 100.0)Social determinants - interpersonal relations3296.5 (4.5)98.6 (90.0 - 100.0)Social determinants - life events2499.8 (0.6)100.0 (97.1 - 100.0)Social determinants - socioeconomic status6198.1 (1.8)98.6 (92.7 - 100.0)Social determinants - transport11100.0 (0.0)100.0 (100.0 - 100.0)Social determinants - Work satisfaction, control and security1399.7 (0.7)100.0 (98.2 - 100.0)Use of health services - access to health services2498.3 (1.8)98.3 (96.6 - 100.0)Use of health services - satisfaction and preferences 9100.0 (0.0)100.0 (100.0 - 100.0)Use of health services - service use2998.8 (0.5)98.6 (98.6 - 100.0)Wellbeing - life satisfaction7100.0 (0.0)100.0 (100.0 - 100.0)Wellbeing - mental health diagnoses1099.9 (0.5)100.0 (98.6 - 100.0)Wellbeing - self-injury5100 (0.0)100.0 (100.0 - 100.0)Parent Questionnaire - Variable Response Fractions By Topic Descriptive Statistics of Response FractionTopicNumber of VariablesMean(Standard Deviation)Median(Range)Behaviours - Risky and antisocial behaviour197.1 (NA)97.1 (97.1 - 97.1)Data Collection4100.0 (0.0)100.0 (100.0 - 100.0)Health status - diagnoses4297.0 (0.6)97.1 (94.3 - 97.1)Health status - injury and disability698.6 (2.4)100.0 (94.3 - 100.0)Health status - sleep1697.0 (0.7)97.1 (94.3 - 97.1)Social determinants - family and household structure3977.4 (31.5)94.3 (0.0 - 100.0)Social determinants - housing and tenure1898.8 (3.5)100.0 (85.7 - 100.0)Social determinants - interpersonal relations2100.0 (0.0)100.0 (100.0 - 100.0)Social determinants - life events7100.0 (0.0)100.0 (100.0 - 100.0)Social determinants - socioeconomic status3699.0 (2.3)100.0 (89.3 - 100.0)Social determinants - transport 1097.1 (0.0)97.1 (97.1 - 97.1)Use of health services - service use4697.5 (10.0)100.0 (33.3 - 100.0)Wellbeing - positive mental health797.1 (0.0)97.1 (97.1 - 97.1)Young Men Questionnaire - Variable Response Fractions By Topic Descriptive Statistics of Response FractionTopicNumber of VariablesMean(Standard Deviation)Median(Range)Behaviours - Alcohol1499.5 (2.1)100.0 (92.3 - 100)Behaviours - Drugs1699.8 (0.9)100.0 (96.3 - 100)Behaviours - nutrition4100.0 (0.0)100.0 (100.0 - 100)Behaviours - physical activity9100.0 (0.0)100.0 (100.0 - 100)Behaviours - Risky and antisocial behaviour10100.0 (0.0)100.0 (100.0 - 100)Behaviours - sexual behaviour1098.6 (3.2)100.0 (90.0 - 100)Behaviours - tobacco1198.5 (5.0)100.0 (83.3 - 100)Behaviours - Weight494.4 (4.8)94.4 (88.9 - 100)Data Collection5100.0 (0.0)100.0 (100.0 - 100)Health status - diagnoses4092.2 (4.3)92.6 (81.5 - 100)Health status - health status196.3 (NA)96.3 (96.3 - 96.3)Health status - injury and disability12100.0 (0.0)100.0 (100.0 - 100)Health status - pubertal development397.5 (2.1)96.3 (96.3 - 100)Health status - sleep16100.0 (0.0)100.0 (100.0 - 100)Knowledge - health literacy298.2 (2.6)98.2 (96.3 - 100)Social determinants - family and household structure62100.0 (0.0)100.0 (100.0 - 100)Social determinants - gender roles and sexuality2499.5 (1.2)100.0 (96.3 - 100)Social determinants - housing and tenure 1299.7 (1.1)100.0 (96.3 - 100)Social determinants - interpersonal relations 2598.7 (1.8)100.0 (96.3 - 100)Social determinants - life events 1199.7 (1.1)100.0 (96.3 - 100)Social determinants - socioeconomic status 2298.3 (3.3)100.0 (88.2 - 100)Social determinants - transport 11100.0 (0.0)100.0 (100.0 - 100)Use of health services - service use 1099.6 (1.2)100.0 (96.3 - 100)Wellbeing - mental health diagnoses 2297.8 (1.9)96.3 (96.3 - 100)Wellbeing - positive mental health 696.3 (0.0)96.3 (96.3 - 96.3)Wellbeing - self-injury) 991.4 (10.5)96.3 (66.7 - 100)Implausible Value Checks Analyses were conducted to assess whether numeric and date/time fields with definitive plausible value ranges contained any implausible values. Overall, 13.3% of checks conducted on data from the boys CAPI (2 out of 15 checks), 17.8% of checks on data from the young men questionnaire (5 out of 28 checks), 12.5% of checks on the adult questionnaire (9 out of 72 checks) and 10.7% of checks on the parent questionnaire (3 out of 28 checks) identified an out-of-range value. For the most part, the discrepancies were relatively minor with generally just a single respondent reporting an out-of-range value or the out-of-range value indicating that the range of plausible values for the variable needed to be extended (for example, the variable for length of relationship in months for the young men questionnaire was given a plausible range of 1 – 11 months; however, some respondents answered using a number of months greater than 12 months, which is still a plausible response). A small number of items were identified that had issue that need to be addressed:All self-complete study instruments:Questionnaire completion date This field had some values that fell outside of the data collection period. This could be rectified by replacing them with a date from the data collection period, however the discrepancies bring into question the usefulness of this field. Adults:Hours per day spent outside on a usual work and non-work day (Question 44)There were a handful of incidences of values of 30 and 45 being reported (outside the range of 0 – 24 hours for a day). These values may correspond to a time in minutes, which could be recoded to a time in hours. Ages of children they live with (Question 104) A total of five respondents stated an age greater than 18 years for one or more of these fields. This suggests that the respondents still consider children over 18 to be ‘children’ and are likely to include details about them despite the instruction to include only ‘children under the age of 18’. To address these discrepancies, Question 103 could be amended to include a row for adult children. Note that this question is also included in the parent questionnaire (question 36). However, no implausible values were detected amongst the parent respondents for this question—this may be due to the smaller overall sample size for parents or potentially relate to the fact that parents of boys may have younger children in general and therefore these issues were not identified in this study instrument.Under-18 year olds: Time goes to sleep and wakes up fieldsBoys CAPI: Question 59 - 62Young Men Questionnaire: Question 95 - 98Parent Questionnaire: Question 5 - 8There were a number of discrepancies across the three study instruments for these questions (see REF _Ref357595348 \r \h Table 15). In the parent questionnaire, there were seven instances where it appeared that the parent had specified an incorrect time (for example, a time for waking up was specified as 7:00 - 8:30 PM and a time of going to bed was specified as 9:30 - 11:00 AM). Even in the boys CAPI there were a couple of times that seemed a little unusual (for example, goes to sleep on a school/work night at 9:30 AM). There are two possible sources of the error on this question: (i) the respondent/interviewer got confused and specified the time for waking up and going to sleep using the wrong fields or (ii) the AM/PM checkbox was incorrectly used (see REF _Ref356993544 \r \h \* MERGEFORMAT Figure 3 for question layout). It was more difficult to identify likely errors for the young men questionnaire, given that respondents could also be working and there was likely to be more variation in their sleeping patterns. Given these discrepancies, the value of information provided by these fields should be considered. If retained, it is possible that some of the discrepancies could be taken care of with some data recoding rules. Implausible Values for Time Goes to Sleep and Wakes Up FieldsStudy InstrumentImplausible Values by Field Number (% of total respondents)Time goes to bed / sleep (school / work night)Time wakes up (school / work day)Time goes to bed / sleep (no school / work the next day)Time wakes up (non-school / work day)Boys CAPI n = 371 (2.7%)0 (0%)2 (5.4%)0 (0%)Parent Questionnaire n = 350 (0%)1 (2.9%)2 (5.7%)4 (11.4%)Young Men Questionnaire n = 271 (3.7%)1 (3.7%)0 (0%)1 (3.7%)Layout of Time Goes to Bed and Wakes Up Fields in the Parent QuestionnaireFor numeric fields that did not have a definitive range of plausible values, the range of values reported was checked to assess whether: i) the accepted range of values for the variable was appropriate for capturing all possible values, and ii) whether the variable may still contain some implausible values. Overall, review of the ranges of values for fields without a definitive range of plausible values did not identify any issues with the ranges set for these variables in the phase 1 pilot dataset.Performance of Questionnaire Skip LogicConditional sections in the adult, young men and parent questionnaires were assessed (see REF _Ref357000360 \r \h Table 24 in Appendix REF _Ref356830777 \r \h 7.1 for a list of these) by considering the proportion of respondents falling into each of the following categories who answered the section:Respondents who met the criteria for answering the section Respondents who did not meet the criteria for answering the section Respondents who did not answer the stem question, which determined whether or not the respondent was required to complete the section. The completion of the section was evaluated by whether or not respondents in each of these categories answered any of the conditional items in the section. Given that the skip logic in the programming for the boys CAPI directed the question routing, the performance of skip logic for the boys CAPI was not assessed.For all three questionnaires all respondents who met the criteria for answering the section answered at least one question in that section. Generally only a small number of respondents (a maximum of five respondents for any one study instrument) to whom the section did not apply answered some of the items in that section. For the most part those responses tended to be null responses that confirmed the respondents’ response to the stem question however, in a few cases, it did appear that the respondent had missed the skip or provided information when they weren’t required to. In all cases, these responses could be managed by using forward-coding or back-coding data cleaning rules. Overall, it appears that the skip logic in the three self-complete questionnaires is performing as intended.Inconsistent Response ChecksWithin Study Instrument Items with Multiple Response OptionsFor ease of completion and to maximise the number of responses obtained, some numeric write-in fields in the study instruments offered respondents the option of responding in different units or using different variables (see REF _Ref357000927 \r \h Figure 4 for some examples and REF _Ref356895105 \r \h Table 25 in Appendix REF _Ref356893706 \r \h 7.1 for a full list of these fields). For example, the length of time young men respondents had been in a relationship could be reported in weeks, months or in years and months and some numeric write-in fields offered respondents the option of using a ‘none’ checkbox for null responses. Where a respondent responded using more than one of the response options, the consistency of the information provided using the different response options was assessed. In general, there was a very low level of multiple response with between 0 and 19% of respondents using more than one response option for any particular item. Across all of the study instruments, multiple responses were provided for just under half of the items with multiple response options. Where more than one response option was used, the data was generally consistent in all cases. For multiple response option fields where respondents had the option of responding using a number field or ‘none’ checkbox, respondents tended to answer using either field and sometimes using both fields—in all cases where the number field and ‘none’ checkbox were used simultaneously, the data were consistent. For the height, weight and waist circumference fields, anthropometric measurements reported using more than one response unit for response were generally very similar, with minor differences most likely due to the different precision of the multiple units of response. Hence, no major issues were identified with numeric write-in items that had multiple response options. Examples of Questions with Multiple Response OptionsA) Young Men Questionnaire Question 48bB)Adult Questionnaire Question 33Inconsistencies between Separate ItemsFor some items within a study instrument, it is possible that respondents may provide data at one item which contradicts the data provided in another item—for example, the age reported for first smoking is greater than current age. Logic checks were conducted to evaluate the consistency of information provided by respondents where it was possible for the respondent to make contradictory responses. Overall, none of the checks on data from the boys CAPI (0/ 10), 10% of the checks on the young men questionnaire (3/30) and 20% (5/25) of checks on the adult questionnaire and 44% (5/9) of checks on the parent questionnaire detected an inconsistent response. Most of the inconsistencies detected were reported by a single respondent only. The following three questions in particular seemed prone to inconsistencies, though not at a high level: Health condition and symptom questionsAdult questionnaire: Question 17 and Question 18Young Men questionnaire: Question 25a and Question 25bParent questionnaire: Question 3 and Question 4Inconsistencies were related to the respondent marking ‘yes’ to ‘have you been treated for or had any symptoms of this condition in the past 12 months’ (column 2) but ‘no’ to ‘has a doctor or other health professional ever told you that you have this condition?’ (column 1). Generally only a handful of respondents reported these inconsistencies. Of note, inconsistencies were detected for the ‘wheezy breathing’ symptom in both the adult and young men questionnaires, when rows for the other health conditions or symptoms rarely resulted in inconsistent responses. It is possible that the phrasing of the headings for column 1 and 2 may have resulted in respondents reporting these inconsistencies—i.e., the respondent felt they had experienced wheezy breathing in the past 12 months but wheezy breathing had never been diagnosed by a doctor. Children under 18 who the respondent lives with Adult questionnaire: Question 104Parent questionnaire: Question 36There were discrepancies related to the total number of children (biological, adopted and step children) the respondent reported having overall (adult question 91b; parent question 44) and the number of children they reported living with (adult question 104; parent question 36) : in a few cases, respondents reported living with more sons, daughters and step children than they reported having overall. The inconsistencies with the health condition and symptom variables could be removed using data recoding rules, but in some cases may also reflect the respondent’s intended response and therefore reflect a possible issue with the item. The inconsistencies for the children under-18 questions are one of a number of issues identified with these questions which appear to be problematic in general. Between Study Instruments Discrepancies between the Responses of Boys and Their Parents A handful of items included in the boys CAPI were also included in the parent questionnaire. The items included in both study instruments fell into two major categories: Basic demographic variables: the son’s and parent’s country of birth, the Aboriginal and Torres Strait Islander status of the boy, whether or not the boy was still at school and whether the boy lived in one or two homes, as well as how often the boy lived in another home if they lived in more than one home. For these items, the purpose of including the construct in both questionnaires was to determine the validity of data provided by the boy. Variables for which the parent’s perspective was of interest and may be expected to vary slightly from their son’s perspective: their son’s general health status and the time their son typically goes to bed and wakes up on school and non-school day. Logic checks were also used to evaluate the consistency of information provided by boys and their parent. There were very few inconsistencies (one to two respondents at most) in the responses provided by boys and their parents for the demographic variables. This suggests that the boy is likely to be an accurate informant of the responses for these variables, such that it is not essential that these variables be included in the parent questionnaire. However, it may be worthwhile retaining these variables for other reasons (e.g., to keep the parent occupied with completing their questionnaire while their son is being interviewed). For the health status and time of waking and going to bed variables, a higher proportion of inconsistencies were detected. In 70.6% of cases the response provided for the boy’s general health status (question 12 of the boys CAPI and question 2 of the parent questionnaire) differed between the boy and their parent. However, for 85% of responses to this question overall, the parent’s response was within ±1 of the boys response on the 5-point Likert response scale. For the time of waking and going to bed variables, between 64.7% and 85.3% of responses were found to be inconsistent, with a higher frequency of inconsistencies for the non-school day fields. These discrepancies are to some degree expected given that the time of waking and sleeping questions are phrased slightly differently in the boys CAPI and parent questionnaire (boys CAPI: question 59 – 63; parent questionnaire: question 5 - 8), and therefore capture slightly different information, and the fact that the boy and their parent have slightly different sets of knowledge about the boy’s sleeping patterns. However, given that a reasonable proportion of implausible values were detected for the time of waking and sleeping fields (see section REF _Ref356899148 \r \h 3.4), it is likely that data cleaning rules will need to be applied before any comparisons can be made between the boy’s and parent’s reports of the boy’s sleeping patterns. Aside from values which appeared to be implausible, time estimates varied in ways that would seem reasonable, with the times reported for most response pairs being within 1 - 2 hours of each other. Evaluation of ‘Other, Specify’ Field Responses‘Other, specify’ fields were included in the study instruments for some closed-coded categorical variables in order to:Capture rare response categories efficiently; Assess whether the chosen options for categorical variables represent the most common response categories; andDetermine the number of ‘other, specify’ fields that may be required for the main wave of data collection. ‘Other, specify’ fields were evaluated by:Assessing the proportion of respondents who provided an ‘other, specify’ response;Assessing the number of ‘other, specify’ fields utilised by respondents (where more than one ‘other, specify’ response option was provided);Assessing the utility of the ‘other, specify’ free-text responses in terms of providing meaningful information in relation to the main categorical variable. For this analysis, ‘other, specify’ responses were either classified according to established codeframes (for example, Australian Bureau of Statistics codeframes for languages, countries, occupations and employment industries) or inductively, based on the responses provided where an established codeframe did not exist or cover all possible options. REF _Ref340822390 \r \h Table 26 in Appendix REF _Ref356831568 \r \h 7.1 provides an overview of the ‘other, specify’ fields in the study instruments. Generally, all respondents to whom ‘other, specify’ fields applied provided an open-text response. This varied from between 1.4% to 78.4% of the sample for individual questions. In most cases the ‘other, specify’ field applied to fewer than 20% of respondents in the total sample. Where multiple ‘other, specify’ fields were available, these were generally under-utilised, such that the total number of ‘other, specify’ fields could be reduced. In terms of assessing the utility of the ‘other, specify’ responses, the following questions were found to have issues: Over-the-Counter (OTC) Medication Adult questionnaire: Question 16bIn total, 7% of adults gave an open-text response with respect to OTC medication none of which captured useful information: four respondents listed a type of supplement and one listed prescription analgesics and other prescription medication. No genuine OTC medications were captured by the open-text fields. It is possible respondents may not understand ‘over-the-counter medication’ as the researchers intend. An explanation of the distinction between OTC medication, supplements and prescription medications may improve the utility of the open-text fields.Actions regarding Health Concerns Adult questionnaire: Questions 141 to 143Young Men questionnaire: Question 39 Boys CAPI: Question 27The open-text field was not highly utilised for these questions. Moreover, ‘other, specify’ responses provided tended to relate to the contextual nature of the health concern, for example ‘it depends on the concern’. The issue that the course of action and source of information for a health concern would depend on the nature of the concern was also raised in the cognitive testing of the questionnaires. However, it would be difficult to account for these contextual considerations without major changes to the item or additional items to probe these contextual differences. Other People Live With Young Men questionnaire: Questions 12, 16 and 19Boys CAPI: Questions 8b, 9c and 10cThis was the only item which received almost no ‘other, please specify’ responses: only one open-text response was given across all questionnaires, and that response could be back-coded into the existing options. Given this result and the comprehensiveness of the existing options, the ‘other, specify’ option is probably not necessary for this question.With respect to the questions for which ‘other, specify’ responses were included as a means to select the optimal codeframe, the following results were observed: Supplements TakenAdult questionnaire: question 49bTwenty-six percent of respondents provided at least one response. Overall, 10% indicated they used some form of multivitamin. Hence, this option should be considered for inclusion in the listed options. Several other supplement types including Vitamin C, Magnesium/Iron and Protein Supplements, were given by multiple respondents and should also be considered for inclusion in the response set.Item Format ChecksMatrix-Style QuestionsMatrix-style questions in the adult, young men and parent questionnaires that consisted of rows of items requiring a yes/no answer were assessed to determine if respondents tended to answer using both the ‘yes’ and ‘no’ response options or whether respondents were more likely to just use the ‘yes’ response for applicable items and leave those that did not apply to them blank instead of marking ‘no’ (see REF _Ref357002268 \r \h Figure 5 for some examples of these types of questions). Response patterns were assessed by computing a variable to record the number of rows for which the respondent answered ‘yes’, the number of rows they answered ‘no’ and the number of rows where they did not provide an answer. The average number of rows with yes, no and missing responses across all respondents was then calculated. The proportion of respondents who responded to the matrix questions using only ‘yes’ was also calculated. In general, most respondents answered using both the ‘yes’ and ‘no’ response options as intended, however, for the items ‘Adults you live with’ (adults question 103, parents question 35) and ‘Children you live with’ (adults question 104, parents question 36) respondents tended to only answer ‘yes’ to rows that applied to them. It would be straightforward to recode responses for the ‘adults you live with’ item from missing to ‘no’ for rows that respondents who answered at least one row in the question did not answer. However, the ‘children you live with’ question has broader issues with other elements in the question and may require simplification for optimal completion.The health condition and symptom variables have yes/no response option columns for lifetime condition/symptom prevalence and past 12 month incidence (see REF _Ref357002268 \r \h Figure 5A for an example). Most respondents answered both of columns (89 – 99% of respondents). Where both columns were not answered, respondents were most likely to complete only column 1 (lifetime) or leave both columns blank. Given that there were not a high number of inconsistencies detected for these questions (see section REF _Ref355941908 \r \h 3.6), recoding rules could be applied to recode the response to column 1 or 2 based on the response to the other column if only one column is answered.Examples of Matrix-style Questions with Yes/No Response OptionsA) Adult Questionnaire Question 17: Health Conditions ExperiencedB) Adult Questionnaire Q104: Details of Children Live WithSingle Response Items Captured as Multiple Response ItemsFor a select number of items in the adult, young men and parent questionnaires (see REF _Ref340647569 \r \h Table 27 in Appendix REF _Ref356831791 \r \h 7.1), data was captured as multiple response items, even though the items were designated as single response items in the questionnaires. This was done to assess whether respondents tended to answer these questions as single response or multiple response items. For the most part, these items were answered as single response items, with only 5.7% of respondents at most answering any of the items with more than one response. The ‘main language spoken at home’ question (adults Q5; parents Q46) was most prone to a low level of multiple response, and may warrant reclassification as a multiple response variable. Comparison between Pilot PhasesTo evaluate if the recruitment method is associated with major differences in completion of the study questionnaires, the missing data analysis of data elements (described in section REF _Ref355943838 \r \h 3.1.1) was repeated using the adult male data from the phase 2 pilot study (household recruitment). REF _Ref355943915 \r \h Table 16 provides a summary of the phase 2 results and a comparison to the results of the phase 1 pilot study (mail-out recruitment). Overall, the proportion of missing data for the adult questionnaire appeared to be similar in both pilots; however, the estimated mean and median proportion of missing data per item was slightly higher in the phase 2 pilot. Given the smaller sample size for the phase 2 pilot, and the corresponding broader confidence interval around the proportion of missing data, it is possible that the higher estimate from the phase 2 pilot may be an artefact of less precise estimates of missing data. Items that were more likely to have missing data in the phase 1 pilot data were also the items more likely to have missing data in phase 2, indicating that recruitment method is unlikely to have a major effect on the performance of the study instruments. Summary of Missing Data Analysis By Study Instrument#Adult Questionnaire Missing Data Analysis Results By Study PilotCharacteristicPhase 1 pilot study(‘mailout’ pilot)Phase 2 pilot study(‘drop & collect’ pilot)Total Number of Respondents7040Total Number of Variables538538Number (%) of variables with >0% Missing Data 296 (55.0)321 (59.7)Number (%) of variables with >10% Missing Data 54 (10.0)75 (13.9)Number (%) of variables with >20% Missing Data 38 (7.1)63 (11.7)Median % Missing Data Per Variable (Range)1.43 (0 - 100)2.5 (0 – 100)Mean % Missing Data Per Variable (Standard Deviation)5.1 (13.8)9.3 (21.0)# Proportion of missing data calculated for each questionnaire item as the proportion of respondents to whom the item applied who did not provide a response for the questionnaire item.A comparison of summaries of the item response fractions by page number, between phase 2 adult respondents and phase 1 adult respondents was also conducted. The results are presented in REF _Ref357002600 \r \h Figure 6. The average median response fractions, summarised by page number, was 98.4% for phase 1 and 95.1% for phase 2 indicating that almost all questionnaire items were answered by most respondents irrespective of page number. As REF _Ref357002633 \r \h Figure 6 shows, there is no evidence of response fractions declining with increasing page number in either pilot. For the phase 2 pilot, only one page had a dramatically different response fraction to phase 1 (page 18; variables about frequency of seeing children who live elsewhere). That page only applied to a handful of respondents (fewer than five in both cases), such that the difference is likely to be an artefact of a small sample size. Overall, it appears that the recruitment method has no effect on the respondent’s desire to complete the study questionnaire. Adult Questionnaire Median Variable Response Fraction by Page NumberA)B)Vertical Axis – Median Variable Response Fraction (%); Horizontal Axis – Page Number. A) Phase 1 pilot study B) Phase 2 pilot studyREFINEMENT OF STUDY INSTRUMENTSAt the conclusion of the instrument evaluation process, a set of recommendations to guide revision of the study instruments for Wave 1 data collection were produced as described in section REF _Ref356825896 \r \h 2.3. REF _Ref356982656 \r \h Table 17 provides a summary of the total number of items for which recommendations were made and for which specific actions were taken. REF _Ref356824773 \r \h Table 18 to REF _Ref355967685 \r \h Table 21 provide the main recommendations and actions taken for each study instrument in more detail. Overall, given that the evaluation of the performance of the study instruments described in section REF _Ref355945275 \r \h 3 only identified a handful of items with some performance issues, other considerations such as reducing participant burden predominated in recommendations for refinement of the study instruments. A limited number of items were removed from the study instruments (10.4% of items across all four study instruments overall); primarily, items were modified or data cleaning rules devised to handle any issues identified with the items.Summary of Recommendations and Actions Taken for Each Study InstrumentParameterProportion of Questions By Study InstrumentNumber (%)Boys CAPIParent QuestionnaireYoung Men QuestionnaireAdult QuestionnaireNumber of questions overall64 (100%)57 (100%)101 (100%)144 (100%)Number of questions for which recommendations were made16 (25%)15 (26%)27 (27%)37 (26%)Actions taken in relation to recommendationsData cleaning rules devisedQuestion modifiedQuestion removed4 (6.3%)5 (7.8%)7 (10.9%)5 (8.8%)2 (3.5%)8 (14.0%)7 (6.9%)8 (7.9%)12 (11.9%)15 (10.4%)11 (7.6%)11 (7.6%)Boys CAPI Refinement - Recommendations and Actions Taken ItemPerformance Issues IdentifiedRecommendationsActions TakenQ8b / 9c / 10c – Who usually lives with youUnder-utilisation of ‘other, specify’ fieldConsider removing this item as the information collected may not lead to meaningful analyses and removal will reduce participant burden. Also the ‘other, specify’ field is not necessary. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ15 & 16 – How many days have breakfast before starting usual activities on weekdays and weekendsNo major issues identifiedFruit and vegetable consumption (Q13 & 14) are adequate markers of diet quality: Consider removing Q15 & 16 on the grounds of uncertain reliability and reduction of participant burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ21 – Impact of pain on activitiesNo major issues identifiedSingle non-specific question of limited analytic value; remove item to reduce participant burden FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ24 – What does first when has a health concernUnder-utilisation of ‘other, specify’ field and limited value of ‘other, specify’ responses providedRemove ‘other, specify’ field as not well utilised and unlikely to provide meaningful information. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):‘Other, specify’ field removed FORMCHECKBOX Item removedQ34 b, c and d – Frequency of bullying by various modes.No major issues identifiedRevise response set to ‘yes/no’ response options rather than the current five-point Likert scale to reduce participant recall burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Response set amended from Likert scale to ‘yes/no’ response option set. FORMCHECKBOX Item removedQ = Question REF _Ref356824773 \r \h \* MERGEFORMAT Table 18 continuedItemPerformance Issues IdentifiedRecommendationsActions TakenQ44c / d – Number of times smoked or used marijuana in life or in past 4 weeksNo major issues identifiedRevise Likert response set for consistency with young men questionnaire FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Likert response set revised FORMCHECKBOX Item removedQ45c / d – Number of times sniffed glue in life or in past 4 weeksNo major issues identifiedRevise Likert response set for consistency with young men questionnaire FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Likert response set revised FORMCHECKBOX Item removedQ46c / d – Number of times used phenoxydine in life or in past 4 weeksNo major issues identifiedRevise Likert response set for consistency with young men questionnaire FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Likert response set revised FORMCHECKBOX Item removedQ47c / d – Number of times used other illegal drugs in life or in past 4 weeksNo major issues identifiedRevise Likert response set for consistency with young men questionnaire FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Likert response set revised FORMCHECKBOX Item removedQ59 – Q63 – Time goes to sleep and wakes up on a school and non-school night/dayImplausible values issuesIn light of implausible values, consider recoding rules for problematic values FORMCHECKBOX Data cleaning rules (specify):Devise recoding rules for implausible extreme values that appear to be due to the incorrect use of AM / PM or time of waking / going to sleep fields FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedParent Questionnaire Refinement - Recommendations and Actions Taken ItemPerformance Issues IdentifiedRecommendationsActions TakenQ3 – health conditions ever been diagnosed with and experienced in the past 12 monthsSome inconsistent responses and missing data issuesApply logical recoding rules to recode missing responses, particular for people who only used the ‘yes’ response option and for people who did not answer both columns. FORMCHECKBOX Data cleaning rules (specify):Data recoding rules to be applied as recommended. FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ4 – symptoms experienced Some inconsistent responses and missing data issuesFor simplicity and reliable data collection, omit column 2. Lifetime prevalence is sufficient for planned analyses. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Column 2 (past 12 months) omitted FORMCHECKBOX Item removedQ5 – Q8 – Time goes to sleep and wakes up on a school and non-school night/dayImplausible values issuesIn light of implausible values, consider recoding rules for problematic values FORMCHECKBOX Data cleaning rules (specify):Devise recoding rules for implausible extreme values that appear to be due to the incorrect use of AM / PM or time of waking / going to bed fields FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ14 – Medical services visited in the past 12 monthsUnder-utilisation of ‘other, specify’ fieldsReduce number of ‘other, specify’ fields. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Number of ‘other, specify’ fields reduced to one. FORMCHECKBOX Item removedQ25 – Q27 – Transport to schoolNo major issues identifiedConsider deleting to ease participant burden. Not a sentinel baseline question. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removed REF _Ref355962187 \r \h \* MERGEFORMAT Table 19 continuedItemPerformance Issues IdentifiedRecommendationsActions TakenQ31 & Q32 – description of homeNo major issues identifiedConsider deleting to ease participant burden. With a drop and collect sampling methodology, the fieldworker will be able to record this information. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ33 – Who home is rented from No major issues identifiedConsider deleting. Not a sentinel baseline question and measure of housing stress captured elsewhere. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ35 & Q36 – Adults / children live withMissing dataImplausible valuesInconsistent responsesGiven questionnaire performance issues, consider deleting. Information about household composition may also be obtained by other means if the drop and collect methodology is used. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedSurvey Completion DateImplausible valuesConsider removing this field since it had some performance issues and the data is not essential. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedYoung Man Questionnaire Refinement - Recommendations and Actions Taken ItemPerformance Issues IdentifiedKey RecommendationsActions TakenQ10 – Transportation to place of study or workNo major issues identifiedConsider deleting to ease participant burden. Not a sentinel baseline question. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ12 / Q16 / Q19 – Who usually lives with youUnder-utilisation of ‘other, specify’ fieldConsider removing item as information collected may not lead to meaningful analyses and removal will reduce participant burden. Also the ‘other, specify’ field not necessary. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ20 & 21 – description of home No major issues identifiedConsider deleting to ease participant burden. With a drop and collect sampling methodology, the fieldworker will be able to record this information. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ25a – health conditions ever been diagnosed with and had in the past 12 monthsSome inconsistent responses and missing data issuesApply logical recoding rules to recode missing responses, particular for people who only used the ‘yes’ response option and for people who did not answer both columns. FORMCHECKBOX Data cleaning rules (specify):Data recoding rules to be applied as suggested. FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ25b – symptoms experiencedSome inconsistent responses and missing data issuesFor simplicity and reliable data collection, omit column 2. Lifetime prevalence sufficient for planned analyses. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Column 2 (past 12 months) dropped FORMCHECKBOX Item removedQ27 – Impact of pain on activitiesNo major issues identifiedSingle non-specific question of limited analytic value. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ = Question REF _Ref355964154 \r \h \* MERGEFORMAT Table 20 continuedItemPerformance Issues IdentifiedKey RecommendationsActions TakenQ35 – waist circumferenceMissing data issuesConsider removing this item given the high level of missing data identified in field testing. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ39 – What does first when has a health concernUnder-utilisation of ‘other, specify’ field and limited value of ‘other, specify’ responses providedRemove ‘other, specify’ field as not well utilised and unlikely to provide meaningful information. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):‘Other, specify’ field removed FORMCHECKBOX Item removedQ45 a & b – Frequency of bullying by various modes.No major issues identifiedRevise response set to ‘yes’ response options rather than the current five-point Likert scale to reduce participant recall burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Response set amended from Likert scale to ‘yes’ response option set. FORMCHECKBOX Item removedQ48 – Current relationshipNo major issues identifiedConsider removing. Not a sentinel baseline question. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ57 & 58 – How many days have breakfast before starting usual activities on weekdays and weekendsNo major issues identified Consider removing Q57 & 58 on the grounds of uncertain reliability and reduction in participant burden. Fruit and vegetable consumption (Q56a & 56b) are adequate markers of diet quality. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removed REF _Ref355964154 \r \h \* MERGEFORMAT Table 20 continuedItemPerformance Issues IdentifiedKey RecommendationsActions TakenQ75 / 77 / 78 / 79 / 80 – Substance use questionsNo major issues identifiedAdd a screening question (i.e., ‘have you ever taken . . .’) so it is easier to determine the subgroup of respondents to whom the questions about age of first use and frequency of use apply. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Screening questions added FORMCHECKBOX Item removedQ81 – Antisocial behaviour questionsNo major issues identifiedConsider removing. Sensitive question and risk profile can be determined from drugs, alcohol and risk-taking questions. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ88 – 90 – Thoughts, plans and actions about killing yourselfSome missing data issuesRecode missing responses for the ‘past 12 months’ variable to ‘no’ responses if the respondent answered ‘no’ to the ‘ever’ variable. FORMCHECKBOX Data cleaning rules (specify):Recoding rules to be applied as suggested FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ95 – Q98 – Time goes to sleep and wakes up on a school and non-school night/dayImplausible values issuesIn light of implausible values, consider recoding rules for problematic values FORMCHECKBOX Data cleaning rules (specify):Devise recoding rules for implausible extreme values that appear to be due to the incorrect use of AM / PM or time of waking / going to sleep fields. FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedSurvey Completion DateImplausible values issuesConsider removing this field since it had some performance issues and the data is not essential FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedAdult Questionnaire Refinement - Recommendations and Actions Taken ItemPerformance Issues IdentifiedRecommendationsActions TakenQ16 a / b – over-the-counter medicationsInvalid ‘other, specify’ responsesReformat question to improve quality of data captured. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Question reformatted to ask specifically about the use of selected over-the-counter medications with a Likert response scale for the frequency of use. Screening question no longer required. FORMCHECKBOX Item removedQ17 – health conditions ever been diagnosed with and experienced in the past 12 monthsSome inconsistent responses and missing data issuesApply logical recoding rules to recode missing responses, particular for people who only used the ‘yes’ response option and for people who did not answer both columns. FORMCHECKBOX Data cleaning rules (specify):Data recoding rules to be applied as suggested. FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ18 – symptoms experiencedSome inconsistent responses and missing data issuesFor simplicity and reliable data collection, omit column 2. Lifetime prevalence sufficient for planned analyses. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Column 2 (past 12 months) dropped FORMCHECKBOX Item removedQ22 – waist circumferenceMissing data issueConsider removing this item given the high level of missing data identified in field testing. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ23 – screening question for prostate cancerNo major issues identifiedAmend age of respondents to whom the item applies from 40 to 50 years to reduce burden among respondents less likely to have experienced symptoms. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Age amended from 40 to 50 years FORMCHECKBOX Item removedQ = Question REF _Ref355967685 \r \h \* MERGEFORMAT Table 21 continuedItemPerformance Issues IdentifiedRecommendationsActions TakenQ33 – 42 – Active Australia SurveySome missing data issues, mostly for null responsesApply sensible recoding rules to recode missing values given complexity of format. FORMCHECKBOX Data cleaning rules (specify):Data cleaning rules to be applied as described. FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ43 & 44 – Time spent sitting or outdoorsImplausible valuesConsider adding a minutes field in addition to the hours field. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Minutes field added FORMCHECKBOX Item removedQ48 a & b – How many days have breakfast before starting usual activities on weekdays and weekendsNo major issues identifiedConsider removing Q48a & b on the grounds of uncertain reliability and reduction in participant burden. Fruit and vegetable consumption (Q46 & 47) are adequate markers of diet quality. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ49 a / b – Supplements takenSome missing data issues for null responsesAdd multivitamin as an option, change the response set to a Likert scale (removing the need for the screener question) and reduce the number of ‘other, specify’ fields FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Suggested modifications made FORMCHECKBOX Item removedQ71 – Personal Wellbeing IndexNo major issues identifiedAdd item about satisfaction with job to this scale so Q119 can be deleted to reduce participant burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):Row for satisfaction with job added to Q71 FORMCHECKBOX Item removed REF _Ref355967685 \r \h \* MERGEFORMAT Table 21 continuedItemPerformance Issues IdentifiedRecommendationsActions TakenQ83 & 84 – Age of first sexual experienceHigh missing data for ‘months’ fieldDelete the ‘months’ field as many respondents did not answer in years and months. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):‘Months’ field deleted FORMCHECKBOX Item removedQ85 & 86 – Number of female and male sexual partners High missing data for number of male sexual partnersRecode missing values based on responses to Q80 & 81 FORMCHECKBOX Data cleaning rules (specify):Missing values for ‘number of males’ fields to be recoded to ‘0’ for heterosexual respondents. FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ93a and 93b – How often sees and has over-night stay with children who live elsewhereSome missing data issuesConsider removing these items. Question format complex, and unlikely to provide interpretable information in any analyses. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ96 & 97 – Description of home No major issues identifiedConsider deleting to ease participant burden. With a drop and collect sampling methodology, the fieldworker will be able to record this information. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ100 – Who home is rented from No major issues identifiedConsider deleting. Not a sentinel baseline question and measure of housing stress captured elsewhere. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ103 & 104 – Adults / children live withMissing dataImplausible valuesInconsistent responsesConsider deleting given questionnaire performance issues. Information about household composition may also be obtained by other means if the drop and collect methodology is used. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removed REF _Ref355967685 \r \h \* MERGEFORMAT Table 21 continuedItemPerformance Issues IdentifiedKey RecommendationsActions TakenQ105 – how many pets hasMissing data issuesLimited analytic potential; therefore, consider deleting to reduce participant burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ119 - Job satisfactionNo major issues identifiedDelete question and add item on job satisfaction to end of Q71 FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ123 – Transportation to place of study or workNo major issues identifiedNot a sentinel baseline question, so consider deleting this question to reduce participant burden FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ133 – Type of discrimination experienced.No major issues identifiedConsider deleting this question since it has limited analytic potential. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ137 Why no private health insurance coverNo major issues identifiedNot a sentinel question for baseline. Consider deleting to reduce participant burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedQ141 – What does first when has a health concernUnder-utilisation of ‘other, specify’ field and limited value of ‘other, specify’ responses providedRemove ‘other, specify’ field as not well utilised and unlikely to provide meaningful information. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify):‘Other, specify’ field removed FORMCHECKBOX Item removed REF _Ref355967685 \r \h \* MERGEFORMAT Table 21 continuedItemPerformance Issues IdentifiedKey RecommendationsActions TakenQ142 – Source of trusted medical adviceNo major issues identifiedOverlaps with another question on help-seeking behaviour therefore consider deleting to reduce participant burden. FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedSurvey Completion DateImplausible values issuesConsider removing this field since it had some performance issues and the data is not essential FORMCHECKBOX Data cleaning rules (specify): FORMCHECKBOX Item modified (specify): FORMCHECKBOX Item removedIMPLICATIONS FOR WAVE 1Analyses conducted to evaluate the performance of the Wave 1 study instruments in pilot testing overwhelming showed that the four study instruments (the boys CAPI, the parent questionnaire, the young men questionnaire and the adult questionnaire) were performing well and capturing data as intended. Overall, there was a low level of missing data across all four study instruments and there were seldom issues related to the reporting of implausible values or inconsistent responses. Only a handful of more than 1000 variables evaluated were found to have some performance issues. Problematic fields tended to be numeric write-in fields or fields with more complicated formats. Often issues related to missing data were related to missing null responses or missing responses that could be reasonably inferred from other data provided. In general, the findings of the analyses agreed well with the findings of the cognitive testing phase of questionnaire development and the conclusions drawn by Roy Morgan Research. Exploration of variable response fractions by page number did not identify any length issues with the questionnaire. It appeared that respondents who started completing the questionnaire intended to complete it. However, this analysis does not allow us to evaluate if the length of the questionnaire was a deterrent to commencing completing the questionnaire. Similarly to the findings of the per page response fraction analysis, exploration of the variables response fractions by topic did not seem to reveal any significant issues with the likelihood that respondents would answer a particular topic of questions. Rather, variation in the response fractions summarised by topic appeared to reflect questions with problematic formats. Where issues were identified, for the most part it seemed that these issues could be rectified with appropriate data cleaning rules or some minor simplifications in the question format. As a result, decisions related to removal of questionnaire items were to a greater extent informed by other considerations, such as participant burden. Overall, only minor modifications were made to the piloted questionnaires.A notable limitation of the analyses presented in this report is the smaller than anticipated number of respondents from the mailout pilot study. As a result, the estimates of the frequency of questionnaire performance issues are likely to be somewhat imprecise and may be subject to sampling variation. Furthermore, since the mailout pilot respondents tended to be of high socioeconomic status in general, it is possible that a higher frequency of performance issues may have identified with a more diverse sample of respondents. However, comparisons between the performance of the adult questionnaire amongst the mailout pilot respondents and the drop & collect pilot respondents did not identify any major differences in the performance of the adult questionnaire between the two samples of respondents, in spite of a more diverse sample of adult respondents being recruited in the drop and collect pilot. However, the sample of adult respondents recruited in the drop and collect pilot was also small, such that it is possible that this small sample of respondents was not large enough to identify all possible issues that may arise with a different sampling frame and/or more diverse sample of respondents. In conclusion, while this analysis does have some limitations, it appears that the piloted Wave 1 study instruments are generally performing well and do not require major changes to ensure valid data is captured in the baseline wave of the study. ADDIN EN.SECTION.REFLIST REFERENCES ADDIN EN.REFLIST 1.Department of Health and Ageing, National Male Health Policy: Building on the Strengths of Australian Males, Department of Health and Ageing, Editor 2010, Commonwealth of Australia: Barton ACT.2.Schlichthorst, M., et al., Ten to Men - The Australian Longitudinal Study on Male Health: An Introduction to the Study Design and Plan, 2012, The University of Melbourne Melbourne.3.Ten to Men Study Team, Ten to Men - The Australian Longitudinal Study on Male Health: Technical Report #2, 2013, The University of Melbourne: Melbourne.4.Ten to Men Study Team, Ten to Men - The Australian Longitudinal Study on Male Health: Dress Rehearsal Analysis Plan Version 1 (17 December 2012), 2012, The University of Melbourne: Melbourne.5.Koelmeyer, R., Ten to Men 2012 Dress Rehearsal: Study Instrument Evaluation Report 1, 2013, The University of Melbourne: Melbourne.6.Sumpton, C., Ten to Men 2012 Dress Rehearsal: Study Instrument Evaluation Report 2, 2013, The University of Melbourne: Melbourne.7.Carskadon, M.A. and C. Acebo, A Self-Administered Rating Scale for Pubertal Development. Journal of Adolescent Health, 1993. 14: p. 190-195.APPENDICESLists of Specific Questionnaire ItemsItems Composed of Composite Data ElementsBoys CAPIYoung Men QuestionnaireAdult QuestionnaireParent QuestionnaireQ1 – Weight #Q2 – country of birthQ2 – country of birthQ5 – time son goes to sleep on a school/work nightQ2 - Height #Q14 – time spent living in first homeQ3 – mother’s country of birthQ6 – time son wakes up on a school/work dayQ3 – Waist measurement #Q17 – time spent living in second homeQ4 – father’s country of birth Q7 – time son goes to sleep on a non-school/non-work nightQ5 – country of birthQ33 – Height #Q5 – language spoken mainly at homeQ8 – time son wakes up on a non-school/non-work dayQ9a – time spent living in first homeQ34 - Weight #Q20 – Height #Q24b – time son lives elsewhereQ10a – time spent living in second homeQ35 – Waist measurement #Q21 - Weight #Q28 – son’s country of birthQ17 – duration of physical activity in the past 7 daysQ44 – age first had oral sexQ22 – Waist measurement #Q38 – amount pays for homeQ29a – mother’s country of birthQ45 – age first had vaginal or anal sexQ34 – time spent walking for leisureQ45 – parent’s country of birthQ29b – father’s country of birthQ48b – Length of relationship #Q36 – time spent walking as a means of transportQ59 – time goes to sleep on a school/work nightQ49a – mother’s country of birthQ38 – time spent doing vigorous gardeningQ60 – time wakes up on a school/work dayQ49b – father’s country of birthQ40 – time spent doing vigorous physical activityQ61 – time goes to sleep on a non-school/non-work nightQ59 – duration of physical activity in the past 7 daysQ42 – time spent doing moderate physical activity Q62 – time wakes up on a non-school/non-work dayQ95 – time goes to sleep on a school/work nightQ54 – Cigarettes smoked per day (ex-smokers)Q96 – time wakes up on a school/work dayQ55 - Cigarettes smoked per day (current smokers)Q97 – time goes to sleep on a non-school/non-work nightQ83 – age first had oral sexQ98 – time wakes up on a non-school/non-work dayQ84 – age first had vaginal or anal sexQ93b – number of overnight stays for children living elsewhere (child 1 – 5)Q101 – amount pays for home# These fields had multiple response options that needed to be consolidated into a single variable/measurement scale.Validated Measures in the piloted Wave 1 Study InstrumentsStudy Instruments Containing MeasureValidated MeasureBoysCAPIParentQuestionnaireYoung Men QuestionnaireAdult QuestionnaireThe Active Australia Survey (Australian Institute of Health and Welfare, 2003)Antisocial behaviour scale (Hawkins, Catalano & Miller, 1992)The Alcohol Use Disorders Identification Test (AUDIT; Barber, Higgins-Biddle, Saunders & Monteiro, 2001) Children with Special Health Care Needs (CSHCN) Screener (The Child and Adolescent Health Measurement Initiative, 1998)Conformity to Masculine Norms Inventory (22-Item Version - CMNI-22; Mahalik, Locke, Ludlow et al. 2003)HILDA Study Job Quality Measure (Leach, Butterworth, Rodgers et al., 2010)The MOS Social Support Survey (Sherbourne & Stewart, 1991)PedsQL General Well-Being Scale (Varni, 1998)Personal Wellbeing Index - Adult (The International Wellbeing Group, 2006)PHQ-9 Depression Rating Scale (Kroenke, Spitzer & Williams, 2001)PHQ-9 Modified for Teens (Spitzer, Williams & Kroenke et al., 2001)Pubertal Development Scale (Carskadon & Acebo, 1993)Self Determination Scale (Ryan & Deci, 2002)SF12 v2.0 Health Survey (Ware, Kosinski & Keller, 2002)Single-item Measure of Job Satisfaction (Warr, Cook and Wall, 1979)Spence Children’s Anxiety Scale (Spence, 1997)Washington Group Short Set of Questions on Disability (Washington Group on Disability Statistics, 2006)Conditional Sections in the Self-Complete QuestionnairesSection DescriptionRelevant Question Numbers for Each Study InstrumentParent QuestionnaireYoung Men QuestionnaireAdult QuestionnaireAlcohol behaviour-67 - 7456 - 63Contact with children under 18 who live elsewhere--92 - 95Description of most severe injury9 -1328 - 3227 - 30Employment details50 - 557 - 9108 - 119Information about drugs other than marijuana-76 - 80-Information about household (one home only)-11 - 13-Information about households (two or more homes)-11 - 19-Modified International prostate symptom score--24 - 25Over-the-counter medications--16a/bSexual behaviour-43 - 4782 - 88Smoking behaviour-62 - 6650 - 55Son travel to school25 - 27--Supplements taken at least once a week--49a/b- = not applicableNumeric Write-in Fields with Multiple Response OptionsItem/ Multiple Response Options/Question Number for Relevant Study Instrument(s)BoysParentsYoung MenAdultsHeightCentimetresFeet & inches‘Don’t know’ checkbox2-3320WeightKilogramsStones & poundsPounds only‘Don’t know’ checkbox1-3421Waist circumferenceCentimetresInches‘Don’t know’ checkbox3-3522Number of sessions of walking for leisureNumber of times‘None’ checkbox---33Number of sessions of walking for transportNumber of times‘None’ checkbox---35Number of sessions of vigorous gardeningNumber of times‘None’ checkbox---37Number of sessions of vigorous physical activityNumber of times‘None’ checkbox---39Number of sessions of moderate physical activityNumber of times‘None’ checkbox---41Number of cigarettes smoked - ex-smokersCigarettes per dayCigarettes per week---54Number of cigarettes smoked - current smokersCigarettes per dayCigarettes per week---55Age when first had oral sexAge in years and months‘Haven’t had oral sex’ checkbox--4483Age when first had vaginal or anal sexAge in years and months‘Haven’t had vaginal or anal sex’ checkbox--4584Length of relationshipWeeksMonthsYears and months--48b-Number of children haveNumber of children‘None’ checkbox---91a & b REF _Ref356895105 \r \h \* MERGEFORMAT Table 25 continuedItem/ Multiple Response Options/Question Number for Relevant Study Instrument(s)BoysParentsYoung MenAdultsNumber of times children living elsewhere stay overnightNumber of nights per time period‘Never’ checkbox---93bAmount household pays for homeAmount per time period‘Nil payments’ checkbox-38-101- = not applicable – item not included ‘Other, specify’ fields in the piloted Wave 1 Study InstrumentsQuestion Number By Study Instrument‘Other, specify’ field descriptionNumber of individual ‘other, specify’ fieldsCodeframe for responsesAdultsYoung MenBoysParentOther country of birth1ABS Standard Australian Classification of Countries codeframe (second edition, 22 Aug 2011)22528Other country of birth (parent/guardian)1ABS Standard Australian Classification of Countries codeframe (second edition, 22 Aug 2011)---45Other country of birth (mother)1ABS Standard Australian Classification of Countries codeframe (second edition, 22 Aug 2011)349a29-Other country of birth (father)1ABS Standard Australian Classification of Countries codeframe (second edition, 22 Aug 2011)449b29-Other language spoken mainly at home1ABS Australian Standard Classification of Languages codeframe (second edition, 16 Aug 2011)53a6a46Main occupation1ABS 1220.0 Australian and New Zealand Classification of Occupations (First Edition Revision 1)110--52Employment Industry1ABS 1292.0 Australian and New Zealand Standard Industrial Classification (2006 Revision 1.0)111---Other people live with (one home only)1Inductive-128b-Other people live with (1st home)1Inductive-169c-Other people live with (2nd home)1Inductive-1910c-Other home ownership arrangements1Inductive but considering 2011 Census Dictionary Classification99--33aOther qualification completed1Inductive but considering 2011 Census Dictionary Classification122--49Other over-the-counter medications4Inductive but considering NHS 2007-08 Classification of Medicines16b---Other supplements taken5Inductive but considering related NHS 2007-08 items and classification of medicines49b---Other health professionals consulted1Inductive but considering related NHS 2007-08 items 134---Other medical services visited4Inductive but considering related NHS 2007-08 items---14Other reasons not to get health care1Inductive139---Other things done first when have a health concern1Inductive1413927-Other source trust most for medical advice1Inductive142---Other person who makes medical appointments1Inductive143---Other forms of discrimination1Inductive133---Single Response Variables Captured as Multiple Response VariablesAdult QuestionnaireYoung Men QuestionnaireParent QuestionnaireQ5 – language mainly speak at homeQ20 – description of home Q31 - description of homeQ50 – ever smoked Q62 – ever smoked Q46 – language mainly speak at home Q56 – ever had an alcoholic drink Q67 – ever had an alcoholic drink Q50 - current employment status Q92 – whether children sometimes or always live elsewhere Q96 – description of home Q108 –current employment status Q111 – employment industry Additional DocumentsDocumentPage NumberDress Rehearsal Data Analysis Plan63Boys CAPI (Paper mark up)167Parent Questionnaire203Young Man Questionnaire215Adult Questionnaire235 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download