Table of Contents



Research & Evaluation Review BinderTable of Contents 1. Aqua Tab - # 1 – Research Context Types of Empirical Research – Page # 1 Experimental Vs. Non-experimental – Causal-comparative study– Independent and Dependent Variables Sampling – Page # 2simple random sampleBiasScales of Measurement – NOIR – Page # 2NominalOrdinalIntervalRatio Quantitative VS. Qualitative – Page # 3Checklist of Guidelines – For literature Review Literature Review Practice 2. Blue Tab - # 2 – Descriptive Statistics Frequency – Page # 1PercentageProportion Communative frequency Communative % Shapes of Distributions – Page # 2Frequency polygon – Bell curveFrequency TablePositive skewNegative skewBimodal skew Mean, Median, Mode – Page # 3 & 4DefinitionsExampleRange and Interquartile Range – Page # 4DefinitionsExampleStandard Deviation – Page # 5 & 6Bell CurveComputation of the Standard Deviation 3. Purple Tab - # 3 Correlational Statistics Correlation – Page # 1UnivariateBivariateMultivariateDirect and Inverse relationships ScattergramPearson r – Page # 2Meaning of #sCoefficient of Determination – Page # 2R squaredMultiple Coefficients – Page # 34. Green Tab - # 4 Inferential Statistics Variations on Random Sampling Biased and Unbiased Sample – Page # 1Freedom from biasRandom sample tableSampling – Page # 2 & 3Stratified random sampleRandom cluster sample Multistage random sample Standard Error of the Mean – Page # 4Hypothesis – Page # 3 & 4Directional/ non-directionalAlternative NullType 1 Error – Page # 4Type 2 Error – Page # 4t – Test formula – Page # 4 & 5t – Test Results (Also, see handout) – Page # 6One-Way ANOVA – Page # 6F testTwo-Way ANOVA – Page # 6Chi Square – Page # 7Handouts:Research Power PointNull HypothesisSignificant TestsStandard Error of the MeanType 1 and Type 2 Errorst – test ANOVA/ MANCOVA Decision TreeUse to determine research ANOVA Table Critical Values of t Test5. Neon Green Tab - # 5 Instrumentation Validity – Page # 1 – 3 Types of ValidityThreats to Internal and external validity Types of Reliability 6. Gray Tab # 6 – Experimental Design Page # 1 & 2Pretest-posttest randomized control groupPosttest-only randomized control groupSolomon randomized four-group designTrue experimental design7. Blue Tab # 7 – Research Definitions8. Purple Tab # 8 – Sample Questions 9. Light Blue Tab # 9 – Abbreviations10. Blue Tab # 10 – Appendix11. Dark Blue Tab # 11 - HW questions and answers12. Green Tab # 12 – In class Exercise questions 13. Neon Green Tab # 13 – Miscellaneous Abbreviationsr? - is the symbol for the coefficient of determination. f = the symbol for frequency.N stands for the number of participants, which is also used to stand for frequency. M and m - The most commonly used symbols for the mean in academic journals are x? - X Bar is also the mean S – the uppercase, italicized letter S is the symbol for the standard deviation of a population. ?? - Eta squared (effect size)d – symbol for effect size Ho = null hypothesis Ha or H1 = alternative hypothesis SEm = standard error of the mean (symbol is σx?) SEP = standard error of the proportion (symbol is σp?) df = degrees of freedomR = multiple correlation coefficientX2 = Chi Square Research Definitions Alternative Hypothesis – regular hypothesis. There is a significant difference between ________ and __________Bell Curve or Normal Curve – a normal distribution with a high point. The most commonly found curve in nature. About 2/3 of the cases lie within one SD unit of the mean. 68% lies within 1 SD unit of the mean. It is used as the basis for a number of inferential statisticsBias – unfair/not an equal change to participate. It is created when some members of a population have a greater chance of being selected than others. Bimodal distributions – has 2 high points and are less frequently found in research Causal-comparative study (aka Ex post facto study) – a non-experimental study in which researchers observe and describe some current condition (lung cancer) and then researchers look to the past to try to identify the possible cause(s) of the condition. Census – is a study in which all members of a population are includedCluster sampling - is when researchers draw groups (or clusters) of participants instead of drawing individuals. Coefficient of determination – to obtain square r. When converted to a percentage, it indicated how much variance on one variable is accounted for by the variance on the other. Communative frequency – how many individuals are in and below a give score level Concurrent Validity Coefficient – is obtained by administering the test and collecting the criterion data at about the same time. Confounded Variable – experiment, variables, or results are flawed, screwed-up, undesirable. (Example: A flaw that was out of the experimenter control, creates an untrue representation)Correlation - is the general term that refers to the extent to which two variables are related across on group of participants.Correlational Research – researchers are interested in the degree of relationship among 2 or more quantitative variables. Ex. Scores on college admission test and GPA, they are quantitative and because individuals vary on both of them, they are variables. Construct validity – is a type of validity that relies on subjective judgments and empirical data. (Construct = collection of related behaviors that are associated in a meaningful way. Content Validity – make judgments on the appropriateness of its contents. Vital for achievement tests. Correlation – refers to the extent to which 2 variables are related across a group of participants. Correlation coefficient ( r? ) – used to check on the degree of the relationship. Value varies from 0.00 to 1.00. Covariate variable – upping the language – means “adjusting for _________” Which could be a variable that affected the study. Allows for fair playing field to make it a control. Criterion Validity – Should correlate with content validity. Use when you want to know if test scores correlate with what you are trying to master. Cumulative percentage – how many individuals are in and below a give percentDegrees of freedom ( df ) – not descriptive statistics. Calculated to obtain probability that the null hypothesis is correct. Changes with different formulas. Can change – depends on what is being studied. For example: df – 2 = T TestDemographics – background characteristics such as: sex, socioeconomic status, religion, age, race, etc.Dependent Variable – are the responses in an experiment Direct or positive relationship – those who score high on one test tend to score high on the other test. And those who score low on one variable tend to score low on the other variable. Effect size: (?? or d )– refers to the magnitude (ie size) of a difference when it is expressed on a standardized scale. ERIC database – is free of charge and available to anyone with access to a computer with the Internet. (eric.) Empirical – is based on observation. Whom, how, when, circumstance Experiment - Is a study in which treatments are given to see how the participants respond to them. External validity - To whom and under what circumstance can the results be generalized? Face validity – judgments are made on whether an instrument appears to be valid on the face of it. Ex. On superficial inspection, does the instrument measure what it is supposed to measure?Frequency – number of people in an experiment or number of people that received a certain scoreFrequency distribution – is a table that shows how many participants have each scoreFrequency polygon – is a drawing that shows has many participants have each scoreFormative Evaluation – information that is collected on the process of implementing a program. Gaussian (Curve) distribution – is commonly called the "normal distribution" and is often described as a "bell-shaped curve".Generalizability – How much you can take information and apply it to the rest of society. How does that population affect the rest of us? Ex. Not generalizable because it did not discuss ethics. Independent Variable – is the treatment or group difference in an experimentInferential Statistics – is the name of the branch of statistics that has statistical techniques that can be used to test the truth of the null hypothesis.Instrument - the generic term for any of type of measurement device Internal Consistency Reliability – use when you want to know if the items on a test assess one, and ONLY one dimension.Internal Validity - Is the treatment, in this particular case, responsible for the observed change (s)? Interrater (interobserver) Reliability – when you want to know whether there is consistency in the rating of some outcome. *Informs you of how much you can believe in scores. Interval – tells us how much participants differ from each other. NO absolute zeros.Interquartile range (IQR) - is defined as the range of the middle 50% of the participants. *It is better than the range because it ignores outliers. To compute: divide numbers into equal groups. Subtract inner numbers to obtain IQR.Levene Test – It is a test of normality – distribution of data. Use to test normality assumptions – ANOVA. You don’t want Levine score to be significant. Likert Scale – is a measuring tool that ranges from “strongly agree to strongly disagree” Mean – is the balance point in a distributionMedian - Has a value of a distribution that has 50% of the cases above it and 50% of the cases below it. It is an average that is defined as the middle point in a distribution. *it is insensitive to extreme scores Mode - An average that is defined as the most frequently occurring score.Multiple correlations – is the correlation between combination of two variables with a 3rd variable. Negative skew - long tail to the left, the median is higher than the meanNominal – NAMING (categorical) because names are used instead of numbers. – HIGHEST level of measurement. Nonexperimental study (aka descriptive study) - is a study in which observations are made to determine the status of what exists at a given point in time without the administration of treatments. (NO treatments are administered.)Null hypothesis – there is nothing going on. No significant group. No difference between…Operationalizing a variable – redefining a variable in terms of physical steps, he or she is creating an operational definition. *Making terms useable/workableOrdinal – puts participants in rank order from high to low. Does not tell the difference between highest and lowest. NO VALUE!Parallel Forms Reliability – Use when you want to know if several different forms of a test are reliable or equivalent. You correlate the scores from one form of the test, with scores from a second form, of the same test of the same content, but not exact same test. Ex. GREParsimony – means simple – most simplest form Pearson r - is the relationship between 2 sets of scores Phenomenology – What we see happening around us. Pilot studies (in reference to sample size) - are studies designed to obtain preliminary information on how new treatments and instruments work. Population – consists of all members of a group. Yield parametersPositive Skew – long tail to the right, the mean is higher than the medianPosttest-only randomized control group (design 2) – fixes the pretest sensitization problem caused in pretest-posttest. No PRETEST. Post-Hoc – (After ANOVA) Will tell you where the significance is. Ex. TukeyPrecision - defined by statisticians is the extent to which the same results would be obtained if another random sample were drawn from the same population. *technical term for discussing the magnitude of sampling errors.*Pretest-posttest randomized control group (design 1) – assigns participants at random to groups, researchers are assured that there are no bias in the assignment. Pretest sensitization (AKA reactive effect of testing) – is the result of a combination of the pretest and the treatment. This problem can be overcome by conducting an experiment without a pretest. Program evaluation – is almost applied research. For example, It is research in which researchers wish to apply the findings directly to such practical decisions as whether to continue funding the program and/or whether to modify it. Purposive sampling - is when researchers purposively select individuals who they believe will be good sources of information. This could be done by observing over a long period of time and choosing only to interview a certain individual or group of individuals of interest. Quantitative Research – Allows for statistical analysis. Formal interview – same questions, same format. Qualitative Research – free flowing interview, not as formal. Research is more language based.Random errors – are created by random selection, are called sampling errors by statisticians. Random sampling- identifies and unbiased sample Range – Difference between highest and lowest scoresRatio - tells us how much participants differ from each other. Has an absolute zero. Sample – is a subset of the population. Ex. 3 students in the math class were surveyed; the 3 students would be the sample. *Yields statisticsSamples of convenience – is biased because you pick who you want in the study or who is convenient to use. Sampling errors – are errors that are created by random samplingSelectional Bias – produces sampling flaws Simple Random Sampling - Is used to eliminate bias in the selection of individuals for a study, it gives each member of a population an equal chance of being selected.Solomon randomized four-group design – is the Best of both designs. It is a combination of Pretest-posttest randomized control group and Posttest-only randomized control groupStandard deviation – is the most frequently used measure of variability. SD is statistic that provides an overall measurement of how much participants’ scores differ from the mean score of their group. 68% unitStratified random sampling – is used to reduce sampling errors. This is stratification in conjunction with random sampling. *IS UNBIASED*Summative evaluation – is the attainment of the ultimate goal at the end of a program. Systematic sampling - is a type of random sampling where every nth individual is selected. The number “n” can be any number. For example, n could equal 2 which would select every second individual.Test-Retest Validity – you use it when you want to know if a test is reliable. Requires 2 test sessionsThreats to internal validity – explanations for changes, other than the treatment. Ex. history, maturation, instrumentation, testing, statistical regressionTrue experimental design – Pretest-posttest randomized control group, Posttest-only randomized control group, and Solomon randomized four-group design are all true experimental designs. They are all characterized by random assignments to treatments. Unbiased sample – gives every member of a population an equal chance of being included in the sample. Validity coefficient – one way to look at a test’s predictive validity is to compute the validity coefficient. It expresses validity. Variability – refers to differences among scores. Synonyms are spread and dispersionVolunteerism – major source of bias. Researchers can call for volunteers, creates bias. Sample QuestionsThe most valuable type of research is?The experiment, which is used to discover cause-and effect relationshipsExperiments emphasize parsimony, which means?interpreting the results in the simplest wayOccam’s Razor suggest that experimenters…..? interpret results in the most simplest wayNondirective is to person-centered as……? Parsimony is to Occam’s RazorAn experiment is said to be confounded when?Undesirable variables are not kept out of the experiment Experimenters should always abide by a code of ethics. The variable you manipulate/control in an experiment is the? Independent variable Hypotheses’ testing is mot closely related to the work of?R.A. Fisher In the social sciences the accepted probability level is?.05 or lessType II error…?Is called a beta error and accepts the null hypothesis when it is false. “p” in relation to a test of significance, equals? probabilityP = 0.5 really means? There is only a 5% chance that there is a difference.95 out of 100 time, the difference that exists. Significant level that is the best to rule out chance factors?.001Type I error occurs when?you reject null when it is trueType II error includes?Beta error and accepts the null when it is falseA counselor educator decides to increase the sample size in her experiment. This will?Reduce Type I and Type II errorsOne group receives no assertiveness training, a 2nd group receives four assertiveness training sessions, and 3rd group receives 6 sessions. Which statistic would you choose to use?ANOVA A researcher utilizes two IVs, the statistic of chose would be?The 2-way ANOVA or MANOVATo complete a t–test you would consult a table value of t. In order to see a significant differences exist in an ANOVA, you would use?table for F values To draw 20% of whites out of a group, you would use?Stratified sampling An operational definition?Outlines a construct – giving variables meaningMike takes an achievement test. In order to predict his scores, if he takes the test again the counselor must know?The standard error of measurement. *If you take the same test – you should score the same. If you don’t then their will be an error. What does a Likert scale measure?Collects data – agree, somewhat agree, neutral, etc. The mean is misleading when?It is skewed and has extreme scoresIf an experiment can be replicated by others, with almost identical findings, then the experiment is said to be?reliable Comparing 2 groups based on their behavior at the end, using a t test. This study could best be described as?Causal comparative research If an ANOVA yields a significant f value, you could rely on ___________ to test significant differences between group means. Post Hoc – ex. Like Tukey To write a bibliography for his thesis, I would most likely use?ERIC, search engine, for primary and secondary references Effect size is identified in literature in several ways, give 2 examples?Eta squared ?? or d One test (tool) had 75 questions; another test (tool) has 150 questions. I want to compare sets/data. What do I do?Factor effect size Why is it difficult to get an effect size higher than 3/-3?The formula is based on standard deviation and it hard to go above 99% equally. What is the importance of the standard error of the mean?confidence intervalDescribe 2 forms of reliability you feel are the most important and why?Test-retest – because it is reliable over timeInternal consistency – becauseDescribe 2 forms of validity you feel are the most important and why?Content, criterion, construct What is the only test of significance that can be uses for a nominal scale of data? Chi Square ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download