How Many Subjects Does It Take To Do A Regression Analysis.

Multivariate Behavioral Research

ISSN: 0027-3171 (Print) 1532-7906 (Online) Journal homepage:

How Many Subjects Does It Take To Do A Regression Analysis

Samuel B. Green

To cite this article: Samuel B. Green (1991) How Many Subjects Does It Take To Do A Regression Analysis, Multivariate Behavioral Research, 26:3, 499-510, DOI: 10.1207/ s15327906mbr2603_7 To link to this article:

Published online: 10 Jun 2010. Submit your article to this journal Article views: 3059 Citing articles: 853 View citing articles

Full Terms & Conditions of access and use can be found at

Multivariate Behavioral Research, 26 (3), 499-510 Copyright O 1991, Lawrence Erlbaum Associates, Inc.

How Many Subjects Does It Take To Do A Regression Analysis?

Samuel B. Green

University of Kansas

Numerous rules-of-thumb have been suggested for determining the minimum number of subjects required to conduct multiple regression analyses. These rules-of-thumb are evaluated by comparing their results against those based on power analyses for tests of hypotheses of multiple and partial correlations. The results did not support the use of rules-of-thumb that simply specify some constant (e.g., 100 subjects) as the minimum number of subjects or a minimum ratio of number of subjects (N) to number of predictors (m). Some support was

obtained for a rule-of-thumb thatN ;r 50 + 8m for the multiple correlation and N ;r 104 + m for

the partial correlation. However, the rule-of-thumb for the multiple correlation yields values too large for N when rn ;c 7, and both rules-of-thumb assume all studies have a medium-size relationship between criterion and predictors. Accordingly, a slightly more complex iule-ofthumb is introduced that estimates minimum sample size as function of effect size as well as the number of predictors. It is argued that researchers should use methods to determine sample size that incorporate effect size.

"How many subjects does it take to do a regression analysis?" It sounds like a line delivered by a comedian at a nightclub for applied statisticians. In fact, the line would probably bring jeers from such an audience in that applied statisticians routinely are asked some variant of this question by researchers and are forced to respond with questions of their own, some of which have no good answers. In particular, because many researchers want a sample size that ensures a reasonable chance of rejecting null hypotheses involving regression parameters, applied statisticiansarelikely topresenttheirresponseswithin apower analyticframework. From this perspective, sample size can be determined if three values are specified: alpha, the probability of committing a Type I error (i.e., incorrectly rejecting the null hypothesis); power, one minus the probability of making a Type I1error (i.e., not rejecting a false null hypothesis); and effect size, the degree to which the criterion variable is related to the predictor variables in the population. Although alpha by tradition is set at .05, the choice of values for power and effect size is less clear and, in some cases, seems rather arbitrary.

As an alternative to determining sample size based on power analytic techniques, some individuals have chosen to offer rules-of-thumb for regression analyses. These rules-of-thumb come in various forms. One form indicates that the number of subjects,N, should alwaysbe equal to or greater than someconstant,

MULTIVARIATE BEHAVIORAL RESEARCH

499

S. Green

A (i.e., N >A), while a second form stipulates a recommended minimum ratio B of subjects-to-predictors (i.e., N 2 B m where m is the number of predictors). Finally, a third form is a more general rule that encompasses the first two (i.e., N zA+Bm).

TabachnickandFidell(1989)suggest,with somehesitancy,intheirmultivariate text that the minimum number of subjects for each predictor or independent variable (IV) in a regression analysis should be 5-to-1. They state the following:

If either standard multiple or hierarchical regression is used, one would like to have 20 times more cases than IVs. That is, if youplan to include 5 IVs, it would be lovely to measure 100 cases. In fact, because of the width of the errors of estimating correlation with small samples, power may be unacceptably low no matter what the

cases-to-IVs ratio if you have fewer than 100 cases. However, a bare minimum

requirement is to have at least 5 times more cases than IVs - at least 25 cases if 5 IVs

are used. (pp. 128-129)

In the first edition of a multivariate primer by Harris (1975), he recommended that

the number of subjectsN > 50 + m. In the revision of his text, he states,

...not too different from the Primer's earlier suggestion that N - m be > 50. More

common is a recommendation that the ratio of N to m be some number, for example,

10. I know of no systematic study of the ratio versus the difference between Nand m

as the importantdeterminant of sampling stability of the regressionweights. However,

ratio rules break down for small values of m ...and there are hints in the literature that

the difference rule is more appropriate. (1985, p. 64.)

Harris' difference rule is an example of the general rule-of-thumb o f N r A +Bm

with A = 50 and B = 1. Harris7call for a study to compare the ratio and difference rules canbe expanded andrecastwithin the contextof the present paper as a request for a study to evaluate whether the A constant is a necessary component of the general rule-of-thumb. If A is unnecessary, the general rule-of-thumb simplifies to the ratio rule-of-thumb, N 2 B m.

Nunnally (1978) makes slightly different recommendations based on an examination of an equation for determining an unbiased estimate of the population squared multiple correlation coefficient (A2)from the sample squared multiple correlation coefficient (R2):

(1)

k2= 1- (1 - R2)[(N - 1)l(N - m)].

He states,

If there are only 2 o r 3 independent variables and no preselection is made among them, 100or more subjects will provide a multiple correlation with little bias. In that case,

500

MULTIVARIATE BEHAVIORAL RESEARCH

S. Green

if the numberof independentvariablesis as large as9 or 10,itwill benecessary to have from 300 to 400 subjects to prevent substantial bias. (p. 180)

Other individuals, based on a variety of justifications specify somewhat different rules-of-thumb. For example, Marks (1966), as cited in Coolley and Lohnes (1971), recommended a minimum of 200 subjects for any regression analysis, while Schmidt (1971) suggested a minimum subject-to-predictor ratio ranging in value from 15-to-1 to 25-to-1. Still other individuals, for example, Pedhazur (1982), discuss rules-of-thumb, but make no general set of recommendations themselves.

An important question is whether researchers who use these rules-of-thumb have designed studies with adequate power. The answer to this question is unknown; however, it is known that many empirical studies do have insufficient power (for a review of this literature, see Cohen, 1988). Perhaps some portion of thesestudieshave used rules-of-thumbforregression analysis(e.g., 5-to-1subjectto-predictor ratio). The purpose of the present study is to determine if researchers who apply such rules-of-thumb are designing studies with low power. Sample sizesforregressionanalyseswillbe determinedfollowingrecommendationsmade by Cohen (1988)in the second editionof hisbookon power analysis. These results will be compared against various rules-of-thumb to judge their adequacy. It was anticipatedthat none of the reviewed rules-of-thumbwould be satisfactoryin that all of them ignore effect size and, accordingly, recommendations for more complex rules-of-thumbwould be required. It is hoped that this presentationwill encourage researchers to struggle with the difficult decisions required by power analysiswith thereaiizationthatsimplisticrules-of-thumbignorethe idiosyncratic characteristics of research studies. In addition, it is hoped that the discussion will encourageappliedstatisticiansto developmethodsfor thedeterminationof sample size that researchers find less esoteric and more useful (see Harris, in press, for one such alternative).

The methods employed in this study impose a few limitations on the conclusions. The sample sizes were based on power tables presented in Cohen (1988) and are slightly different from those that would be obtained using other power tables (for a more in-depth discussion, see Gatsonis & Sampson, 3 989). Also, thepower analysesassumethatthe regression analysesincludea11predictors and do not allow for the preselection of predictor variables (e.g., stepwise regression). More generally, in order to have a focused presentation, it was necessaryto developmethodswhich suggestthat decisionmakingwith regression analysis is simpler than what should occur in practice. For example, researchers need to consider carefully prior to determining sample size whether they can reduce the number of predictor variables by forming a priori linear combinations of two or more predictors and whether the variables are reliable and are highly

MULTIVARIATE BEHAVIORAL RESEARCH

501

S. Green

interrelated. Most importantly, researchers need to attend seriously to the issue of estimating effectsize based on their knowledge of a research area and the methods of their study rather than assume a value for an effect size offered by Cohen or discussed in this paper (see Cohen, 1988 for a detailed discussion).

Sample Sizes Required to Evaluate Multiple Correlation

Coeficients with a Power of .SO

I began by consideringwhat sample size is required to evaluate the hypothesis that the multiple correlation between the predictors and the quantitative variable

01) is equal to zero with a power of 3 0 . To conduct power analyses, choices of

values for alpha, power, and effect size were made: 1.Alpha was set at .05, the traditional level of significance. 2. Power was set at .80, a value proposed by Cohen (1988) as appropriate for

a wide range of behavioral research areas. He argued that the setting of power, the probability of not committing aTypeI1error, isto some extentarbitrary,but should be dependent on the loss associated with making this error. Inasmuch as power is partially a function of alpha, it also should be a function of the loss associated with a Type I error. He suggested that typically across the behavioral sciences, a 4 to 1ratio reflects the relative seriousness of a Type I error to a Type I1 error. Consequently, when alpha is set equal to .05, the probability of a Type I1 error should be 4 x .05 = .20 and power would be 1- .20 = 3 0 .

3. Cohen (1988) stresses two indexes of effect size for regression anaIysis,f and the better known R2. The two indexes are directly related:

Although Cohen argues that the choice of values for effect size (R2or f ) should depend on the research area, he proposes, as a convention,R2sof .02, .13, and .26 (f2 of .02, .15, and .35) to serve as operational definitions for the descriptors small, medium,andlarge,respectively. Cohendiscussestheseselectionsratherextensively and indicates that they agree with his subjective judgment of small, medium, and large effect sizes obtained in behavioral sciences. These three values are used in the current study.

Calculations as outlined by Cohen (1988) were performed to determine sample sizes for the selected values of alpha, power, and effect size using tables @p. 448-455) he provided for this purpose. The tables have entries in which the number of predictors for regression analyses may assume 23 different values, ranging from 1predictor to 120predictors. While sample sizes were determined for all 23 values, for sake of simplicity, only those for 15 of the 23 values are presented in the first three columns of Table 1. Sample sizes are not presented in

502

MULTIVARIATE BEHAVIORAL RESEARCH

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download