Appendix 1: The Journal Selection Process



The Journal Selection Process

We began by examining the disciplines within the business school as categorized by AACSB (1998) and their relative sizes as surveyed each year by AACSB (1998). AACSB (1998) defines 13 specific disciplines (or groups of disciplines) within the business school: accounting, corporate strategy/business policy/business and society, economics, finance, human resource management, insurance, international business, management/organizational behavior, marketing, management information systems, management science/operations research, production/operations management, and real estate.

We deliberately choose to exclude economics because this discipline is commonly found outside of the business school. We combined strategy, and human resource management into the overall management/organizational behavior category used by AACSB because of their relatively small size and their conceptual similarity, as has been done previously (e.g., Glick, et al., 1997; Johnson & Podsakoff, 1994; Niemi, 1988). Likewise, we combined insurance and real estate into one group due to their small size as was done previously (Niemi, 1988), and added international business to this group, also due to its small size. This resulted in eight discipline groups: accounting, finance, insurance/international business/real estate, management, management information systems, management science, marketing, and production/operations management.

Our goals were to develop a research "bread basket" of business school journals that was representative of research in the 13 business school disciplines defined by AACSB (1998). Within each discipline, there are many “good” journals. For example, Glick, et al.(1997) and Johnson & Podsakoff (1994) identify more than 30 “good” journals in the management group, which accounts for about one fifth of business school faculty (3,457 out of 15,474). While this number of journals is useful for evaluating one discipline, it becomes challenging to find and analyze the 150 or so "good" journals this number would imply for the business school as a whole.

Instead, we decided to focus on a smaller set of the "best" journals in each discipline. Thus our measure of business school research productivity is deliberately biased to only one form of business research: publications in leading research journals. We exclude other forms of research such as articles in "good" but lower ranked journals, conference papers, books, book chapters, and articles in practitioner journals.

Previous business school ranking research has used a set of 16 journals (Henry & Burch, 1974) and a set of 14 journals (Niemi, 1988). We decided to similarly sized set of 15 journals that had at least one journal from each of the disciplines in AACSB (1998). We selected a target number of journals in each discipline, relative to the total number of faculty in that discipline, so that the overall set would not overemphasize one discipline to the determent of others (our analyses also explicitly control for the size of each discipline). Since there are approximately 15,000 faculty in U.S. business schools (AACSB, 1998), this meant a target of about one journal per every 1000 faculty in a discipline, which resulted in targets of: 3-4 for accounting, 2-3 for finance, 3-4 for management, 1-2 for MIS, 2-3 for marketing, and 1 each for insurance, international business, real estate, management science, and production/operations management. We should emphasize that these were initial targets, not firm quotas, because maintaining a relative number of journals in each discipline and ensuring each discipline is represented are somewhat conflicting goals.

One problem with this discipline-based approach is that it forced us to categorize journals to specific disciplines. Management Science, for example, is a multi-disciplinary journal, serving several disciplines. Because the majority of Management Science articles are in the management science discipline, we classified it as a management science journal. The alternative was to parse out articles by discipline and then rank Management Science within each discipline, in which case, articles in some disciplines would be included (e.g., operations management) but articles in others would be excluded (e.g., management) because Management Science would not be of sufficient quality to be selected (see the analyses below). We believe this alternative of counting some but not all articles in Management Science is more problematic than simply assigning them all to the management science discipline.

We wanted to balance our selection of journals between perceptions of researchers and citation analyses because we believed both approaches can help triangulate on a small set of "best" journals (Jones et al., 1996). We began by searching for articles and unpublished reports rating journals within each discipline based on the perceptions of faculty in that discipline and based on citation analyses that were published within the same time period as our study (1986-1997). We also examined articles and unpublished reports ranking universities within disciplines and articles that analyzed previous research to identify the journals those studies used as the "best" in their disciplines.

From these lists we excluded practitioner-oriented journals, such as Harvard Business Review and Sloan Management Review. While these are valuable and important journals, our focus is on journals whose primary mission is the publication of academic research. Likewise, since our focus was on business research, we excluded journals published by organizations with "home" discipline outside of the business school (e.g., American Sociological Review and Journal of Applied Psychology for management, and Communications of the ACM and IEEE Transactions on Software Engineering for management information systems). We also excluded annual volumes and those with a high proportion of invited articles (e.g., Research in Organizational Behavior). Finally, since our focus was on U.S. business schools only, we excluded non-U.S. journals (e.g., Contemporary Accounting Research).

From each of these articles, we identified a candidate set of "best" journals (with the exclusions above). From each article, we recorded the top x journals in our candidate list, where x was the our maximum target for that area (e.g., top 4 for accounting, top 3 for finance). When articles reported an unordered set of "top" journals we examined them all and recorded the journal if some other article included it on their ranked list. The tables below present the candidate journals from this analysis. A number indicates the journal's rank in our set of discipline-specific journals, while "top" indicates it was present on an unordered list. In some cases, articles in the list simply reuse data from previous studies to identify their "top" articles in each discipline. We choose to include this reuse of previous rankings because it shows a selection of one of the prior rankings over another and reconfirms that a list of top journals published years before is still considered to be appropriate. Our ranking may differ from the ranking listed in the original source due to our exclusion of journals. For example, Johnson and Podsakoff (1994) rank management journals in the following order: 1. ASQ, 2. Research in Organizational Behavior, 3. AMR, 4. AMJ, 5. Journal of Applied Psychology, 6. SMJ, and 7. OBHDP. We exclude Research in Organizational Behavior (an annual volume) and Journal of Applied Psychology (published by the American Psychological Association) to get our top five management journals (ASQ, AMR, AMJ, SMJ, and OBHDP). Likewise, we omit journals published in other disciplines; for example, we omit the two finance journals (JF and JFQA) that Reeve and Hutchinson (1988) include in their list of accounting journals.

The next step was to triangulate these previous rankings with a cross-discipline citation analysis of journal rankings. For this step, we used the Social Sciences Citation Index (SSCI) Journal Citation Report (1995, 1996) that calculates the "impact" of each journal based on the number of times a journal is cited divided by the number of articles it publishes. We also used Starbuck's (1997) adjusted impact measure which uses time series analyses to predict future impact and adjusts for the average number of citations in each discipline. While the SSCI impact statistic has limitations, its offers a different perspective to help triangulate on the "best" journals. One problem is that SSCI does not calculate the impact factor for all journals, including several of our candidate journals. A second problem is that SSCI can be unstable from year to year (e.g., the impact factor for JFE increased by more than 50% from 1995 to 1996, moving it from third place to first place among finance journals). For a discussion of the limitations of the impact factor, see Linde (1998), Nisonger (1994), and Starbuck (1997). The tables below report the impact factor and rank based on impact factor for the top x journals in each discipline, where x is the number of target journals. Blanks indicate that the impact factor was not calculated for that journal.

We then selected the journals for our analyses. We first looked at the top journals based on the set of published articles rating journals. We then compared the top journals in this set to the top journals based on the citation analyses. Generally, the two produced the same two sets of journals. Where there were disagreements, we weighted the published articles more heavily than the citation analyses because there were usually more published articles and the citation analyses are more subject to year-by-year fluctuations.

In accounting, with a target of 3-4 journals, the top three journals were clear: JAR, AR, and JAE.. In finance, with a target of 2-3 journals, two journals were clearly the "best" (JF, JFE). JFQA and RFS are also highly ranked, but we chose to select the top two journals, rather than the top four. In insurance, we chose JRI, in real estate, REE, and in international business, JIBS. In management, with a target of 3-4 journals, four journals stood out: ASQ, AMJ, AMR, and SMJ. In management information systems, with a target of 1-2 journals, the top two were clear: MISQ and ISR. In management science, with a target of one journal, it was difficult to distinguish between MS and OR, so we chose to include both. In marketing, with a target of 2-3 journals, there were clearly three "best" journals: JM, JMR, and JCR. In production/operations management, with a target of one journal, MS was ranked highest. Since MS was already included in our management science list, and publishes research beyond production/operations management, we decided to choose a specialist journal that focused specifically on production/operations management. We re-examined the ranking articles to identify the journals ranked first or second, omitting another broad-spectrum journal (Decision Sciences) and choose JOM.

Finally, we reviewed the resulting list of 20 journals with 16 faculty members (two in each discipline) to ensure it seemed appropriate.

References

AACSB. 1997. Response to Information Request on Faculty Composition. American Association of Collegiate Schools of Business.

AACSB. 1998. Faculty Positions Filled, Vacant, and Planned Growth. Newsline 28, 3, 27.

Alexander, J.C., Jr., & Mabry, R. H. 1994. Relative significance of journals, authors, and articles cited in financial research. The Journal of Finance 49, 697-712.

Armstrong, J. S. & Sperry, T. 1994. The Ombudsman: Business School Prestige -- Research versus Teaching. Interfaces, 24: 2, 13-43.

Barman, S., Tersine, R. J., & Buckley, M. R. 1991. An Empirical Assessment of the Perceived Relevance and Quality of POM-Related Journals by Academicians. Journal of Operations Management 10 2, 194-212.

Baumgartner, H. & Homburg, C. 1996. Applications of Structural Equation Modeling in Marketing and Consumer Research: A Review. International Journal of Research in Marketing, 13, 2, 139-161.

Beatty, B. L., Lamy, R.E. Peacock, P. R. & Saladin, B.A. 1996. The faculty-in-residence program, Business Horizons 39, 55-64.

Borokhovich, K. A., Bricker, R. J., Brunarski, K. R. &. Simkins, B. J. 1995. Finance research productivity and influence, The Journal of Finance 50, 1691-1717.

Borokhovich, K. A., Bricker, R. J., Brunarski, K. R. &. Simkins, B. J. 1998. Finance research productivity and influence by Topic Area, The Journal of Financial Education, 24, Spring, 8-20.

Borokhovich, K. A., Bricker, R.J. & Simkins, B. J. 1994. Journal communication and influence in financial research, The Journal of Finance 49, 713-725.

Brown, L.D. & Huefner, R. J. 1994. The Familiarity with and Perceived Quality of Accounting Journals: Views of Senior Accounting Faculty in Leading U.S. MBA Programs. Contemporary Accounting Research, 11, 1 223-250.

Bush, A. J. & Grant, E. S. 1994. Analyzing the Content of Marketing Journals to Assess Trends in Sales Force Research: 1980-1992. Journal of Personal Selling & Sales Management, 14, 3, 57-68.

Byrne, J. A., 1990. Is research in the ivory tower ‘fuzzy, irrelevant, pretentious’?, Business Week, October 29, 1990, 62-66.

Cox, L. A., & Gustavson, S. G. 1990. Leading contributors to insurance research, The Journal of Risk and Insurance 57, 260-281.

D’Aveni, R.A. (1996) A Multiple-constituency, Status-based Approach to Interorganizational Mobility of Faculty and Input-output competition among top business schools. Organization Science, 7, 166-189.

Diaz, J., III, Black, R.T., & Rabinanski, J. 1996. A Note on Ranking Real Estate Journals. Real Estate Economics. 24, 4, 551-563.

Economist. 1997. Inside the Knowledge Factory, October 4.

Extejt, M. & Smith, J.E. 1990. The Behavioral Sciences and Management: An Evaluation of Relative Journals. Journal of Management 16, 539-551.

Fielden, J. S. 1988. Co-authorships made in heaven, Journal of Management Consulting 4, 24-27.

Fielden, J. S., & Gibbons, J. D. 1991. Merit myopia and business school faculty publications, Business Horizons 34, 8-12.

Franke, R. H., Edlund, T. W., & Oster, F. 1990. The Development of Strategic Management: Journal Quality and Article Impact. Strategic Management Journal 11, 3, 243-253.

Ganish, G.K., Chandy, P.R., Henderson, G.V. 1990. Awareness and Evaluation of Selected Marketing Journals Inside and Outside the Discipline: An Empirical Study. Akron Business & Economic Review, 21, 4 93-106.

Glick, W., McKelvey, W., Cooper, M., Huber, G., & Zmud, W. 1997. Survey Feedback from Surveys of Journal Reputations, Report of the 1997 INFORMS Committee Review of Organization Science.

Gillenson, M. L. & Stutz, J. D. 1991. Academic issues in MIS: Journals and books, MIS Quarterly, December 1991, 447-452.

Goh, C. H., Holsapple, C. W., Johnson, L. E., & Tanner, J. 1997. Evaluating and Classifying POM Journals. Journal of Operations Management. 15, 2, 123-138..

Gomez-Mejia, L.R. & Balkin, D.B. 1992. Determinants of Faculty Pay: An Agency Theory Perspective. Academy of Management Journal, 35, 5, 921-955.

Gordon, M.E. & Purvis, J.E. 1991. Journal Publication Records as a Measure of Research Performance in Industrial Relations. Industrial and Labor Relations Review, 45, 194-201.

Hardgrave, B.C. & Walstrom, K. A. 1997. Forums for MIS Scholars. Communications of the ACM, 40, 11, 119-124.

Henderson, G.V., Ganesh, G.K. & Chandy, P.R. 1990. Across-Discipline Journal Awareness and Evaluation: Implications for the Promotion and Tenure Process. Journal of Economics & Business, 42, 4, 325-351.

Henry, W. R. & Burch, E. E. 1974. Institutional Contributions to Scholarly Journals of Business. Journal of Business, 47, 56-66.

Hollman, K. W., Murrey, J. H., Jr., & Homaifar, G. 1991. The structure and disciplinary boundaries of insurance: A citational analysis of JRI articles, The Journal of Risk and Insurance 58, 714-721.

Holsapple, C., Johnson, L., Manakyan, H., & Tanner, J. 1994. Business Computing Research Journals: A Normalized Citation Analysis. Journal of Management Information Systems 11, 1, 131-140.

Hubbard, R. & Vetter, D.E. 1996. An Empirical Comparison of Published Replication Research in Accounting, Finance, Management, and Marketing. Journal of Business Research, 35, 2, 153-164.

Hull, R. P. & Wright, G. B. 1990. Faculty Perceptions of Journal Quality: An Update. Accounting Horizons, 4, 77-98.

Institute for Scientific Information, 1994-1997, Journal Citation Reports, Social Sciences Edition, Philadelphia.

Im, K.S., Kim, K.Y., & Kim, J.J. "An Assessment of Individual and Institutional Research Productivity in MIS," Decision Line, December/January, 1998, 8-12.

Johnson, J. L. & Podsakoff, P. M. 1994. "Journal Influence in the Field of Management: An Analysis using Salancik Index in a Dependency Network.. Academy of Management Journal, 37, 1392-1407.

Jones, M. J., Brinn, T. & Pendlebury, M. 1996. Journal Evaluation Methodologies: A Balanced Response. Omega-International Journal of Management Science 24, 5, 607-612.

Kaplan, R.S. & Norton, D.P. (1992) The Balanced Scorecard – Measures that Drive Performance. Harvard Business Review, January/February, 71-79.

Kirkpatrick, S. A. & Locke, E. A. 1992. The Development of Measures of Faculty Scholarship. Group & Organizational Management, 17, 5-23.

Kozub, R. M., Sanders, D.L. & Raabe, W. A. 1990. Measuring Tax Faculty Research Publication Records. Journal of the American Taxation Association, 12, 94-101

Lending, D. & Wetherbe, J.C. 1992. Update on MIS Research: A Profile of Leading Journals and U.S. Universities. DataBase 23, 3, 5-11.

Linde, A. 1998. On the pitfalls of journal ranking by impact factor. European Journal Of Oral Sciences 106, 1, 525-526.

Luke, R. H. & Doke, E.R. 1987. Marketing Journal Hierarchies: Faculty Perceptions, 1986-87. Journal of the Academy of Marketing Science 15,1 74-78.

Lukka, K. & Kasanen, E. 1996. Is Accounting a Global or Local Discipline? Evidence from Major Research Journals. Accounting, Organizations & Society, 21, 7, 755-773.

Mabry, R.H. & Sharplin, A. D. 1985. The Relative Importance of Journals Used in Finance Research. Journal of Financial Research, 8, 4, 287-296

Malhotra, M. K. & Kher, H. K. 1996. Institutional Research Productivity in Production and Operations Management. Journal of Operations Management, 14, 1 55-77.

March, J.G. 1996. Continuity and change in theories of organizational action. Administrative Sciences Quarterly, 41, 278-287.

March J.G. 1991. Exploration and exploitation in organizational learning. Organization Science. 2, 71-87.

March, J.G. & Sutton, R.I. 1997. Organizational performance as a dependent variable. Organization Science, 8, 698-706.

McDowell, J. M. & Amacher, R. C. 1986. Economic Value of an In-House Editorship. Public Choice, 48, 101-112.

MacMillan, I.C. 1989. Delineating a Forum for Business Policy Scholars. Strategic Management Journal, 10, 391-395

Meredith, J. R., Raturi, A., & Kaplan, B. 1989. Alternative Research Paradigms in Operations. Journal of Operations Management. 8, 297-326.

Northcraft, G. B., Neale, M. A., & Huber, V. L. 1988. Decision bias and social influence in human resource decision making. In K. Rowland & G. Ferris eds. Research in Personnel and Human Resource Management. Greenwich, CT: JAI Press, 157-189.

Niemi, A. W., 1987. Institutional contributions to the leading finance journals, 1975 through 1986: A note, The Journal of Finance 42, 1389-1397.

Niemi, A. W., 1988. Research productivity of American business schools, 1975-85, Review of Business and Economic Research, Spring 1988, 1-17.

Nisonger, T.E. 1994. A Methodological Issue Concerning the Use of Social-Sciences Citation Index Journal Citation Reports Impact Factor Data for Journal Ranking. Library Acquisitions: Practice and Theory 18, 4, 447-458.

Ohio State. 1997. Finance Department Research Publication Records, 1992-1996.

Pol, L.G. 1991. Demographic Contributions to marketing: An Assessment. Journal of the Academy of Marketing Science, 19, 1 53-59.

Prather, J. & Rueschhoff, N. 1996. An Analysis of International Accounting Research in U.S. Academic Accounting Journals, 1980 through 1993. Accounting Horizons, 10, 1, 1-17.

Reeve, R. C. & Hutchinson, P. J. 1988. The Contribution of Non-U.S. Institutions to Academic Accounting Journals. Abacus 24, 1 90-94.

Reisman, A. & Kirschnick, F. 1994. The Devolution of OR/MS: Implications from a Statistical Content Analysis in Papers in Flagship Journals. Operations Research, 42, 4, 577-588.

Reisman, A. & Kirschnick, F. 1995. Research Strategies Used by OR/MS Workers as Shown by an Analysis of Papers in Flagship Journals. Operations Research, 43, 5, 731-740.

Risks, D.A. 1992. The Filed of International Business and JBIS' Role in It. Journal of Business Administration, 21 2, 77-84.

Scudder, G.D. & Hill, C. A. 1998. A Review and Classification of Empirical Research in Operations Management. Journal of Operations Management, 16, 91-101.

Southern Methodist University 1997. Edwin L. Cox School of Business Annual Report, Dallas, Texas.

Sports Illustrated. 1997. Jock Schools U.S.A. April 28, 52-82.

Stahl, M. J., Leap, T. L. & Wei, Z. Z. 1988. Publication in leading management journals as a measure of institutional research productivity, Academy of Management Journal 31, 707-720.

Starbuck, W. 1997. Estimates of the 1997 Impact Ratings. Unpublished Web document. and

Tsui, A.S. (1990) A Multiple-constituency Model of Effectiveness: An Empirical Examination at the Human Resource Subunit Level. ???? 35, 458-483.

van Fleet, D. 1995. Journal Lists. Personal correspondence.

Vincent, V. C. & Moville, W. D. 1993. Ethical considerations for streaming business publications, Journal of Business Ethics 12, 37-43.

Vokurka, R.J., 1996. The Relative Importance of Journals Used in Operations Management Research: A Citation Analysis. Journal of Operations Management, 14, 345-355.

Walstrom, K., Hardgrave, B., & Wilson, R., 1995. Forums for Management Information Systems Scholars. Communications of the ACM, 38:3, 93-102.

Web, J.R. & Albert, J. D. 1995. Evaluating the Real Estate Journals: The Mainstream Finance Perspective. Journal of Real Estate Research, 10, 2, 217-226.

Whitman, M. E., Hendricksom, A. R. & Townsend, A. M. 1999. Research Commentary. Academic Rewards for Teaching, Research, and Service: Data and Discourse. Information Systems Research, 10, 2, 99-109.

Young, S. T., Baird, B. C., and Pullman, M. E. 1996. POM Research Productivity in U.S. Business Schools. Journal of Operations Management, 14, 41-53.

Zivney, T. L. & Bertin, W. J. 1992. Publish or perish: What the competition is really doing. The Journal of Finance 47, 295-329.

| |Previous Rankings |Citation Analyses |

|Accounting | | |

| |1 |2 |

| |1 |2 |

| |1 |2 |

| |1 |2 |

| |1 |2 |

| |1 |2 |

| |1 |2 |

|1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |ISI, 1995 |ISI, 1996 |ISI, 1997 |Starbuck, 1997 | | | | | | | | | | | | |Impact |Rank |Impact |Rank |Impact |Rank |Impact |Rank | |MS | |top |1 |1 |1 |top |1 |2 |1 |top |.705 |1 |.890 |1 |.830 |1 |.840 |1 | |JOM |top |top |2 |2 |2 |top |2 | |2 |top | | | | | | | | | |Int J Prod Res | | | | | |top | |1 | |top | | | | | | | | | |

1. Stahl, et al., 1988 (previous rankings)

2. Meredith, et al., 1989 (author opinions)

3. Barman, et al. 1991 (faculty survey)

4. van Fleet, 1995 (faculty survey)

5. Vokurka, 1996 (citation analysis)

6. Malhotra & Kher, 1996 (previous rankings)

7. Young, et al., 1996 (previous research)

8. Goh, et al., 1997 (citation analysis)

9. Glick, et al., 1997 (faculty survey)

10. Scudder & Hill, 1998 (author opinions)

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download