REVIEW OF JOURNAL CITATION INDICATORS .uk
Evaluating Journal Quality: A Review of Journal Citation Indicators and Ranking in Business and ManagementJohn Mingers (Corresponding author)Kent Business School, University of Kent, Canterbury CT7 2PE, UKj.mingers@kent.ac.uk01227 824008Liying YangNational Science Library, Chinese Academy of Sciences, 33 Beisihuan Xilu, Beijing 100190, Chinayangly@mail.las.Abstract: Evaluating the quality of academic journal is becoming increasing important within the context of research performance evaluation. Traditionally, journals have been ranked by peer review lists such as that of the Association of Business Schools (UK) or though their journal impact factor (JIF). However, several new indicators have been developed, such as the h-index, SJR, SNIP and the Eigenfactor which take into account different factors and therefore have their own particular biases. In this paper we evaluate these metrics both theoretically and also through an empirical study of a large set of business and management journals. We show that even though the indicators appear highly correlated in fact they lead to large differences in journal rankings. We contextualise our results in terms of the UK’s large scale research assessment exercise (the RAE/REF) and particularly the ABS journal ranking list. We conclude that no one indicator is superior but that the h-index (which includes the productivity of a journal) and SNIP (which aims to normalise for field effects) may be the most effective at the moment.Keywords: OR in scientometrics, ABS journal list, Eigenfactor, h-index, impact factor, journal indicators, journal ranking, normalisation, REF, SJR, SNIPAcknowledgementWe would like to thank all the reviewers for their very helpful comments.Accepted by European Journal of Operational Research, evaluation of research performance, whether at the level of individuals, departments, research groups or whole universities, is becoming ever more important and the results of exercises such as the UK’s Research Excellence Framework (REF) have major consequences in terms of funding and individual academics’ careers. The primary driver of an evaluation is an assessment of the quality of an individual research output, generally a journal paper. The evaluation can be done by peer review, as in the REF, or citations can be used as a proxy for quality – although they are really indicators of impact. The focus on quality of research has led to a focus on the quality of the publishing journal itself. There are several reasons for this: helping researchers decide where to target their papers; competition between the journals; and in many cases illicitly using the quality of the journal as a proxy for the quality of the papers published in it. Journal quality, in turn, can also be evaluated either by peer review or by citation indicators such as the journal impact factor (JIF). Peer review has been the primary form in the past for journal ranking lists such as that of the Association of Business Schools (ABS) ADDIN EN.CITE <EndNote><Cite><Author>Association of Business Schools</Author><Year>2010</Year><RecNum>2383</RecNum><DisplayText>(Association of Business Schools, 2010)</DisplayText><record><rec-number>2383</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2383</key></foreign-keys><ref-type name="Web Page">12</ref-type><contributors><authors><author>Association of Business Schools,</author></authors></contributors><titles><title>Academic journal quality guide</title></titles><dates><year>2010</year><pub-dates><date>1/3/2010</date></pub-dates></dates><publisher>Association of Business Schools</publisher><urls><related-urls><url>;(Association of Business Schools, 2010). Many of these lists for business and management are available from the Harzing website ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2009</Year><RecNum>1694</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>1694</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">1694</key></foreign-keys><ref-type name="Web Page">12</ref-type><contributors><authors><author>Harzing, A-W</author></authors></contributors><titles><title>Journal Quality List</title></titles><volume>2015</volume><number>January</number><edition>55th</edition><dates><year>2009</year></dates><publisher>Anne-Wil Harzing</publisher><urls><related-urls><url>;(2009). Some of these lists, such as ABS, are a hybrid in that they use citation indicators to inform the peer review.The practice of judging a paper by the journal in which it is published has become endemic within large scale evaluations, such as the UK’s REF, where huge numbers of papers need to be graded but, as we shall see, this is not a practice to be recommended. Within business and management, in preparation for the 2014 REF, the ABS Guide was used by Schools to choose both papers and individual academics to be submitted, despite extensive criticism of the Guide from UK academics PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5NaW5nZXJzPC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48
UmVjTnVtPjM4MDI8L1JlY051bT48RGlzcGxheVRleHQ+KEh1c3NhaW4sIDIwMTM7IE1pbmdlcnMg
JmFtcDsgV2lsbG1vdHQsIDIwMTM7IFdhbGtlciBldCBhbC4sIDIwMTU7IFdpbGxtb3R0LCAyMDEx
KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4zODAyPC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2Uy
cjJ3MmFkZmVwIj4zODAyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu
YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5N
aW5nZXJzLCBKLjwvYXV0aG9yPjxhdXRob3I+V2lsbG1vdHQsIEguPC9hdXRob3I+PC9hdXRob3Jz
PjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRheWxvcml6aW5nIGJ1c2luZXNzIHNjaG9v
bCByZXNlYXJjaDogT24gdGhlICZxdW90O29uZSBiZXN0IHdheSZxdW90OyBwZXJmb3JtYXRpdmUg
ZWZmZWN0cyBvZiBqb3VybmFsIHJhbmtpbmcgbGlzdHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+
SHVtYW4gUmVsYXRpb25zPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjEwNTEtMTA3
MzwvcGFnZXM+PHZvbHVtZT42Njwvdm9sdW1lPjxudW1iZXI+ODwvbnVtYmVyPjxkYXRlcz48eWVh
cj4yMDEzPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPldhbGtlcjwvQXV0aG9yPjxZZWFyPjIwMTU8L1llYXI+PFJlY051bT40ODI5PC9SZWNO
dW0+PHJlY29yZD48cmVjLW51bWJlcj40ODI5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40
ODI5PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IlJlcG9ydCI+Mjc8L3JlZi10
eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XYWxrZXIsIEouPC9hdXRob3I+PGF1
dGhvcj5TYWx0ZXIsIEEuPC9hdXRob3I+PGF1dGhvcj5TYWxhbmRyYSwgUi48L2F1dGhvcj48L2F1
dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SW5pdGlhbCBGaW5kaW5ncyBmcm9t
IHRoZSBTdXJ2ZXkgb2YgVUsgQnVzaW5lc3MgQWNhZGVtaWNzPC90aXRsZT48L3RpdGxlcz48ZGF0
ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+SGVubGV5IEJ1c2luZXNzIFNj
aG9vbDwvcHVibGlzaGVyPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vaGx5LmFjL0J1
c2luZXNzLUFjYWRlbWljcyA8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+SHVzc2FpbjwvQXV0aG9yPjxZZWFyPjIwMTM8L1llYXI+PFJlY051
bT40MjMyPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40MjMyPC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2Uy
cjJ3MmFkZmVwIj40MjMyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu
YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5I
dXNzYWluLCBTLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs
ZT5Kb3VybmFsIEZldGlzaGlzbSBhbmQgdGhlIOKAmFNpZ24gb2YgdGhlIDTigJkgaW4gdGhlIEFC
UyBndWlkZTogQSBRdWVzdGlvbiBvZiBUcnVzdD88L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+T3Jn
YW5pemF0aW9uPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHZvbHVtZT5vbmxpbmU8L3ZvbHVt
ZT48ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48ZWxlY3Ryb25p
Yy1yZXNvdXJjZS1udW0+ZG9pOiAxMC4xMTc3LzEzNTA1MDg0MTM1MDY3NjMgPC9lbGVjdHJvbmlj
LXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5XaWxsbW90dDwvQXV0
aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051bT4zNzk5PC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj4zNzk5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
cDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zNzk5PC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XaWxsbW90dCwgSHVnaDwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Kb3VybmFsIGxpc3QgZmV0aXNoaXNtIGFu
ZCB0aGUgcGVydmVyc2lvbiBvZiBzY2hvbGFyc2hpcDogcmVhY3Rpdml0eSBhbmQgdGhlIEFCUyBs
aXN0PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk9yZ2FuaXphdGlvbjwvc2Vjb25kYXJ5LXRpdGxl
PjwvdGl0bGVzPjxwYWdlcz40MjktNDQyPC9wYWdlcz48dm9sdW1lPjE4PC92b2x1bWU+PG51bWJl
cj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5KdWx5
IDEsIDIwMTE8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1
cmw+aHR0cDovL29yZy5zYWdlcHViLmNvbS9jb250ZW50LzE4LzQvNDI5LmFic3RyYWN0PC91cmw+
PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT4xMC4xMTc3LzEz
NTA1MDg0MTE0MDM1MzI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48
L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5NaW5nZXJzPC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48
UmVjTnVtPjM4MDI8L1JlY051bT48RGlzcGxheVRleHQ+KEh1c3NhaW4sIDIwMTM7IE1pbmdlcnMg
JmFtcDsgV2lsbG1vdHQsIDIwMTM7IFdhbGtlciBldCBhbC4sIDIwMTU7IFdpbGxtb3R0LCAyMDEx
KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4zODAyPC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2Uy
cjJ3MmFkZmVwIj4zODAyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu
YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5N
aW5nZXJzLCBKLjwvYXV0aG9yPjxhdXRob3I+V2lsbG1vdHQsIEguPC9hdXRob3I+PC9hdXRob3Jz
PjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRheWxvcml6aW5nIGJ1c2luZXNzIHNjaG9v
bCByZXNlYXJjaDogT24gdGhlICZxdW90O29uZSBiZXN0IHdheSZxdW90OyBwZXJmb3JtYXRpdmUg
ZWZmZWN0cyBvZiBqb3VybmFsIHJhbmtpbmcgbGlzdHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+
SHVtYW4gUmVsYXRpb25zPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjEwNTEtMTA3
MzwvcGFnZXM+PHZvbHVtZT42Njwvdm9sdW1lPjxudW1iZXI+ODwvbnVtYmVyPjxkYXRlcz48eWVh
cj4yMDEzPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48
QXV0aG9yPldhbGtlcjwvQXV0aG9yPjxZZWFyPjIwMTU8L1llYXI+PFJlY051bT40ODI5PC9SZWNO
dW0+PHJlY29yZD48cmVjLW51bWJlcj40ODI5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl
eSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40
ODI5PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IlJlcG9ydCI+Mjc8L3JlZi10
eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XYWxrZXIsIEouPC9hdXRob3I+PGF1
dGhvcj5TYWx0ZXIsIEEuPC9hdXRob3I+PGF1dGhvcj5TYWxhbmRyYSwgUi48L2F1dGhvcj48L2F1
dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SW5pdGlhbCBGaW5kaW5ncyBmcm9t
IHRoZSBTdXJ2ZXkgb2YgVUsgQnVzaW5lc3MgQWNhZGVtaWNzPC90aXRsZT48L3RpdGxlcz48ZGF0
ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+SGVubGV5IEJ1c2luZXNzIFNj
aG9vbDwvcHVibGlzaGVyPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vaGx5LmFjL0J1
c2luZXNzLUFjYWRlbWljcyA8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+SHVzc2FpbjwvQXV0aG9yPjxZZWFyPjIwMTM8L1llYXI+PFJlY051
bT40MjMyPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40MjMyPC9yZWMtbnVtYmVyPjxmb3Jl
aWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2Uy
cjJ3MmFkZmVwIj40MjMyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu
YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5I
dXNzYWluLCBTLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs
ZT5Kb3VybmFsIEZldGlzaGlzbSBhbmQgdGhlIOKAmFNpZ24gb2YgdGhlIDTigJkgaW4gdGhlIEFC
UyBndWlkZTogQSBRdWVzdGlvbiBvZiBUcnVzdD88L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+T3Jn
YW5pemF0aW9uPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHZvbHVtZT5vbmxpbmU8L3ZvbHVt
ZT48ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48ZWxlY3Ryb25p
Yy1yZXNvdXJjZS1udW0+ZG9pOiAxMC4xMTc3LzEzNTA1MDg0MTM1MDY3NjMgPC9lbGVjdHJvbmlj
LXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5XaWxsbW90dDwvQXV0
aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051bT4zNzk5PC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj4zNzk5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
cDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zNzk5PC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XaWxsbW90dCwgSHVnaDwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Kb3VybmFsIGxpc3QgZmV0aXNoaXNtIGFu
ZCB0aGUgcGVydmVyc2lvbiBvZiBzY2hvbGFyc2hpcDogcmVhY3Rpdml0eSBhbmQgdGhlIEFCUyBs
aXN0PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk9yZ2FuaXphdGlvbjwvc2Vjb25kYXJ5LXRpdGxl
PjwvdGl0bGVzPjxwYWdlcz40MjktNDQyPC9wYWdlcz48dm9sdW1lPjE4PC92b2x1bWU+PG51bWJl
cj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PHB1Yi1kYXRlcz48ZGF0ZT5KdWx5
IDEsIDIwMTE8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1
cmw+aHR0cDovL29yZy5zYWdlcHViLmNvbS9jb250ZW50LzE4LzQvNDI5LmFic3RyYWN0PC91cmw+
PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT4xMC4xMTc3LzEz
NTA1MDg0MTE0MDM1MzI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48
L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Hussain, 2013; Mingers & Willmott, 2013; Walker et al., 2015; Willmott, 2011). It should be noted that the Business and Management REF Panel has repeatedly stated that they do not use and journal lists, and they have informally issued some data justifying this position, but this has not stopped the wholesale use of lists within business schoolsThis paper will discuss the results primarily within the UK context, but these large scale research evaluations also occur in Australia ADDIN EN.CITE <EndNote><Cite><Author>Northcott</Author><Year>2010</Year><RecNum>4240</RecNum><DisplayText>(Northcott & Linacre, 2010)</DisplayText><record><rec-number>4240</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4240</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Northcott, D.</author><author>Linacre, S.</author></authors></contributors><titles><title>Producing spaces for academic discourse:The impact of research assessment exercises and journal quality rankings</title><secondary-title>Australian Accounting Review</secondary-title></titles><pages>38-54</pages><volume>20</volume><number>1</number><dates><year>2010</year></dates><urls></urls></record></Cite></EndNote>(Northcott & Linacre, 2010), New Zealand ADDIN EN.CITE <EndNote><Cite><Author>Hicks</Author><Year>2012</Year><RecNum>4906</RecNum><DisplayText>(Hicks, 2012)</DisplayText><record><rec-number>4906</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4906</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hicks, Diana</author></authors></contributors><titles><title>Performance-based university research funding systems</title><secondary-title>Research policy</secondary-title></titles><pages>251-261</pages><volume>41</volume><number>2</number><dates><year>2012</year></dates><isbn>0048-7333</isbn><urls></urls></record></Cite></EndNote>(Hicks, 2012) and Italy ADDIN EN.CITE <EndNote><Cite><Author>Rebora</Author><Year>2013</Year><RecNum>4905</RecNum><DisplayText>(Rebora & Turri, 2013)</DisplayText><record><rec-number>4905</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4905</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Rebora, Gianfranco</author><author>Turri, Matteo</author></authors></contributors><titles><title>The UK and Italian research assessment exercises face to face</title><secondary-title>Research policy</secondary-title></titles><pages>1657-1666</pages><volume>42</volume><number>9</number><dates><year>2013</year></dates><isbn>0048-7333</isbn><urls></urls></record></Cite></EndNote>(Rebora & Turri, 2013). These developments increase the importance of journal quality indicators, whether used in combination with peer review (as in the ABS list) or used instead of peer review. It is vital that the indicators available are accurate, robust, transparent and unbiased so that users, especially non-bibliometricians, can use then confidently ADDIN EN.CITE <EndNote><Cite><Author>Wouters</Author><Year>2015</Year><RecNum>4827</RecNum><DisplayText>(Wouters et al., 2015)</DisplayText><record><rec-number>4827</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4827</key></foreign-keys><ref-type name="Report">27</ref-type><contributors><authors><author>Wouters, Paul</author><author>Thelwall, Mike</author><author>Kousha, Kayvan</author><author>Waltman, Ludo</author><author>de Rijcke, Sarah</author><author>Rushforth, Alex</author><author>Franssen, Thomas</author></authors></contributors><titles><title>The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management</title></titles><dates><year>2015</year></dates><pub-location>London</pub-location><publisher>HEFCE</publisher><urls></urls><electronic-resource-num>10.13140/RG.2.1.4929.1363</electronic-resource-num></record></Cite></EndNote>(Wouters et al., 2015). For many years the journal impact factor (JIF) was the predominant journal metric despite considerable criticism, but recently there has been a spate of new ones including the Eigenfactor, the h-index, SJR and SNIP and it is important to understand how these differ from each other, and the degree of their validityPEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Nb2VkPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVj
TnVtPjQ4NDU8L1JlY051bT48RGlzcGxheVRleHQ+KE1vZWQsIDIwMTU7IFN0cmF1YiAmYW1wOyBB
bmRlcnNvbiwgMjAxMCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+NDg0NTwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDg0NTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJFbGVjdHJvbmljIEFydGljbGUiPjQzPC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+TW9lZCwgSC48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+Q29tcHJlaGVuc2l2ZSBpbmRpY2F0b3IgY29tcGFyaXNvbnMgaW50ZWxs
aWdpYmxlIHRvIG5vbi1leHBlcnRzOiBUaGUgY2FzZSBvZiB0d28gU05JUCB2ZXJzaW9uczwvdGl0
bGU+PC90aXRsZXM+PG51bS12b2xzPmFyeGl2Lm9yZy9wZGYvMTUxMC4wNTEyODwvbnVtLXZvbHM+
PGRhdGVzPjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPmFyeGl2Lm9yZzwvcHVi
bGlzaGVyPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwczovL2FyeGl2Lm9yZy88L3VybD48
L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+U3RyYXVi
PC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVjTnVtPjQ4NTA8L1JlY051bT48cmVjb3JkPjxy
ZWMtbnVtYmVyPjQ4NTA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRi
LWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjQ4NTA8L2tleT48L2Zv
cmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iTmV3c3BhcGVyIEFydGljbGUiPjIzPC9yZWYtdHlw
ZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+U3RyYXViLCBEZXRtYXI8L2F1dGhvcj48
YXV0aG9yPkFuZGVyc29uLCBDaGFkPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPkpvdXJuYWwgUXVhbGl0eSBhbmQgQ2l0YXRpb25zOiBDb21tb24gTWV0cmlj
cyBhbmQgQ29uc2lkZXJhdGlvbnMgYWJvdXQgVGhlaXIgVXNlPC90aXRsZT48c2Vjb25kYXJ5LXRp
dGxlPk1JUyBRdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48YWx0LXRpdGxlPk1JUyBRdWFydGVy
bHk8L2FsdC10aXRsZT48L3RpdGxlcz48cGFnZXM+aWlpLXhpaTwvcGFnZXM+PHZvbHVtZT4zNDwv
dm9sdW1lPjxrZXl3b3Jkcz48a2V5d29yZD5QRUVSIHJldmlldyAoUHJvZmVzc2lvbmFsIHBlcmZv
cm1hbmNlKTwva2V5d29yZD48a2V5d29yZD5TQ0hPTEFSTFkgcGVyaW9kaWNhbHMgLS0gRXZhbHVh
dGlvbjwva2V5d29yZD48a2V5d29yZD5TQ0hPTEFSTFkgcHVibGlzaGluZyAtLSBFdmFsdWF0aW9u
PC9rZXl3b3JkPjxrZXl3b3JkPkJJQkxJT0dSQVBISUNBTCBjaXRhdGlvbnM8L2tleXdvcmQ+PGtl
eXdvcmQ+VEVDSE5JQ0FMIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+QU5OT1RBVElPTlMgJmFt
cDsgY2l0YXRpb25zIChMYXcpPC9rZXl3b3JkPjxrZXl3b3JkPkFDQURFTUlDIGRpc2NvdXJzZTwv
a2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDEwPC95ZWFyPjwvZGF0ZXM+PHB1Ymxp
c2hlcj5NSVMgUXVhcnRlcmx5PC9wdWJsaXNoZXI+PGlzYm4+MDI3Njc3ODM8L2lzYm4+PHdvcmst
dHlwZT5FZGl0b3JpYWw8L3dvcmstdHlwZT48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDov
L3NlYXJjaC5lYnNjb2hvc3QuY29tL2xvZ2luLmFzcHg/ZGlyZWN0PXRydWUmYW1wO2RiPWJ0aCZh
bXA7QU49NDg0NzY1MzkmYW1wO3NpdGU9ZWhvc3QtbGl2ZTwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48cmVtb3RlLWRhdGFiYXNlLW5hbWU+YnRoPC9yZW1vdGUtZGF0YWJhc2UtbmFtZT48cmVt
b3RlLWRhdGFiYXNlLXByb3ZpZGVyPkVCU0NPaG9zdDwvcmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVy
PjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Nb2VkPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVj
TnVtPjQ4NDU8L1JlY051bT48RGlzcGxheVRleHQ+KE1vZWQsIDIwMTU7IFN0cmF1YiAmYW1wOyBB
bmRlcnNvbiwgMjAxMCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+NDg0NTwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDg0NTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJFbGVjdHJvbmljIEFydGljbGUiPjQzPC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+TW9lZCwgSC48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+Q29tcHJlaGVuc2l2ZSBpbmRpY2F0b3IgY29tcGFyaXNvbnMgaW50ZWxs
aWdpYmxlIHRvIG5vbi1leHBlcnRzOiBUaGUgY2FzZSBvZiB0d28gU05JUCB2ZXJzaW9uczwvdGl0
bGU+PC90aXRsZXM+PG51bS12b2xzPmFyeGl2Lm9yZy9wZGYvMTUxMC4wNTEyODwvbnVtLXZvbHM+
PGRhdGVzPjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPmFyeGl2Lm9yZzwvcHVi
bGlzaGVyPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwczovL2FyeGl2Lm9yZy88L3VybD48
L3JlbGF0ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+U3RyYXVi
PC9BdXRob3I+PFllYXI+MjAxMDwvWWVhcj48UmVjTnVtPjQ4NTA8L1JlY051bT48cmVjb3JkPjxy
ZWMtbnVtYmVyPjQ4NTA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRi
LWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjQ4NTA8L2tleT48L2Zv
cmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iTmV3c3BhcGVyIEFydGljbGUiPjIzPC9yZWYtdHlw
ZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+U3RyYXViLCBEZXRtYXI8L2F1dGhvcj48
YXV0aG9yPkFuZGVyc29uLCBDaGFkPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPkpvdXJuYWwgUXVhbGl0eSBhbmQgQ2l0YXRpb25zOiBDb21tb24gTWV0cmlj
cyBhbmQgQ29uc2lkZXJhdGlvbnMgYWJvdXQgVGhlaXIgVXNlPC90aXRsZT48c2Vjb25kYXJ5LXRp
dGxlPk1JUyBRdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48YWx0LXRpdGxlPk1JUyBRdWFydGVy
bHk8L2FsdC10aXRsZT48L3RpdGxlcz48cGFnZXM+aWlpLXhpaTwvcGFnZXM+PHZvbHVtZT4zNDwv
dm9sdW1lPjxrZXl3b3Jkcz48a2V5d29yZD5QRUVSIHJldmlldyAoUHJvZmVzc2lvbmFsIHBlcmZv
cm1hbmNlKTwva2V5d29yZD48a2V5d29yZD5TQ0hPTEFSTFkgcGVyaW9kaWNhbHMgLS0gRXZhbHVh
dGlvbjwva2V5d29yZD48a2V5d29yZD5TQ0hPTEFSTFkgcHVibGlzaGluZyAtLSBFdmFsdWF0aW9u
PC9rZXl3b3JkPjxrZXl3b3JkPkJJQkxJT0dSQVBISUNBTCBjaXRhdGlvbnM8L2tleXdvcmQ+PGtl
eXdvcmQ+VEVDSE5JQ0FMIHdyaXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+QU5OT1RBVElPTlMgJmFt
cDsgY2l0YXRpb25zIChMYXcpPC9rZXl3b3JkPjxrZXl3b3JkPkFDQURFTUlDIGRpc2NvdXJzZTwv
a2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDEwPC95ZWFyPjwvZGF0ZXM+PHB1Ymxp
c2hlcj5NSVMgUXVhcnRlcmx5PC9wdWJsaXNoZXI+PGlzYm4+MDI3Njc3ODM8L2lzYm4+PHdvcmst
dHlwZT5FZGl0b3JpYWw8L3dvcmstdHlwZT48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDov
L3NlYXJjaC5lYnNjb2hvc3QuY29tL2xvZ2luLmFzcHg/ZGlyZWN0PXRydWUmYW1wO2RiPWJ0aCZh
bXA7QU49NDg0NzY1MzkmYW1wO3NpdGU9ZWhvc3QtbGl2ZTwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48cmVtb3RlLWRhdGFiYXNlLW5hbWU+YnRoPC9yZW1vdGUtZGF0YWJhc2UtbmFtZT48cmVt
b3RlLWRhdGFiYXNlLXByb3ZpZGVyPkVCU0NPaG9zdDwvcmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVy
PjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+AG==
ADDIN EN.CITE.DATA (Moed, 2015; Straub & Anderson, 2010).The purpose of this paper is to evaluate the indicators that are currently available in terms of these four criteria – accuracy, robustness, transparency and unbiasedness. It is evident that any metric has its own particular biases; that is, it will tend to favour certain kinds at the expense of others – that is after all the point of measuring something. Some of these biases will be explicit and indeed designed-in. Others will be implicit, perhaps not recognised, and may be undesirable. The review will analyse the theory of the different indicators, looking for their explicit and implicit biases, and then test these observations on a sample of journals from the business and management area. There are four sections which cover: a review of the different indicators; methodology and data collection; empirical results and comparisons with the ABS list; and recommendations.REVIEW OF JOURNAL CITATION INDICATORSThe use of citations to track the performance of journals was initiated by Garfield in 1955 ADDIN EN.CITE <EndNote><Cite><Author>Garfield</Author><Year>1955</Year><RecNum>4495</RecNum><DisplayText>(Garfield, 1955)</DisplayText><record><rec-number>4495</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4495</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Garfield, E. </author></authors></contributors><titles><title>Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas</title><secondary-title>Science</secondary-title></titles><pages>108-111</pages><volume>122</volume><number>3159</number><dates><year>1955</year></dates><urls></urls></record></Cite></EndNote>(Garfield, 1955) and he established the first citation indexes (Science Citation Index) and the company, the Institute for Scientific Information (ISI). Although, prior to that, the first analysis of papers citing a journal’s publications occurred in 1927 ADDIN EN.CITE <EndNote><Cite><Author>Gross</Author><Year>1927</Year><RecNum>4836</RecNum><DisplayText>(Gross & Gross, 1927)</DisplayText><record><rec-number>4836</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4836</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gross, P.</author><author>Gross, E. </author></authors></contributors><titles><title>College libraries and chemical education</title><secondary-title>Science </secondary-title></titles><volume>66</volume><number>385-389</number><dates><year>1927</year></dates><urls></urls></record></Cite></EndNote>(Gross & Gross, 1927) and Shepard’s Citations is a legal citing service started in 1873.Mingers and Leydesdorff ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2015</Year><RecNum>4717</RecNum><DisplayText>(2015b)</DisplayText><record><rec-number>4717</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4717</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Leydesdorff, L.</author></authors></contributors><titles><title>A review of theory and practice in scientometrics</title><secondary-title>European Journal of Operational Research</secondary-title></titles><periodical><full-title>European Journal of Operational Research</full-title></periodical><pages>1-19</pages><volume>246</volume><number>1</number><dates><year>2015</year></dates><urls></urls><electronic-resource-num>10.1016/j.ejor.2015.04.002</electronic-resource-num></record></Cite></EndNote>(2015b) and Cronin and Sugimoto ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2014</Year><RecNum>4394</RecNum><DisplayText>(2014)</DisplayText><record><rec-number>4394</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4394</key></foreign-keys><ref-type name="Edited Book">28</ref-type><contributors><authors><author>Cronin, B.</author><author>Sugimoto, C.</author></authors></contributors><titles><title>Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact</title></titles><dates><year>2014</year></dates><pub-location>London</pub-location><publisher>MIT Press</publisher><urls></urls></record></Cite></EndNote>(2014) provide reviews of the current field of scientometrics and bibliometrics as a whole. Waltman ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2015</Year><RecNum>4838</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>4838</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4838</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Waltman, Ludo</author></authors></contributors><titles><title>A review of the literature on citation impact indicators</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>365-391</pages><volume>10</volume><number>2</number><dates><year>2015</year></dates><urls></urls></record></Cite></EndNote>(2015) is a review of the literature on citation impact indicators including their use for journals although it does not include SNIP. There have also been several reviews of the use of indicators in research evaluation generally, some favourable ADDIN EN.CITE <EndNote><Cite><Author>van Raan</Author><Year>2005</Year><RecNum>3705</RecNum><DisplayText>(Abramo & D’Angelo, 2011; Moed, 2007; van Raan, 2005b)</DisplayText><record><rec-number>3705</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3705</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>van Raan, A.</author></authors></contributors><titles><title>Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods</title><secondary-title>Scientometrics</secondary-title></titles><pages>133-143</pages><volume>62</volume><number>1</number><dates><year>2005</year></dates><urls></urls></record></Cite><Cite><Author>Moed</Author><Year>2007</Year><RecNum>4392</RecNum><record><rec-number>4392</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4392</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Moed, H.</author></authors></contributors><titles><title>The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review</title><secondary-title>Science and Public Policy</secondary-title></titles><pages>575-583</pages><volume>34</volume><number>8</number><dates><year>2007</year></dates><urls></urls></record></Cite><Cite><Author>Abramo</Author><Year>2011</Year><RecNum>3708</RecNum><record><rec-number>3708</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3708</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Abramo, G.</author><author>D’Angelo, C. </author></authors></contributors><titles><title>Evaluating research: from informed peer review to bibliometric</title><secondary-title>Scientometrics</secondary-title></titles><pages>499-514</pages><volume>87</volume><number>3</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(Abramo & D’Angelo, 2011; Moed, 2007; van Raan, 2005b) and some less so PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5FdmlkZW5jZSBMdGQ8L0F1dGhvcj48WWVhcj4yMDA3PC9Z
ZWFyPjxSZWNOdW0+MjY3MjwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oRXZpZGVuY2UgTHRkLCAyMDA3
OyBMZXlkZXNkb3JmZiAmYW1wOyBNaWxvamV2acSHLCAyMDE1OyBXb3V0ZXJzIGV0IGFsLiwgMjAx
NSk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjY3MjwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dl
MnIydzJhZGZlcCI+MjY3Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJSZXBv
cnQiPjI3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+RXZpZGVuY2Ug
THRkLDwvYXV0aG9yPjwvYXV0aG9ycz48dGVydGlhcnktYXV0aG9ycz48YXV0aG9yPlVuaXZlcnNp
dGllcyBVSzwvYXV0aG9yPjwvdGVydGlhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5UaGUgdXNlIG9mIGJpYmxpb21ldHJpY3MgdG8gbWVhc3VyZSByZXNlYXJjaCBxdWFs
aXR5IGluIFVLIGhpZ2hlciBlZHVjYXRpb24gaW5zdGl0dXRpb25zPC90aXRsZT48L3RpdGxlcz48
ZGF0ZXM+PHllYXI+MjAwNzwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+TGVlZHM8L3B1Yi1s
b2NhdGlvbj48cHVibGlzaGVyPkV2aWRlbmNlIEx0ZDwvcHVibGlzaGVyPjx1cmxzPjxyZWxhdGVk
LXVybHM+PHVybD5odHRwOi8vd3d3LmhlZmNlLmFjLnVrL3B1YnMvcmRyZXBvcnRzLzIwMDcvcmQx
OSU1RjA3LzwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+
PEF1dGhvcj5Xb3V0ZXJzPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjQ4Mjc8L1Jl
Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ4Mjc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48
a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAi
PjQ4Mjc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iUmVwb3J0Ij4yNzwvcmVm
LXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldvdXRlcnMsIFBhdWw8L2F1dGhv
cj48YXV0aG9yPlRoZWx3YWxsLCBNaWtlPC9hdXRob3I+PGF1dGhvcj5Lb3VzaGEsIEtheXZhbjwv
YXV0aG9yPjxhdXRob3I+V2FsdG1hbiwgTHVkbzwvYXV0aG9yPjxhdXRob3I+ZGUgUmlqY2tlLCBT
YXJhaDwvYXV0aG9yPjxhdXRob3I+UnVzaGZvcnRoLCBBbGV4PC9hdXRob3I+PGF1dGhvcj5GcmFu
c3NlbiwgVGhvbWFzPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp
dGxlPlRoZSBNZXRyaWMgVGlkZTogUmVwb3J0IG9mIHRoZSBJbmRlcGVuZGVudCBSZXZpZXcgb2Yg
dGhlIFJvbGUgb2YgTWV0cmljcyBpbiBSZXNlYXJjaCBBc3Nlc3NtZW50IGFuZCBNYW5hZ2VtZW50
PC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+TG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5IRUZDRTwvcHVibGlzaGVyPjx1
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTMxNDAvUkcuMi4xLjQ5Mjku
MTM2MzwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+TGV5ZGVzZG9yZmY8L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+NTEwMDwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NTEwMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxr
ZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+
NTEwMDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUi
PjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+TGV5ZGVzZG9yZmYs
IExvZXQ8L2F1dGhvcj48YXV0aG9yPk1pbG9qZXZpxIcsIFN0YcWhYTwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UaGUgQ2l0YXRpb24gSW1wYWN0IG9mIEdl
cm1hbiBTb2Npb2xvZ3kgSm91cm5hbHM6IFNvbWUgUHJvYmxlbXMgd2l0aCB0aGUgVXNlIG9mIFNj
aWVudG9tZXRyaWMgSW5kaWNhdG9ycyBpbiBKb3VybmFsIGFuZCBSZXNlYXJjaCBFdmFsdWF0aW9u
czwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5Tb3ppYWxlIFdlbHQsIDY2KDIpLCAxOTMtMjA0LiA8
L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MTkzLTIwNDwvcGFnZXM+PHZvbHVtZT42
Njwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0
ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5FdmlkZW5jZSBMdGQ8L0F1dGhvcj48WWVhcj4yMDA3PC9Z
ZWFyPjxSZWNOdW0+MjY3MjwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oRXZpZGVuY2UgTHRkLCAyMDA3
OyBMZXlkZXNkb3JmZiAmYW1wOyBNaWxvamV2acSHLCAyMDE1OyBXb3V0ZXJzIGV0IGFsLiwgMjAx
NSk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjY3MjwvcmVjLW51bWJlcj48Zm9y
ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dl
MnIydzJhZGZlcCI+MjY3Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJSZXBv
cnQiPjI3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+RXZpZGVuY2Ug
THRkLDwvYXV0aG9yPjwvYXV0aG9ycz48dGVydGlhcnktYXV0aG9ycz48YXV0aG9yPlVuaXZlcnNp
dGllcyBVSzwvYXV0aG9yPjwvdGVydGlhcnktYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVz
Pjx0aXRsZT5UaGUgdXNlIG9mIGJpYmxpb21ldHJpY3MgdG8gbWVhc3VyZSByZXNlYXJjaCBxdWFs
aXR5IGluIFVLIGhpZ2hlciBlZHVjYXRpb24gaW5zdGl0dXRpb25zPC90aXRsZT48L3RpdGxlcz48
ZGF0ZXM+PHllYXI+MjAwNzwveWVhcj48L2RhdGVzPjxwdWItbG9jYXRpb24+TGVlZHM8L3B1Yi1s
b2NhdGlvbj48cHVibGlzaGVyPkV2aWRlbmNlIEx0ZDwvcHVibGlzaGVyPjx1cmxzPjxyZWxhdGVk
LXVybHM+PHVybD5odHRwOi8vd3d3LmhlZmNlLmFjLnVrL3B1YnMvcmRyZXBvcnRzLzIwMDcvcmQx
OSU1RjA3LzwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+
PEF1dGhvcj5Xb3V0ZXJzPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjQ4Mjc8L1Jl
Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ4Mjc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48
a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAi
PjQ4Mjc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iUmVwb3J0Ij4yNzwvcmVm
LXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldvdXRlcnMsIFBhdWw8L2F1dGhv
cj48YXV0aG9yPlRoZWx3YWxsLCBNaWtlPC9hdXRob3I+PGF1dGhvcj5Lb3VzaGEsIEtheXZhbjwv
YXV0aG9yPjxhdXRob3I+V2FsdG1hbiwgTHVkbzwvYXV0aG9yPjxhdXRob3I+ZGUgUmlqY2tlLCBT
YXJhaDwvYXV0aG9yPjxhdXRob3I+UnVzaGZvcnRoLCBBbGV4PC9hdXRob3I+PGF1dGhvcj5GcmFu
c3NlbiwgVGhvbWFzPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp
dGxlPlRoZSBNZXRyaWMgVGlkZTogUmVwb3J0IG9mIHRoZSBJbmRlcGVuZGVudCBSZXZpZXcgb2Yg
dGhlIFJvbGUgb2YgTWV0cmljcyBpbiBSZXNlYXJjaCBBc3Nlc3NtZW50IGFuZCBNYW5hZ2VtZW50
PC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxwdWItbG9j
YXRpb24+TG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5IRUZDRTwvcHVibGlzaGVyPjx1
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTMxNDAvUkcuMi4xLjQ5Mjku
MTM2MzwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+TGV5ZGVzZG9yZmY8L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+NTEwMDwvUmVj
TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NTEwMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxr
ZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+
NTEwMDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUi
PjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+TGV5ZGVzZG9yZmYs
IExvZXQ8L2F1dGhvcj48YXV0aG9yPk1pbG9qZXZpxIcsIFN0YcWhYTwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UaGUgQ2l0YXRpb24gSW1wYWN0IG9mIEdl
cm1hbiBTb2Npb2xvZ3kgSm91cm5hbHM6IFNvbWUgUHJvYmxlbXMgd2l0aCB0aGUgVXNlIG9mIFNj
aWVudG9tZXRyaWMgSW5kaWNhdG9ycyBpbiBKb3VybmFsIGFuZCBSZXNlYXJjaCBFdmFsdWF0aW9u
czwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5Tb3ppYWxlIFdlbHQsIDY2KDIpLCAxOTMtMjA0LiA8
L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MTkzLTIwNDwvcGFnZXM+PHZvbHVtZT42
Njwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0
ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Evidence Ltd, 2007; Leydesdorff & Milojevi?, 2015; Wouters et al., 2015).Sources of citation dataBefore discussing the indicators it is important to examine the limitations of the sources of citation data. Traditionally, the major source of citation data was the Thomson Reuters ISI Web of Science (WoS) which is a specialised database covering all the papers in around 12,000 journals. It also covers some conference proceedings and is beginning to cover books. In 2004 a similar database was started by Elsevier called Scopusthat covers 20,000 journals and also conferences and books. These databases capture the information directly from the journals that they cover and are generally reasonably accurate. A third source, since 2004, is Google Scholar (GS) based on the Google search engine. This works in an entirely different way by searching the Web to find relevant citations from whatever sources it can. Giles and Khabsa ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2014</Year><RecNum>4839</RecNum><DisplayText>(2014)</DisplayText><record><rec-number>4839</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4839</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Giles, L</author><author>Khabsa, M</author></authors></contributors><titles><title>The Number of scholarly documents on the Web</title><secondary-title>PLoS ONE</secondary-title></titles><pages>e93949</pages><volume>9</volume><number>5</number><dates><year>2014</year></dates><urls></urls></record></Cite></EndNote>(2014) estimated that GS indexes about 100 million English-language documents.Many studies have shown that the coverage of WoS and Scopus differs significantly between different fields, particularly between the natural sciences, where coverage is very good, the social sciences where it is moderate and variable, and the arts and humanities where it is generally poor PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IRUZDRTwvQXV0aG9yPjxZZWFyPjIwMDg8L1llYXI+PFJl
Y051bT4yNjg4PC9SZWNOdW0+PERpc3BsYXlUZXh0PihIRUZDRSwgMjAwODsgTGFyaXZpw6hyZSBl
dCBhbC4sIDIwMDY7IE1haGRpIGV0IGFsLiwgMjAwODsgTW9lZCAmYW1wOyBWaXNzZXIsIDIwMDgp
PC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjI2ODg8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJy
MncyYWRmZXAiPjI2ODg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iUmVwb3J0
Ij4yNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkhFRkNFPC9hdXRo
b3I+PC9hdXRob3JzPjx0ZXJ0aWFyeS1hdXRob3JzPjxhdXRob3I+SEVGQ0U8L2F1dGhvcj48L3Rl
cnRpYXJ5LWF1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Q291bnRpbmcgd2hh
dCBpcyBtZWFzdXJlZCBvciBtZWFzdXJpbmcgd2hhdCBjb3VudHM8L3RpdGxlPjwvdGl0bGVzPjxk
YXRlcz48eWVhcj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHB1Ymxpc2hlcj5IRUZDRTwvcHVibGlzaGVy
Pjxpc2JuPjIwMDgvMTQ8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cu
aGVmY2UuYWMudWsvcHVicy9oZWZjZS8yMDA3LzA3XzM0LzwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5NYWhkaTwvQXV0aG9yPjxZZWFyPjIw
MDg8L1llYXI+PFJlY051bT4zMjEzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zMjEzPC9y
ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdy
ZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjEzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10
eXBlIG5hbWU9IlJlcG9ydCI+Mjc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5NYWhkaSwgUy48L2F1dGhvcj48YXV0aG9yPkQmYXBvcztFc3RlLCBQLjwvYXV0aG9yPjxh
dXRob3I+TmVlbHksIEEuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPkNpdGF0aW9uIGNvdW50czogQXJlIHRoZXkgZ29vZCBwcmVkaWN0b3JzIG9mIFJBRSBz
Y29yZXM/PC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MjAwODwveWVhcj48L2RhdGVzPjxw
dWItbG9jYXRpb24+TG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5BSU0gUmVzZWFyY2g8
L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+TW9l
ZDwvQXV0aG9yPjxZZWFyPjIwMDg8L1llYXI+PFJlY051bT4zMjE2PC9SZWNOdW0+PHJlY29yZD48
cmVjLW51bWJlcj4zMjE2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjE2PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IlJlcG9ydCI+Mjc8L3JlZi10eXBlPjxjb250cmli
dXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Nb2VkLCBILjwvYXV0aG9yPjxhdXRob3I+Vmlzc2VyLCBN
LjwvYXV0aG9yPjwvYXV0aG9ycz48dGVydGlhcnktYXV0aG9ycz48YXV0aG9yPkhFRkNFPC9hdXRo
b3I+PC90ZXJ0aWFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFwcHJh
aXNhbCBvZiBDaXRhdGlvbiBEYXRhIFNvdXJjZXM8L3RpdGxlPjwvdGl0bGVzPjxkYXRlcz48eWVh
cj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5MZWlkZW48L3B1Yi1sb2NhdGlvbj48
cHVibGlzaGVyPkNlbnRyZSBmb3IgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSBTdHVkaWVzLCBMZWlk
ZW4gVW5pdmVyc2l0eTwvcHVibGlzaGVyPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8v
d3d3LmhlZmNlLmFjLnVrL3B1YnMvcmRyZXBvcnRzLzIwMDgvcmQxN18wOC88L3VybD48L3JlbGF0
ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+TGFyaXZpw6hyZTwv
QXV0aG9yPjxZZWFyPjIwMDY8L1llYXI+PFJlY051bT40NTAxPC9SZWNOdW0+PHJlY29yZD48cmVj
LW51bWJlcj40NTAxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1p
ZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40NTAxPC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5MYXJpdmnDqHJlLCBWLjwvYXV0aG9yPjxhdXRo
b3I+QXJjaGFtYmF1bHQsIMOJLjwvYXV0aG9yPjxhdXRob3I+R2luZ3JhcywgWS48L2F1dGhvcj48
YXV0aG9yPlZpZ25vbGEtR2FnbsOpLCDDiS4gPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0
b3JzPjx0aXRsZXM+PHRpdGxlPlRoZSBwbGFjZSBvZiBzZXJpYWxzIGluIHJlZmVyZW5jaW5nIHBy
YWN0aWNlczogQ29tcGFyaW5nIG5hdHVyYWwgc2NpZW5jZXMgYW5kIGVuZ2luZWVyaW5nIHdpdGgg
c29jaWFsIHNjaWVuY2VzIGFuZCBodW1hbml0aWVzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpv
dXJuYWwgb2YgdGhlIEFtZXJpY2FuIFNvY2lldHkgZm9yIEluZm9ybWF0aW9uIFNjaWVuY2UgYW5k
IFRlY2hub2xvZ3k8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+OTk3LTEwMDQ8L3Bh
Z2VzPjx2b2x1bWU+NTc8L3ZvbHVtZT48bnVtYmVyPjg8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAw
NjwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5IRUZDRTwvQXV0aG9yPjxZZWFyPjIwMDg8L1llYXI+PFJl
Y051bT4yNjg4PC9SZWNOdW0+PERpc3BsYXlUZXh0PihIRUZDRSwgMjAwODsgTGFyaXZpw6hyZSBl
dCBhbC4sIDIwMDY7IE1haGRpIGV0IGFsLiwgMjAwODsgTW9lZCAmYW1wOyBWaXNzZXIsIDIwMDgp
PC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjI2ODg8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJy
MncyYWRmZXAiPjI2ODg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iUmVwb3J0
Ij4yNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkhFRkNFPC9hdXRo
b3I+PC9hdXRob3JzPjx0ZXJ0aWFyeS1hdXRob3JzPjxhdXRob3I+SEVGQ0U8L2F1dGhvcj48L3Rl
cnRpYXJ5LWF1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Q291bnRpbmcgd2hh
dCBpcyBtZWFzdXJlZCBvciBtZWFzdXJpbmcgd2hhdCBjb3VudHM8L3RpdGxlPjwvdGl0bGVzPjxk
YXRlcz48eWVhcj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHB1Ymxpc2hlcj5IRUZDRTwvcHVibGlzaGVy
Pjxpc2JuPjIwMDgvMTQ8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cu
aGVmY2UuYWMudWsvcHVicy9oZWZjZS8yMDA3LzA3XzM0LzwvdXJsPjwvcmVsYXRlZC11cmxzPjwv
dXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5NYWhkaTwvQXV0aG9yPjxZZWFyPjIw
MDg8L1llYXI+PFJlY051bT4zMjEzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zMjEzPC9y
ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdy
ZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjEzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10
eXBlIG5hbWU9IlJlcG9ydCI+Mjc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1
dGhvcj5NYWhkaSwgUy48L2F1dGhvcj48YXV0aG9yPkQmYXBvcztFc3RlLCBQLjwvYXV0aG9yPjxh
dXRob3I+TmVlbHksIEEuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPkNpdGF0aW9uIGNvdW50czogQXJlIHRoZXkgZ29vZCBwcmVkaWN0b3JzIG9mIFJBRSBz
Y29yZXM/PC90aXRsZT48L3RpdGxlcz48ZGF0ZXM+PHllYXI+MjAwODwveWVhcj48L2RhdGVzPjxw
dWItbG9jYXRpb24+TG9uZG9uPC9wdWItbG9jYXRpb24+PHB1Ymxpc2hlcj5BSU0gUmVzZWFyY2g8
L3B1Ymxpc2hlcj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+TW9l
ZDwvQXV0aG9yPjxZZWFyPjIwMDg8L1llYXI+PFJlY051bT4zMjE2PC9SZWNOdW0+PHJlY29yZD48
cmVjLW51bWJlcj4zMjE2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjE2PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IlJlcG9ydCI+Mjc8L3JlZi10eXBlPjxjb250cmli
dXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Nb2VkLCBILjwvYXV0aG9yPjxhdXRob3I+Vmlzc2VyLCBN
LjwvYXV0aG9yPjwvYXV0aG9ycz48dGVydGlhcnktYXV0aG9ycz48YXV0aG9yPkhFRkNFPC9hdXRo
b3I+PC90ZXJ0aWFyeS1hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFwcHJh
aXNhbCBvZiBDaXRhdGlvbiBEYXRhIFNvdXJjZXM8L3RpdGxlPjwvdGl0bGVzPjxkYXRlcz48eWVh
cj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHB1Yi1sb2NhdGlvbj5MZWlkZW48L3B1Yi1sb2NhdGlvbj48
cHVibGlzaGVyPkNlbnRyZSBmb3IgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSBTdHVkaWVzLCBMZWlk
ZW4gVW5pdmVyc2l0eTwvcHVibGlzaGVyPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8v
d3d3LmhlZmNlLmFjLnVrL3B1YnMvcmRyZXBvcnRzLzIwMDgvcmQxN18wOC88L3VybD48L3JlbGF0
ZWQtdXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+TGFyaXZpw6hyZTwv
QXV0aG9yPjxZZWFyPjIwMDY8L1llYXI+PFJlY051bT40NTAxPC9SZWNOdW0+PHJlY29yZD48cmVj
LW51bWJlcj40NTAxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1p
ZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40NTAxPC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5MYXJpdmnDqHJlLCBWLjwvYXV0aG9yPjxhdXRo
b3I+QXJjaGFtYmF1bHQsIMOJLjwvYXV0aG9yPjxhdXRob3I+R2luZ3JhcywgWS48L2F1dGhvcj48
YXV0aG9yPlZpZ25vbGEtR2FnbsOpLCDDiS4gPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0
b3JzPjx0aXRsZXM+PHRpdGxlPlRoZSBwbGFjZSBvZiBzZXJpYWxzIGluIHJlZmVyZW5jaW5nIHBy
YWN0aWNlczogQ29tcGFyaW5nIG5hdHVyYWwgc2NpZW5jZXMgYW5kIGVuZ2luZWVyaW5nIHdpdGgg
c29jaWFsIHNjaWVuY2VzIGFuZCBodW1hbml0aWVzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpv
dXJuYWwgb2YgdGhlIEFtZXJpY2FuIFNvY2lldHkgZm9yIEluZm9ybWF0aW9uIFNjaWVuY2UgYW5k
IFRlY2hub2xvZ3k8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+OTk3LTEwMDQ8L3Bh
Z2VzPjx2b2x1bWU+NTc8L3ZvbHVtZT48bnVtYmVyPjg8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAw
NjwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (HEFCE, 2008; Larivière et al., 2006; Mahdi et al., 2008; Moed & Visser, 2008). In contrast, the coverage of GS is generally higher, and does not differ so much between subject areas, but the reliability and quality of its data can be poor ADDIN EN.CITE <EndNote><Cite><Author>Amara</Author><Year>2012</Year><RecNum>4416</RecNum><DisplayText>(Amara & Landry, 2012)</DisplayText><record><rec-number>4416</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4416</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Amara, Nabil</author><author>Landry, Réjean</author></authors></contributors><titles><title>Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science</title><secondary-title>Scientometrics</secondary-title><alt-title>Scientometrics</alt-title></titles><pages>553-581</pages><volume>93</volume><number>3</number><keywords><keyword>Contributions record</keyword><keyword>Citations</keyword><keyword>h-index</keyword><keyword>ISI</keyword><keyword>Google Scholar</keyword><keyword>Business scholars</keyword></keywords><dates><year>2012</year><pub-dates><date>2012/12/01</date></pub-dates></dates><publisher>Springer Netherlands</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(Amara & Landry, 2012).There have also been many comparisons of WoS, Scopus and Google Scholar in different disciplines PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5GcmFuY2VzY2hldDwvQXV0aG9yPjxZZWFyPjIwMTA8L1ll
YXI+PFJlY051bT40NDE4PC9SZWNOdW0+PERpc3BsYXlUZXh0PihBZHJpYWFuc2UgJmFtcDsgUmVu
c2xlaWdoLCAyMDEzOyBBbWFyYSAmYW1wOyBMYW5kcnksIDIwMTI7IEZyYW5jZXNjaGV0LCAyMDEw
OyBHYXJjw61hLVDDqXJleiwgMjAxMDsgSGFyemluZyAmYW1wOyB2YW4gZGVyIFdhbCwgMjAwODsg
SmFjc28sIDIwMDU7IE1laG8gJmFtcDsgUm9nZXJzLCAyMDA4OyBNZWhvICZhbXA7IFlhbmcsIDIw
MDc7IE1pbmdlcnMgJmFtcDsgTGlwaXRha2lzLCAyMDEwKTwvRGlzcGxheVRleHQ+PHJlY29yZD48
cmVjLW51bWJlcj40NDE4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40NDE4PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5GcmFuY2VzY2hldCwgTWFzc2ltbzwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5BIGNvbXBhcmlzb24g
b2YgYmlibGlvbWV0cmljIGluZGljYXRvcnMgZm9yIGNvbXB1dGVyIHNjaWVuY2Ugc2Nob2xhcnMg
YW5kIGpvdXJuYWxzIG9uIFdlYiBvZiBTY2llbmNlIGFuZCBHb29nbGUgU2Nob2xhcjwvdGl0bGU+
PHNlY29uZGFyeS10aXRsZT5TY2llbnRvbWV0cmljczwvc2Vjb25kYXJ5LXRpdGxlPjxhbHQtdGl0
bGU+U2NpZW50b21ldHJpY3M8L2FsdC10aXRsZT48L3RpdGxlcz48cGFnZXM+MjQzLTI1ODwvcGFn
ZXM+PHZvbHVtZT44Mzwvdm9sdW1lPjxudW1iZXI+MTwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29y
ZD5CaWJsaW9tZXRyaWMgaW5kaWNhdG9yczwva2V5d29yZD48a2V5d29yZD5oIEluZGV4PC9rZXl3
b3JkPjxrZXl3b3JkPlB1YmxpY2F0aW9uIGFuZCBjaXRhdGlvbiBkYXRhIHNvdXJjZXM8L2tleXdv
cmQ+PGtleXdvcmQ+Q29ycmVsYXRpb24gYW5hbHlzaXM8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0
ZXM+PHllYXI+MjAxMDwveWVhcj48cHViLWRhdGVzPjxkYXRlPjIwMTAvMDQvMDE8L2RhdGU+PC9w
dWItZGF0ZXM+PC9kYXRlcz48cHVibGlzaGVyPlNwcmluZ2VyIE5ldGhlcmxhbmRzPC9wdWJsaXNo
ZXI+PGlzYm4+MDEzOC05MTMwPC9pc2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8v
ZHguZG9pLm9yZy8xMC4xMDA3L3MxMTE5Mi0wMDktMDAyMS0yPC91cmw+PC9yZWxhdGVkLXVybHM+
PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT4xMC4xMDA3L3MxMTE5Mi0wMDktMDAyMS0y
PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48bGFuZ3VhZ2U+RW5nbGlzaDwvbGFuZ3VhZ2U+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+R2FyY8OtYS1Qw6lyZXo8L0F1dGhvcj48WWVhcj4y
MDEwPC9ZZWFyPjxSZWNOdW0+NDQxNzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQxNzwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3
cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQxNzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYt
dHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+R2FyY8OtYS1Qw6lyZXosIE1pZ3VlbCBBLjwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5BY2N1cmFjeSBhbmQgY29tcGxldGVuZXNz
IG9mIHB1YmxpY2F0aW9uIGFuZCBjaXRhdGlvbiByZWNvcmRzIGluIHRoZSBXZWIgb2YgU2NpZW5j
ZSwgUHN5Y0lORk8sIGFuZCBHb29nbGUgU2Nob2xhcjogQSBjYXNlIHN0dWR5IGZvciB0aGUgY29t
cHV0YXRpb24gb2YgaCBpbmRpY2VzIGluIFBzeWNob2xvZ3k8L3RpdGxlPjxzZWNvbmRhcnktdGl0
bGU+Sm91cm5hbCBvZiB0aGUgQW1lcmljYW4gU29jaWV0eSBmb3IgSW5mb3JtYXRpb24gU2NpZW5j
ZSBhbmQgVGVjaG5vbG9neTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4yMDcwLTIw
ODU8L3BhZ2VzPjx2b2x1bWU+NjE8L3ZvbHVtZT48bnVtYmVyPjEwPC9udW1iZXI+PGRhdGVzPjx5
ZWFyPjIwMTA8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPldpbGV5IFN1YnNjcmlwdGlvbiBTZXJ2
aWNlcywgSW5jLiwgQSBXaWxleSBDb21wYW55PC9wdWJsaXNoZXI+PGlzYm4+MTUzMi0yODkwPC9p
c2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vZHguZG9pLm9yZy8xMC4xMDAyL2Fz
aS4yMTM3MjwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1u
dW0+MTAuMTAwMi9hc2kuMjEzNzI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwv
Q2l0ZT48Q2l0ZT48QXV0aG9yPkFtYXJhPC9BdXRob3I+PFllYXI+MjAxMjwvWWVhcj48UmVjTnVt
PjQ0MTY8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ0MTY8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJy
MncyYWRmZXAiPjQ0MTY8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h
bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkFt
YXJhLCBOYWJpbDwvYXV0aG9yPjxhdXRob3I+TGFuZHJ5LCBSw6lqZWFuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkNvdW50aW5nIGNpdGF0aW9ucyBpbiB0
aGUgZmllbGQgb2YgYnVzaW5lc3MgYW5kIG1hbmFnZW1lbnQ6IHdoeSB1c2UgR29vZ2xlIFNjaG9s
YXIgcmF0aGVyIHRoYW4gdGhlIFdlYiBvZiBTY2llbmNlPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxl
PlNjaWVudG9tZXRyaWNzPC9zZWNvbmRhcnktdGl0bGU+PGFsdC10aXRsZT5TY2llbnRvbWV0cmlj
czwvYWx0LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz41NTMtNTgxPC9wYWdlcz48dm9sdW1lPjkzPC92
b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPkNvbnRyaWJ1dGlvbnMg
cmVjb3JkPC9rZXl3b3JkPjxrZXl3b3JkPkNpdGF0aW9uczwva2V5d29yZD48a2V5d29yZD5oLWlu
ZGV4PC9rZXl3b3JkPjxrZXl3b3JkPklTSTwva2V5d29yZD48a2V5d29yZD5Hb29nbGUgU2Nob2xh
cjwva2V5d29yZD48a2V5d29yZD5CdXNpbmVzcyBzY2hvbGFyczwva2V5d29yZD48L2tleXdvcmRz
PjxkYXRlcz48eWVhcj4yMDEyPC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+MjAxMi8xMi8wMTwvZGF0
ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjxwdWJsaXNoZXI+U3ByaW5nZXIgTmV0aGVybGFuZHM8L3B1
Ymxpc2hlcj48aXNibj4wMTM4LTkxMzA8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0
dHA6Ly9keC5kb2kub3JnLzEwLjEwMDcvczExMTkyLTAxMi0wNzI5LTI8L3VybD48L3JlbGF0ZWQt
dXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwMDcvczExMTkyLTAxMi0w
NzI5LTI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjxsYW5ndWFnZT5FbmdsaXNoPC9sYW5ndWFn
ZT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5BZHJpYWFuc2U8L0F1dGhvcj48WWVhcj4y
MDEzPC9ZZWFyPjxSZWNOdW0+NDQxNTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQxNTwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3
cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQxNTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYt
dHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+QWRyaWFhbnNlLCBMZXNsaWU8L2F1dGhvcj48YXV0aG9yPlJlbnNsZWln
aCwgQ2hyaXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
V2ViIG9mIFNjaWVuY2UsIFNjb3B1cyBhbmQgR29vZ2xlIFNjaG9sYXI8L3RpdGxlPjxzZWNvbmRh
cnktdGl0bGU+VGhlIEVsZWN0cm9uaWMgTGlicmFyeTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVz
PjxwYWdlcz43MjctNzQ0PC9wYWdlcz48dm9sdW1lPjMxPC92b2x1bWU+PG51bWJlcj42PC9udW1i
ZXI+PGRhdGVzPjx5ZWFyPjIwMTM8L3llYXI+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1
cmw+aHR0cDovL3d3dy5lbWVyYWxkaW5zaWdodC5jb20vZG9pL2Ficy8xMC4xMTA4L0VMLTEyLTIw
MTEtMDE3NDwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1u
dW0+ZG9pOjEwLjExMDgvRUwtMTItMjAxMS0wMTc0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48
L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5NZWhvPC9BdXRob3I+PFllYXI+MjAwODwvWWVh
cj48UmVjTnVtPjMyMjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjMyMjE8L3JlYy1udW1i
ZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNy
cGF2ZWN3ZTJyMncyYWRmZXAiPjMyMjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFt
ZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48
YXV0aG9yPk1laG8sIEwuPC9hdXRob3I+PGF1dGhvcj5Sb2dlcnMsIFkuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkNpdGF0aW9uIGNvdW50aW5nLCBjaXRh
dGlvbiByYW5raW5nLCBhbmQgaC1pbmRleCBvZiBodW1hbi1jb21wdXRlciBpbnRlcmFjdGlvbiBy
ZXNlYXJjaGVyczogQSBjb21wYXJpc29uIG9mIFNjb3B1cyBhbmQgV2ViIG9mIFNjaWVuY2U8L3Rp
dGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBBbWVyaWNhbiBTb2NpZXR5IGZvciBJbmZvcm1h
dGlvbiBTY2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBh
Z2VzPjE3MTEtMTcyNjwvcGFnZXM+PHZvbHVtZT41OTwvdm9sdW1lPjxudW1iZXI+MTE8L251bWJl
cj48ZGF0ZXM+PHllYXI+MjAwODwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5NZWhvPC9BdXRob3I+PFllYXI+MjAwNzwvWWVhcj48UmVjTnVt
PjI0NTk8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI0NTk8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJy
MncyYWRmZXAiPjI0NTk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h
bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPk1l
aG8sIEwuPC9hdXRob3I+PGF1dGhvcj5ZYW5nLCBLLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRy
aWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbXBhY3Qgb2YgZGF0YSBzb3VyY2VzIG9uIGNpdGF0aW9u
IGNvdW50cyBhbmQgcmFua2luZ3Mgb2YgTElTIGZhY3VsdHk6IFdlYiBvZiBTY2llbmNlLCBTY29w
dXMgYW5kIEdvb2dsZSBTY2hvbGFyPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgQW1l
cmljYW4gU29jaWV0eSBmb3IgSW5mb3JtYXRpb24gU2NpZW5jZSBhbmQgVGVjaG5vbG9neTwvc2Vj
b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4yMTA1LTIxMjU8L3BhZ2VzPjx2b2x1bWU+NTg8
L3ZvbHVtZT48bnVtYmVyPjEzPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDc8L3llYXI+PC9kYXRl
cz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SGFyemluZzwvQXV0
aG9yPjxZZWFyPjIwMDg8L1llYXI+PFJlY051bT4zMjYxPC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj4zMjYxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
cDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjYxPC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IYXJ6aW5nLCBBLi1XLjwvYXV0aG9yPjxhdXRob3I+
dmFuIGRlciBXYWwsIFIuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPkdvb2dsZSBTY2hvbGFyIGFzIGEgbmV3IHNvdXJjZSBmb3IgY2l0YXRpb24gYW5hbHlz
aXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+RXRoaWNzIGluIFNjaWVuY2UgYW5kIEVudmlyb25t
ZW50YWwgUG9saXRpY3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+NjEtNzM8L3Bh
Z2VzPjx2b2x1bWU+ODwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHVy
bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkphY3NvPC9BdXRob3I+PFll
YXI+MjAwNTwvWWVhcj48UmVjTnVtPjMyMjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjMy
MjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2
ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjMyMjM8L2tleT48L2ZvcmVpZ24ta2V5cz48
cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y
cz48YXV0aG9ycz48YXV0aG9yPkphY3NvLCBQLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1
dG9ycz48dGl0bGVzPjx0aXRsZT5BcyB3ZSBtYXkgc2VhcmNoIC0gQ29tcGFyaXNvbiBvZiBtYWpv
ciBmZWF0dXJlcyBvZiB0aGUgV2ViIG9mIFNjaWVuY2UsIFNjb3B1cywgYW5kIEdvb2dsZSBTY2hv
bGFyIGNpdGF0aW9uLWJhc2VkIGFuZCBjaXRhdGlvbi1lbmhhbmNlZCBkYXRhYmFzZXM8L3RpdGxl
PjxzZWNvbmRhcnktdGl0bGU+Q3VycmVudCBTY2llbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs
ZXM+PHBhZ2VzPjE1MzctMTU0NzwvcGFnZXM+PHZvbHVtZT44OTwvdm9sdW1lPjxudW1iZXI+OTwv
bnVtYmVyPjxkYXRlcz48eWVhcj4yMDA1PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk1pbmdlcnM8L0F1dGhvcj48WWVhcj4yMDEwPC9ZZWFy
PjxSZWNOdW0+MzU3MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzU3MDwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3Jw
YXZlY3dlMnIydzJhZGZlcCI+MzU3MDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1l
PSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+TWluZ2VycywgSi48L2F1dGhvcj48YXV0aG9yPkxpcGl0YWtpcywgRS48L2F1dGhvcj48
L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Q291bnRpbmcgdGhlIGNpdGF0
aW9uczogQSBjb21wYXJpc29uIG9mIFdlYiBvZiBTY2llbmNlIGFuZCBHb29nbGUgU2Nob2xhciBp
biB0aGUgZmllbGQgb2YgbWFuYWdlbWVudDwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TY2llbnRv
bWV0cmljcyA8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+NjEzLTYyNTwvcGFnZXM+
PHZvbHVtZT44NTwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDEwPC95
ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5GcmFuY2VzY2hldDwvQXV0aG9yPjxZZWFyPjIwMTA8L1ll
YXI+PFJlY051bT40NDE4PC9SZWNOdW0+PERpc3BsYXlUZXh0PihBZHJpYWFuc2UgJmFtcDsgUmVu
c2xlaWdoLCAyMDEzOyBBbWFyYSAmYW1wOyBMYW5kcnksIDIwMTI7IEZyYW5jZXNjaGV0LCAyMDEw
OyBHYXJjw61hLVDDqXJleiwgMjAxMDsgSGFyemluZyAmYW1wOyB2YW4gZGVyIFdhbCwgMjAwODsg
SmFjc28sIDIwMDU7IE1laG8gJmFtcDsgUm9nZXJzLCAyMDA4OyBNZWhvICZhbXA7IFlhbmcsIDIw
MDc7IE1pbmdlcnMgJmFtcDsgTGlwaXRha2lzLCAyMDEwKTwvRGlzcGxheVRleHQ+PHJlY29yZD48
cmVjLW51bWJlcj40NDE4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40NDE4PC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5GcmFuY2VzY2hldCwgTWFzc2ltbzwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5BIGNvbXBhcmlzb24g
b2YgYmlibGlvbWV0cmljIGluZGljYXRvcnMgZm9yIGNvbXB1dGVyIHNjaWVuY2Ugc2Nob2xhcnMg
YW5kIGpvdXJuYWxzIG9uIFdlYiBvZiBTY2llbmNlIGFuZCBHb29nbGUgU2Nob2xhcjwvdGl0bGU+
PHNlY29uZGFyeS10aXRsZT5TY2llbnRvbWV0cmljczwvc2Vjb25kYXJ5LXRpdGxlPjxhbHQtdGl0
bGU+U2NpZW50b21ldHJpY3M8L2FsdC10aXRsZT48L3RpdGxlcz48cGFnZXM+MjQzLTI1ODwvcGFn
ZXM+PHZvbHVtZT44Mzwvdm9sdW1lPjxudW1iZXI+MTwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29y
ZD5CaWJsaW9tZXRyaWMgaW5kaWNhdG9yczwva2V5d29yZD48a2V5d29yZD5oIEluZGV4PC9rZXl3
b3JkPjxrZXl3b3JkPlB1YmxpY2F0aW9uIGFuZCBjaXRhdGlvbiBkYXRhIHNvdXJjZXM8L2tleXdv
cmQ+PGtleXdvcmQ+Q29ycmVsYXRpb24gYW5hbHlzaXM8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0
ZXM+PHllYXI+MjAxMDwveWVhcj48cHViLWRhdGVzPjxkYXRlPjIwMTAvMDQvMDE8L2RhdGU+PC9w
dWItZGF0ZXM+PC9kYXRlcz48cHVibGlzaGVyPlNwcmluZ2VyIE5ldGhlcmxhbmRzPC9wdWJsaXNo
ZXI+PGlzYm4+MDEzOC05MTMwPC9pc2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8v
ZHguZG9pLm9yZy8xMC4xMDA3L3MxMTE5Mi0wMDktMDAyMS0yPC91cmw+PC9yZWxhdGVkLXVybHM+
PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT4xMC4xMDA3L3MxMTE5Mi0wMDktMDAyMS0y
PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48bGFuZ3VhZ2U+RW5nbGlzaDwvbGFuZ3VhZ2U+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+R2FyY8OtYS1Qw6lyZXo8L0F1dGhvcj48WWVhcj4y
MDEwPC9ZZWFyPjxSZWNOdW0+NDQxNzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQxNzwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3
cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQxNzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYt
dHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+R2FyY8OtYS1Qw6lyZXosIE1pZ3VlbCBBLjwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5BY2N1cmFjeSBhbmQgY29tcGxldGVuZXNz
IG9mIHB1YmxpY2F0aW9uIGFuZCBjaXRhdGlvbiByZWNvcmRzIGluIHRoZSBXZWIgb2YgU2NpZW5j
ZSwgUHN5Y0lORk8sIGFuZCBHb29nbGUgU2Nob2xhcjogQSBjYXNlIHN0dWR5IGZvciB0aGUgY29t
cHV0YXRpb24gb2YgaCBpbmRpY2VzIGluIFBzeWNob2xvZ3k8L3RpdGxlPjxzZWNvbmRhcnktdGl0
bGU+Sm91cm5hbCBvZiB0aGUgQW1lcmljYW4gU29jaWV0eSBmb3IgSW5mb3JtYXRpb24gU2NpZW5j
ZSBhbmQgVGVjaG5vbG9neTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4yMDcwLTIw
ODU8L3BhZ2VzPjx2b2x1bWU+NjE8L3ZvbHVtZT48bnVtYmVyPjEwPC9udW1iZXI+PGRhdGVzPjx5
ZWFyPjIwMTA8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPldpbGV5IFN1YnNjcmlwdGlvbiBTZXJ2
aWNlcywgSW5jLiwgQSBXaWxleSBDb21wYW55PC9wdWJsaXNoZXI+PGlzYm4+MTUzMi0yODkwPC9p
c2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vZHguZG9pLm9yZy8xMC4xMDAyL2Fz
aS4yMTM3MjwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1u
dW0+MTAuMTAwMi9hc2kuMjEzNzI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwv
Q2l0ZT48Q2l0ZT48QXV0aG9yPkFtYXJhPC9BdXRob3I+PFllYXI+MjAxMjwvWWVhcj48UmVjTnVt
PjQ0MTY8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQ0MTY8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJy
MncyYWRmZXAiPjQ0MTY8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h
bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkFt
YXJhLCBOYWJpbDwvYXV0aG9yPjxhdXRob3I+TGFuZHJ5LCBSw6lqZWFuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkNvdW50aW5nIGNpdGF0aW9ucyBpbiB0
aGUgZmllbGQgb2YgYnVzaW5lc3MgYW5kIG1hbmFnZW1lbnQ6IHdoeSB1c2UgR29vZ2xlIFNjaG9s
YXIgcmF0aGVyIHRoYW4gdGhlIFdlYiBvZiBTY2llbmNlPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxl
PlNjaWVudG9tZXRyaWNzPC9zZWNvbmRhcnktdGl0bGU+PGFsdC10aXRsZT5TY2llbnRvbWV0cmlj
czwvYWx0LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz41NTMtNTgxPC9wYWdlcz48dm9sdW1lPjkzPC92
b2x1bWU+PG51bWJlcj4zPC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPkNvbnRyaWJ1dGlvbnMg
cmVjb3JkPC9rZXl3b3JkPjxrZXl3b3JkPkNpdGF0aW9uczwva2V5d29yZD48a2V5d29yZD5oLWlu
ZGV4PC9rZXl3b3JkPjxrZXl3b3JkPklTSTwva2V5d29yZD48a2V5d29yZD5Hb29nbGUgU2Nob2xh
cjwva2V5d29yZD48a2V5d29yZD5CdXNpbmVzcyBzY2hvbGFyczwva2V5d29yZD48L2tleXdvcmRz
PjxkYXRlcz48eWVhcj4yMDEyPC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+MjAxMi8xMi8wMTwvZGF0
ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjxwdWJsaXNoZXI+U3ByaW5nZXIgTmV0aGVybGFuZHM8L3B1
Ymxpc2hlcj48aXNibj4wMTM4LTkxMzA8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0
dHA6Ly9keC5kb2kub3JnLzEwLjEwMDcvczExMTkyLTAxMi0wNzI5LTI8L3VybD48L3JlbGF0ZWQt
dXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwMDcvczExMTkyLTAxMi0w
NzI5LTI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjxsYW5ndWFnZT5FbmdsaXNoPC9sYW5ndWFn
ZT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5BZHJpYWFuc2U8L0F1dGhvcj48WWVhcj4y
MDEzPC9ZZWFyPjxSZWNOdW0+NDQxNTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQxNTwv
cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3
cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQxNTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYt
dHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxh
dXRob3JzPjxhdXRob3I+QWRyaWFhbnNlLCBMZXNsaWU8L2F1dGhvcj48YXV0aG9yPlJlbnNsZWln
aCwgQ2hyaXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
V2ViIG9mIFNjaWVuY2UsIFNjb3B1cyBhbmQgR29vZ2xlIFNjaG9sYXI8L3RpdGxlPjxzZWNvbmRh
cnktdGl0bGU+VGhlIEVsZWN0cm9uaWMgTGlicmFyeTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVz
PjxwYWdlcz43MjctNzQ0PC9wYWdlcz48dm9sdW1lPjMxPC92b2x1bWU+PG51bWJlcj42PC9udW1i
ZXI+PGRhdGVzPjx5ZWFyPjIwMTM8L3llYXI+PC9kYXRlcz48dXJscz48cmVsYXRlZC11cmxzPjx1
cmw+aHR0cDovL3d3dy5lbWVyYWxkaW5zaWdodC5jb20vZG9pL2Ficy8xMC4xMTA4L0VMLTEyLTIw
MTEtMDE3NDwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1u
dW0+ZG9pOjEwLjExMDgvRUwtMTItMjAxMS0wMTc0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48
L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5NZWhvPC9BdXRob3I+PFllYXI+MjAwODwvWWVh
cj48UmVjTnVtPjMyMjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjMyMjE8L3JlYy1udW1i
ZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNy
cGF2ZWN3ZTJyMncyYWRmZXAiPjMyMjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFt
ZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48
YXV0aG9yPk1laG8sIEwuPC9hdXRob3I+PGF1dGhvcj5Sb2dlcnMsIFkuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkNpdGF0aW9uIGNvdW50aW5nLCBjaXRh
dGlvbiByYW5raW5nLCBhbmQgaC1pbmRleCBvZiBodW1hbi1jb21wdXRlciBpbnRlcmFjdGlvbiBy
ZXNlYXJjaGVyczogQSBjb21wYXJpc29uIG9mIFNjb3B1cyBhbmQgV2ViIG9mIFNjaWVuY2U8L3Rp
dGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBBbWVyaWNhbiBTb2NpZXR5IGZvciBJbmZvcm1h
dGlvbiBTY2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBh
Z2VzPjE3MTEtMTcyNjwvcGFnZXM+PHZvbHVtZT41OTwvdm9sdW1lPjxudW1iZXI+MTE8L251bWJl
cj48ZGF0ZXM+PHllYXI+MjAwODwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48
L0NpdGU+PENpdGU+PEF1dGhvcj5NZWhvPC9BdXRob3I+PFllYXI+MjAwNzwvWWVhcj48UmVjTnVt
PjI0NTk8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjI0NTk8L3JlYy1udW1iZXI+PGZvcmVp
Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJy
MncyYWRmZXAiPjI0NTk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h
bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPk1l
aG8sIEwuPC9hdXRob3I+PGF1dGhvcj5ZYW5nLCBLLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRy
aWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbXBhY3Qgb2YgZGF0YSBzb3VyY2VzIG9uIGNpdGF0aW9u
IGNvdW50cyBhbmQgcmFua2luZ3Mgb2YgTElTIGZhY3VsdHk6IFdlYiBvZiBTY2llbmNlLCBTY29w
dXMgYW5kIEdvb2dsZSBTY2hvbGFyPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgQW1l
cmljYW4gU29jaWV0eSBmb3IgSW5mb3JtYXRpb24gU2NpZW5jZSBhbmQgVGVjaG5vbG9neTwvc2Vj
b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4yMTA1LTIxMjU8L3BhZ2VzPjx2b2x1bWU+NTg8
L3ZvbHVtZT48bnVtYmVyPjEzPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDc8L3llYXI+PC9kYXRl
cz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+SGFyemluZzwvQXV0
aG9yPjxZZWFyPjIwMDg8L1llYXI+PFJlY051bT4zMjYxPC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj4zMjYxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
cDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjYxPC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IYXJ6aW5nLCBBLi1XLjwvYXV0aG9yPjxhdXRob3I+
dmFuIGRlciBXYWwsIFIuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPkdvb2dsZSBTY2hvbGFyIGFzIGEgbmV3IHNvdXJjZSBmb3IgY2l0YXRpb24gYW5hbHlz
aXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+RXRoaWNzIGluIFNjaWVuY2UgYW5kIEVudmlyb25t
ZW50YWwgUG9saXRpY3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+NjEtNzM8L3Bh
Z2VzPjx2b2x1bWU+ODwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDA4PC95ZWFyPjwvZGF0ZXM+PHVy
bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkphY3NvPC9BdXRob3I+PFll
YXI+MjAwNTwvWWVhcj48UmVjTnVtPjMyMjM8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjMy
MjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2
ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjMyMjM8L2tleT48L2ZvcmVpZ24ta2V5cz48
cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y
cz48YXV0aG9ycz48YXV0aG9yPkphY3NvLCBQLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1
dG9ycz48dGl0bGVzPjx0aXRsZT5BcyB3ZSBtYXkgc2VhcmNoIC0gQ29tcGFyaXNvbiBvZiBtYWpv
ciBmZWF0dXJlcyBvZiB0aGUgV2ViIG9mIFNjaWVuY2UsIFNjb3B1cywgYW5kIEdvb2dsZSBTY2hv
bGFyIGNpdGF0aW9uLWJhc2VkIGFuZCBjaXRhdGlvbi1lbmhhbmNlZCBkYXRhYmFzZXM8L3RpdGxl
PjxzZWNvbmRhcnktdGl0bGU+Q3VycmVudCBTY2llbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs
ZXM+PHBhZ2VzPjE1MzctMTU0NzwvcGFnZXM+PHZvbHVtZT44OTwvdm9sdW1lPjxudW1iZXI+OTwv
bnVtYmVyPjxkYXRlcz48eWVhcj4yMDA1PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk1pbmdlcnM8L0F1dGhvcj48WWVhcj4yMDEwPC9ZZWFy
PjxSZWNOdW0+MzU3MDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzU3MDwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3Jw
YXZlY3dlMnIydzJhZGZlcCI+MzU3MDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1l
PSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+TWluZ2VycywgSi48L2F1dGhvcj48YXV0aG9yPkxpcGl0YWtpcywgRS48L2F1dGhvcj48
L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Q291bnRpbmcgdGhlIGNpdGF0
aW9uczogQSBjb21wYXJpc29uIG9mIFdlYiBvZiBTY2llbmNlIGFuZCBHb29nbGUgU2Nob2xhciBp
biB0aGUgZmllbGQgb2YgbWFuYWdlbWVudDwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TY2llbnRv
bWV0cmljcyA8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+NjEzLTYyNTwvcGFnZXM+
PHZvbHVtZT44NTwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDEwPC95
ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Adriaanse & Rensleigh, 2013; Amara & Landry, 2012; Franceschet, 2010; García-Pérez, 2010; Harzing & van der Wal, 2008; Jacso, 2005; Meho & Rogers, 2008; Meho & Yang, 2007; Mingers & Lipitakis, 2010). The general conclusions of these studies are:That the coverage of research outputs, including books and reports, is much higher in GS, usually around 90%, and that this is reasonably constant across the subjects. This means that GS has a comparatively greater advantage in the non-science subjects where Scopus and WoS are weak.Partly, but not wholly, because of the coverage, GS generates a significantly greater number of citations for any particular work. This can range from two times to five times as many. This is because the citations come from a wide range of sources, not being limited to the journals that are included in the other databases.However, the data quality in GS is very poor with many entries being duplicated because of small differences in spellings or dates and many of the citations coming from a variety of non-research sources. With regard to the last point, it could be argued that the type of citation does not necessarily matter – it is still impact.There is also an issue about normalising Google Scholar data – see the later section on normalisation ADDIN EN.CITE <EndNote><Cite><Author>Bornmann</Author><Year>2016</Year><RecNum>5099</RecNum><DisplayText>(Bornmann et al., 2016)</DisplayText><record><rec-number>5099</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">5099</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bornmann, Lutz</author><author>Thor, Andreas</author><author>Marx, Werner</author><author>Schier, Hermann</author></authors></contributors><titles><title>The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute</title><secondary-title>Journal of the Association for Information Science and Technology</secondary-title></titles><pages>n/a-n/a</pages><keywords><keyword>bibliometrics</keyword></keywords><dates><year>2016</year></dates><isbn>2330-1643</isbn><urls><related-urls><url>;(Bornmann et al., 2016).Even with the specialised databases (WoS and Scopus) there are still issues around error-correction and disambiguation of common names.Basic Journal Citation indicatorsAfter a paper has been published it may be cited in later papers. The basic unit is the number of times a paper, or a collection of papers, has been cited over a particular time window. In the case of journals, it will be the number of citations to documents of a particular type that were published in the journal over a specific period. Thus, it is generally necessary to specify two time periods, which may be the same, one for the cited papers and one for the citing papers.Total CitationsThe most basic citation metric is simply the total number of citations received by papers in a journal in the relevant time periods. This measure is accurate, robust and transparent. However, it is not unbiased since it is dependent on many factors, some of which do not relate to quality and should therefore be controlled for. The first is the number of papers that the journal publishes. Clearly the more papers published per year the more citations that will accrue but it could be argued that this degree of productivity is not the same as quality or impact. This leads to the mean or average number of citations. The second is that different research fields have very different citation practices and generally the sciences have a much greater citation density than the social sciences or humanities, for example molecular biology was found to have citation rates ten times greater than computer science ADDIN EN.CITE <EndNote><Cite><Author>Iglesias</Author><Year>2007</Year><RecNum>3571</RecNum><DisplayText>(Iglesias & Pecharromán, 2007)</DisplayText><record><rec-number>3571</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3571</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Iglesias, J.</author><author>Pecharromán, C.</author></authors></contributors><titles><title>Scaling the h-index for different scientific ISI fields</title><secondary-title>Scientometrics</secondary-title></titles><pages>303-320</pages><volume>73</volume><number>3</number><keywords><keyword>Computer Science</keyword></keywords><dates><year>2007</year></dates><publisher>Akadémiai Kiadó, co-published with Springer Science+Business Media B.V., Formerly Kluwer Academic Publishers B.V.</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(Iglesias & Pecharromán, 2007). But there may be significant differences even within a multidisciplinary field such as business and management ADDIN EN.CITE <EndNote><Cite><Author>Mingers</Author><Year>2015</Year><RecNum>4448</RecNum><DisplayText>(Mingers & Leydesdorff, 2015a)</DisplayText><record><rec-number>4448</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4448</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Leydesdorff, L.</author></authors></contributors><titles><title>Identifying research fields within business and management: A journal cross-citation analysis</title><secondary-title>Journal of the Operational Research Society</secondary-title></titles><periodical><full-title>Journal of the Operational Research Society</full-title></periodical><pages>1370-1384</pages><volume>66</volume><number>8</number><edition>13/10/2014</edition><dates><year>2015</year></dates><urls></urls><electronic-resource-num> 10.1057/jors.2014.113</electronic-resource-num></record></Cite></EndNote>(Mingers & Leydesdorff, 2015a). This is also related to the average number of authors for a paper – in lab-based science particularly, papers can have many authors effectively increasing the overall number of citations. This is a major problem which makes it very difficult to compare journals across different research fields without some form of field or source normalization. A third factor is the quality of the citation itself – should all citations, from whatever source, be considered equal or is a citation from a highly prestigious journal worth more than one from an obscure journal?Average number of citations per paper (CPP or IPP)Dividing the total citations by the number of papers generating them gives the citations per paper (CPP) ADDIN EN.CITE <EndNote><Cite><Author>van Raan</Author><Year>2003</Year><RecNum>2656</RecNum><DisplayText>(van Raan, 2003)</DisplayText><record><rec-number>2656</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2656</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>van Raan, A.</author></authors></contributors><titles><title>The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments</title><secondary-title>Technology Assessment - Theory and Practice</secondary-title></titles><pages>20-29</pages><volume>1</volume><number>12</number><dates><year>2003</year></dates><urls></urls></record></Cite></EndNote>(van Raan, 2003) or impact per paper (IPP) ADDIN EN.CITE <EndNote><Cite><Author>Moed</Author><Year>2010</Year><RecNum>4275</RecNum><DisplayText>(Moed, 2010b)</DisplayText><record><rec-number>4275</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4275</key></foreign-keys><ref-type name="Electronic Article">43</ref-type><contributors><authors><author>Moed, H.</author></authors></contributors><titles><title>The Source-Normalized Impact per Paper (SNIP) is a valid and sophisticated indicator of journal citation impact</title><secondary-title>arXiv preprint</secondary-title></titles><num-vols>arXiv:1005.4906, 2010</num-vols><dates><year>2010</year></dates><publisher></publisher><urls><related-urls><url>;(Moed, 2010b). This completely normalises for the number of papers but does not normalise for field. Another limitation is that citation data is always very highly skewed ADDIN EN.CITE <EndNote><Cite><Author>Seglen</Author><Year>1992</Year><RecNum>4489</RecNum><DisplayText>(Seglen, 1992)</DisplayText><record><rec-number>4489</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4489</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Seglen, Per O</author></authors></contributors><titles><title>The skewness of science</title><secondary-title>Journal of the American Society for Information Science</secondary-title></titles><pages>628-638</pages><volume>43</volume><number>9</number><dates><year>1992</year></dates><isbn>1097-4571</isbn><urls></urls></record></Cite></EndNote>(Seglen, 1992) and so mean rates may well be distorted by extreme values. This has led to the use of non-parametric measures such as the number or proportion of highly-cited papers PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Cb3JubWFubjwvQXV0aG9yPjxZZWFyPjIwMTM8L1llYXI+
PFJlY051bT40NDExPC9SZWNOdW0+PERpc3BsYXlUZXh0PihCb3JubWFubiBldCBhbC4sIDIwMTM7
IExleWRlc2RvcmZmICZhbXA7IEJvcm5tYW5uLCAyMDExOyBMZXlkZXNkb3JmZiBldCBhbC4sIDIw
MTEpPC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjQ0MTE8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3
ZTJyMncyYWRmZXAiPjQ0MTE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91
cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y
PkJvcm5tYW5uLCBMdXR6PC9hdXRob3I+PGF1dGhvcj5MZXlkZXNkb3JmZiwgTG9ldDwvYXV0aG9y
PjxhdXRob3I+TXV0eiwgUsO8ZGlnZXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+VGhlIHVzZSBvZiBwZXJjZW50aWxlcyBhbmQgcGVyY2VudGlsZSByYW5r
IGNsYXNzZXMgaW4gdGhlIGFuYWx5c2lzIG9mIGJpYmxpb21ldHJpYyBkYXRhOiBPcHBvcnR1bml0
aWVzIGFuZCBsaW1pdHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBJbmZvcm1l
dHJpY3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MTU4LTE2NTwvcGFnZXM+PHZv
bHVtZT43PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPlBlcmNl
bnRpbGU8L2tleXdvcmQ+PGtleXdvcmQ+UGVyY2VudGlsZSByYW5rIGNsYXNzPC9rZXl3b3JkPjxr
ZXl3b3JkPkNpdGF0aW9uIGltcGFjdDwva2V5d29yZD48a2V5d29yZD5SZWZlcmVuY2Ugc2V0PC9r
ZXl3b3JkPjxrZXl3b3JkPk5vcm1hbGl6ZWQgY2l0YXRpb24gaW1wYWN0PC9rZXl3b3JkPjwva2V5
d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTM8L3llYXI+PC9kYXRlcz48aXNibj4xNzUxLTE1Nzc8L2lz
Ym4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20v
c2NpZW5jZS9hcnRpY2xlL3BpaS9TMTc1MTE1NzcxMjAwMDgzMTwvdXJsPjwvcmVsYXRlZC11cmxz
PjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAx
Ni9qLmpvaS4yMDEyLjEwLjAwMTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+TGV5ZGVzZG9yZmY8L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxS
ZWNOdW0+NDQ5MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQ5MjwvcmVjLW51bWJlcj48
Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZl
Y3dlMnIydzJhZGZlcCI+NDQ5Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJK
b3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRo
b3I+TGV5ZGVzZG9yZmYsIExvZXQ8L2F1dGhvcj48YXV0aG9yPkJvcm5tYW5uLCBMdXR6PC9hdXRo
b3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkludGVncmF0ZWQgaW1w
YWN0IGluZGljYXRvcnMgY29tcGFyZWQgd2l0aCBpbXBhY3QgZmFjdG9yczogQW4gYWx0ZXJuYXRp
dmUgcmVzZWFyY2ggZGVzaWduIHdpdGggcG9saWN5IGltcGxpY2F0aW9uczwvdGl0bGU+PHNlY29u
ZGFyeS10aXRsZT5Kb3VybmFsIG9mIHRoZSBBbWVyaWNhbiBTb2NpZXR5IGZvciBJbmZvcm1hdGlv
biBTY2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2Vz
PjIxMzMtMjE0NjwvcGFnZXM+PHZvbHVtZT42Mjwvdm9sdW1lPjxudW1iZXI+MTE8L251bWJlcj48
ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+V2lsZXkgU3Vic2NyaXB0
aW9uIFNlcnZpY2VzLCBJbmMuLCBBIFdpbGV5IENvbXBhbnk8L3B1Ymxpc2hlcj48aXNibj4xNTMy
LTI4OTA8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9keC5kb2kub3JnLzEw
LjEwMDIvYXNpLjIxNjA5PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJl
c291cmNlLW51bT4xMC4xMDAyL2FzaS4yMTYwOTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+TGV5ZGVzZG9yZmY8L0F1dGhvcj48WWVhcj4yMDEx
PC9ZZWFyPjxSZWNOdW0+NDQ5MTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQ5MTwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQ5MTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+TGV5ZGVzZG9yZmYsIExvZXQ8L2F1dGhvcj48YXV0aG9yPkJvcm5tYW5uLCBM
dXR6PC9hdXRob3I+PGF1dGhvcj5NdXR6LCBSw7xkaWdlcjwvYXV0aG9yPjxhdXRob3I+T3B0aG9m
LCBUb2JpYXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
VHVybmluZyB0aGUgdGFibGVzIG9uIGNpdGF0aW9uIGFuYWx5c2lzIG9uZSBtb3JlIHRpbWU6IFBy
aW5jaXBsZXMgZm9yIGNvbXBhcmluZyBzZXRzIG9mIGRvY3VtZW50czwvdGl0bGU+PHNlY29uZGFy
eS10aXRsZT5Kb3VybmFsIG9mIHRoZSBBbWVyaWNhbiBTb2NpZXR5IGZvciBJbmZvcm1hdGlvbiBT
Y2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjEz
NzAtMTM4MTwvcGFnZXM+PHZvbHVtZT42Mjwvdm9sdW1lPjxudW1iZXI+NzwvbnVtYmVyPjxkYXRl
cz48eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PHB1Ymxpc2hlcj5XaWxleSBTdWJzY3JpcHRpb24g
U2VydmljZXMsIEluYy4sIEEgV2lsZXkgQ29tcGFueTwvcHVibGlzaGVyPjxpc2JuPjE1MzItMjg5
MDwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAw
Mi9hc2kuMjE1MzQ8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3Vy
Y2UtbnVtPjEwLjEwMDIvYXNpLjIxNTM0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29y
ZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Cb3JubWFubjwvQXV0aG9yPjxZZWFyPjIwMTM8L1llYXI+
PFJlY051bT40NDExPC9SZWNOdW0+PERpc3BsYXlUZXh0PihCb3JubWFubiBldCBhbC4sIDIwMTM7
IExleWRlc2RvcmZmICZhbXA7IEJvcm5tYW5uLCAyMDExOyBMZXlkZXNkb3JmZiBldCBhbC4sIDIw
MTEpPC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjQ0MTE8L3JlYy1udW1iZXI+PGZv
cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3
ZTJyMncyYWRmZXAiPjQ0MTE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91
cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y
PkJvcm5tYW5uLCBMdXR6PC9hdXRob3I+PGF1dGhvcj5MZXlkZXNkb3JmZiwgTG9ldDwvYXV0aG9y
PjxhdXRob3I+TXV0eiwgUsO8ZGlnZXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+VGhlIHVzZSBvZiBwZXJjZW50aWxlcyBhbmQgcGVyY2VudGlsZSByYW5r
IGNsYXNzZXMgaW4gdGhlIGFuYWx5c2lzIG9mIGJpYmxpb21ldHJpYyBkYXRhOiBPcHBvcnR1bml0
aWVzIGFuZCBsaW1pdHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBJbmZvcm1l
dHJpY3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MTU4LTE2NTwvcGFnZXM+PHZv
bHVtZT43PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGtleXdvcmRzPjxrZXl3b3JkPlBlcmNl
bnRpbGU8L2tleXdvcmQ+PGtleXdvcmQ+UGVyY2VudGlsZSByYW5rIGNsYXNzPC9rZXl3b3JkPjxr
ZXl3b3JkPkNpdGF0aW9uIGltcGFjdDwva2V5d29yZD48a2V5d29yZD5SZWZlcmVuY2Ugc2V0PC9r
ZXl3b3JkPjxrZXl3b3JkPk5vcm1hbGl6ZWQgY2l0YXRpb24gaW1wYWN0PC9rZXl3b3JkPjwva2V5
d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTM8L3llYXI+PC9kYXRlcz48aXNibj4xNzUxLTE1Nzc8L2lz
Ym4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly93d3cuc2NpZW5jZWRpcmVjdC5jb20v
c2NpZW5jZS9hcnRpY2xlL3BpaS9TMTc1MTE1NzcxMjAwMDgzMTwvdXJsPjwvcmVsYXRlZC11cmxz
PjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAx
Ni9qLmpvaS4yMDEyLjEwLjAwMTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+TGV5ZGVzZG9yZmY8L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxS
ZWNOdW0+NDQ5MjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQ5MjwvcmVjLW51bWJlcj48
Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZl
Y3dlMnIydzJhZGZlcCI+NDQ5Mjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJK
b3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRo
b3I+TGV5ZGVzZG9yZmYsIExvZXQ8L2F1dGhvcj48YXV0aG9yPkJvcm5tYW5uLCBMdXR6PC9hdXRo
b3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkludGVncmF0ZWQgaW1w
YWN0IGluZGljYXRvcnMgY29tcGFyZWQgd2l0aCBpbXBhY3QgZmFjdG9yczogQW4gYWx0ZXJuYXRp
dmUgcmVzZWFyY2ggZGVzaWduIHdpdGggcG9saWN5IGltcGxpY2F0aW9uczwvdGl0bGU+PHNlY29u
ZGFyeS10aXRsZT5Kb3VybmFsIG9mIHRoZSBBbWVyaWNhbiBTb2NpZXR5IGZvciBJbmZvcm1hdGlv
biBTY2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2Vz
PjIxMzMtMjE0NjwvcGFnZXM+PHZvbHVtZT42Mjwvdm9sdW1lPjxudW1iZXI+MTE8L251bWJlcj48
ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+V2lsZXkgU3Vic2NyaXB0
aW9uIFNlcnZpY2VzLCBJbmMuLCBBIFdpbGV5IENvbXBhbnk8L3B1Ymxpc2hlcj48aXNibj4xNTMy
LTI4OTA8L2lzYm4+PHVybHM+PHJlbGF0ZWQtdXJscz48dXJsPmh0dHA6Ly9keC5kb2kub3JnLzEw
LjEwMDIvYXNpLjIxNjA5PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJl
c291cmNlLW51bT4xMC4xMDAyL2FzaS4yMTYwOTwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PC9y
ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+TGV5ZGVzZG9yZmY8L0F1dGhvcj48WWVhcj4yMDEx
PC9ZZWFyPjxSZWNOdW0+NDQ5MTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQ5MTwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQ5MTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+TGV5ZGVzZG9yZmYsIExvZXQ8L2F1dGhvcj48YXV0aG9yPkJvcm5tYW5uLCBM
dXR6PC9hdXRob3I+PGF1dGhvcj5NdXR6LCBSw7xkaWdlcjwvYXV0aG9yPjxhdXRob3I+T3B0aG9m
LCBUb2JpYXM8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
VHVybmluZyB0aGUgdGFibGVzIG9uIGNpdGF0aW9uIGFuYWx5c2lzIG9uZSBtb3JlIHRpbWU6IFBy
aW5jaXBsZXMgZm9yIGNvbXBhcmluZyBzZXRzIG9mIGRvY3VtZW50czwvdGl0bGU+PHNlY29uZGFy
eS10aXRsZT5Kb3VybmFsIG9mIHRoZSBBbWVyaWNhbiBTb2NpZXR5IGZvciBJbmZvcm1hdGlvbiBT
Y2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjEz
NzAtMTM4MTwvcGFnZXM+PHZvbHVtZT42Mjwvdm9sdW1lPjxudW1iZXI+NzwvbnVtYmVyPjxkYXRl
cz48eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PHB1Ymxpc2hlcj5XaWxleSBTdWJzY3JpcHRpb24g
U2VydmljZXMsIEluYy4sIEEgV2lsZXkgQ29tcGFueTwvcHVibGlzaGVyPjxpc2JuPjE1MzItMjg5
MDwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAw
Mi9hc2kuMjE1MzQ8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3Vy
Y2UtbnVtPjEwLjEwMDIvYXNpLjIxNTM0PC9lbGVjdHJvbmljLXJlc291cmNlLW51bT48L3JlY29y
ZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Bornmann et al., 2013; Leydesdorff & Bornmann, 2011; Leydesdorff et al., 2011).Journal impact factor (JIF) The JIF was the first, and is the most well-known journal metric. This was originally developed by Garfield and Sher ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>1963</Year><RecNum>4505</RecNum><DisplayText>(1963)</DisplayText><record><rec-number>4505</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4505</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Garfield, E.</author><author>Sher, I. H.</author></authors></contributors><titles><title>New factors in the evaluation of scientific literature through citation indexing</title><secondary-title>American Documentation</secondary-title></titles><pages>195-201</pages><volume>14</volume><number>3</number><dates><year>1963</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1936-6108</isbn><urls><related-urls><url>;(1963) as a way of choosing journals to include in their newly-created science citation index (SCI). It is simply a two year mean citations per paper based on the number of citations in year t to papers published in the previous two years. JIF is published every year for all the journals that are included in Thompson Reuters’ Web of Science and is viewed as highly influential. Garfield recommended that it should be used in combination with another metric, the “cited half-life” which measures how long citations last for – it is the median age of papers cited by a journal in a particular JCR year. WoS also calculates the immediacy index which is a 1-year JIF, i.e., the mean citations in year t to papers published in year t. The JIF has several limitations ADDIN EN.CITE <EndNote><Cite><Author>Gl?nzel</Author><Year>2002</Year><RecNum>2193</RecNum><DisplayText>(Gl?nzel & Moed, 2002; Harzing & Van der Wal, 2009)</DisplayText><record><rec-number>2193</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2193</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gl?nzel, W.</author><author>Moed, H. K.</author></authors></contributors><titles><title>Journal impact measures in bibliometric research</title><secondary-title>Scientometrics</secondary-title></titles><pages>171-193</pages><volume>53</volume><number>2</number><dates><year>2002</year></dates><urls></urls></record></Cite><Cite><Author>Harzing</Author><Year>2009</Year><RecNum>3466</RecNum><record><rec-number>3466</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3466</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Harzing, A.-W.</author><author>Van der Wal, R.</author></authors></contributors><titles><title>A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business?</title><secondary-title>J. Am. Soc. for Information Science and Technology</secondary-title></titles><pages>41-46</pages><volume>60</volume><number>1</number><dates><year>2009</year></dates><urls></urls></record></Cite></EndNote>(Gl?nzel & Moed, 2002; Harzing & Van der Wal, 2009): JIF depends heavily on the research field. For instance, in 2013 the top journal in cell biology had a JIF of 36.5 and Nature has one of 42.4 while the top journal in management, Academy of Management Review, has a JIF of only 7.8 and many are less than 1. The two-year window. This is a very short time period for many disciplines, especially given the lead time between submitting a paper and having it published which may itself be two years. The 5-year JIF is better in this respect ADDIN EN.CITE <EndNote><Cite><Author>Campanario</Author><Year>2011</Year><RecNum>4479</RecNum><DisplayText>(Campanario, 2011)</DisplayText><record><rec-number>4479</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4479</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Campanario, Juan Miguel</author></authors></contributors><titles><title>Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window</title><secondary-title>Scientometrics</secondary-title></titles><pages>189-204</pages><volume>87</volume><number>1</number><dates><year>2011</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(Campanario, 2011). There is a lack of transparency in the way the JIF is calculated and this casts doubt on the results. Brumback ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2008</Year><RecNum>2655</RecNum><DisplayText>(2008)</DisplayText><record><rec-number>2655</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2655</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Brumback, R.</author></authors></contributors><titles><title>Worshipping false idols: the impact factor dilemma</title><secondary-title>Journal Child Neurology</secondary-title></titles><pages>365-367</pages><volume>23</volume><dates><year>2008</year></dates><urls></urls></record></Cite></EndNote>(2008) studied reviews journals and could not reproduce the appropriate figures. It is highly dependent on which types of papers are included in the denominator. Pislyakov ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2009</Year><RecNum>4472</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>4472</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4472</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Pislyakov, Vladimir</author></authors></contributors><titles><title>Comparing two “thermometers”: Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus</title><secondary-title>Scientometrics</secondary-title></titles><pages>541-550</pages><volume>79</volume><number>3</number><dates><year>2009</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(2009) compared JIFs calculated from WoS and Scopus data and found important differences mainly due to the different coverage of the databases. The situation may be improved more recently.It is possible for journals to deliberately distort the results by, for example, publishing many review articles which are more highly cited; publishing short reports or book reviews that get cited but are not included in the count of papers; publishing yearly overviews of the research published in the journal or pressuring authors to gratuitously reference excessive papers from the journal PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XaWxoaXRlPC9BdXRob3I+PFllYXI+MjAxMjwvWWVhcj48
UmVjTnVtPjQ1NDA8L1JlY051bT48RGlzcGxheVRleHQ+KExvd3J5IGV0IGFsLiwgMjAxMzsgTW9l
ZCwgMjAwMDsgV2lsaGl0ZSAmYW1wOyBGb25nLCAyMDEyKTwvRGlzcGxheVRleHQ+PHJlY29yZD48
cmVjLW51bWJlcj40NTQwPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40NTQwPC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XaWxoaXRlLCBBLjwvYXV0aG9yPjxhdXRo
b3I+Rm9uZywgRS48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0
bGU+Q29lcmNpdmUgY2l0YXRpb24gaW4gYWNhZGVtaWMgcHVibGlzaGluZzwvdGl0bGU+PHNlY29u
ZGFyeS10aXRsZT5TY2llbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjU0Mi01
NDM8L3BhZ2VzPjx2b2x1bWU+MzM1PC92b2x1bWU+PG51bWJlcj42MDY4PC9udW1iZXI+PGRhdGVz
Pjx5ZWFyPjIwMTI8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD
aXRlPjxBdXRob3I+TW9lZDwvQXV0aG9yPjxZZWFyPjIwMDA8L1llYXI+PFJlY051bT40NjA5PC9S
ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40NjA5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVw
Ij40NjA5PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs
ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Nb2VkLCBILjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5CaWJsaW9tZXRy
aWMgaW5kaWNhdG9ycyByZWZsZWN0IHB1YmxpY2F0aW9uIGFuZCBtYW5hZ2VtZW50IHN0cmF0ZWdp
ZXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NpZW50b21ldHJpY3M8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+MzIzLTM0NjwvcGFnZXM+PHZvbHVtZT40Nzwvdm9sdW1lPjxudW1i
ZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDAwPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxz
PjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkxvd3J5PC9BdXRob3I+PFllYXI+MjAxMzwv
WWVhcj48UmVjTnVtPjUwNjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjUwNjE8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3
dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjUwNjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUg
bmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9y
cz48YXV0aG9yPkxvd3J5LCBQYXVsIEJlbmphbWluPC9hdXRob3I+PGF1dGhvcj5Nb29keSwgR3Jl
Z29yeSBELjwvYXV0aG9yPjxhdXRob3I+R2Fza2luLCBKYW1lczwvYXV0aG9yPjxhdXRob3I+R2Fs
bGV0dGEsIERlbm5pcyBGLjwvYXV0aG9yPjxhdXRob3I+SHVtcGhlcnlzLCBTZWFuIEwuPC9hdXRo
b3I+PGF1dGhvcj5CYXJsb3csIEpvcmRhbiBCLjwvYXV0aG9yPjxhdXRob3I+V2lsc29uLCBEYXZp
ZCBXLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5FdmFs
dWF0aW5nIGpvdXJuYWwgcXVhbGl0eSBhbmQgdGhlIEFzc29jaWF0aW9uIGZvciBJbmZvcm1hdGlv
biBTeXN0ZW1zIFNlbmlvciBTY2hvbGFycyZhcG9zOyBqb3VybmFsIGJhc2tldCB2aWEgYmlibGlv
bWV0cmljIG1lYXN1cmVzOiBEbyBleHBlcnQgam91cm5hbCBhc3Nlc3NtZW50cyBhZGQgdmFsdWU/
PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk1JUyBxdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48
L3RpdGxlcz48cGFnZXM+OTkzLUEyMTwvcGFnZXM+PHZvbHVtZT4zNzwvdm9sdW1lPjxudW1iZXI+
NDwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5JTkZPUk1BVElPTiBzdG9yYWdlICZhbXA7IHJl
dHJpZXZhbCBzeXN0ZW1zPC9rZXl3b3JkPjxrZXl3b3JkPk1BTkFHRU1FTlQgaW5mb3JtYXRpb24g
c3lzdGVtczwva2V5d29yZD48a2V5d29yZD5TQ0lFTlRPTUVUUklDUzwva2V5d29yZD48a2V5d29y
ZD5JTVBBQ1QgZmFjdG9yIChDaXRhdGlvbiBhbmFseXNpcyk8L2tleXdvcmQ+PGtleXdvcmQ+UEVS
SU9ESUNBTFM8L2tleXdvcmQ+PGtleXdvcmQ+Sk9VUk5BTCBwcm9kdWN0aXZpdHk8L2tleXdvcmQ+
PGtleXdvcmQ+QklCTElPTUVUUklDUzwva2V5d29yZD48a2V5d29yZD5BSVMgU2VuaW9yIFNjaG9s
YXJzIGJhc2tldCBvZiBqb3VybmFsczwva2V5d29yZD48a2V5d29yZD5jb21wb3NpdGUgcmFua2lu
ZyBvciByYXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+ZXhwZXJ0IG9waW5pb248L2tleXdvcmQ+PGtl
eXdvcmQ+aC1pbmRleDwva2V5d29yZD48a2V5d29yZD5pbXBhY3QgZmFjdG9yPC9rZXl3b3JkPjxr
ZXl3b3JkPkluZm9ybWF0aW9uIHN5c3RlbXMgam91cm5hbCByYW5raW5nczwva2V5d29yZD48a2V5
d29yZD5qb3VybmFsIHF1YWxpdHk8L2tleXdvcmQ+PGtleXdvcmQ+bm9tb2xvZ2llcyBmb3IgZGlz
c2VtaW5hdGlvbiBvZiBzY2llbnRpZmljIGtub3dsZWRnZTwva2V5d29yZD48a2V5d29yZD5zZWxm
LWNpdGF0aW9uPC9rZXl3b3JkPjxrZXl3b3JkPlNlblMtNjwva2V5d29yZD48a2V5d29yZD5TZW5T
LTg8L2tleXdvcmQ+PGtleXdvcmQ+c29jaWFsIG5ldHdvcmsgYW5hbHlzaXM8L2tleXdvcmQ+PC9r
ZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+TUlTIFF1
YXJ0ZXJseTwvcHVibGlzaGVyPjxpc2JuPjAyNzY3NzgzPC9pc2JuPjxhY2Nlc3Npb24tbnVtPjkx
OTA2MjQxPC9hY2Nlc3Npb24tbnVtPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vc2Vh
cmNoLmVic2NvaG9zdC5jb20vbG9naW4uYXNweD9kaXJlY3Q9dHJ1ZSZhbXA7ZGI9YnRoJmFtcDtB
Tj05MTkwNjI0MSZhbXA7c2l0ZT1laG9zdC1saXZlPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxz
PjxyZW1vdGUtZGF0YWJhc2UtbmFtZT5idGg8L3JlbW90ZS1kYXRhYmFzZS1uYW1lPjxyZW1vdGUt
ZGF0YWJhc2UtcHJvdmlkZXI+RUJTQ09ob3N0PC9yZW1vdGUtZGF0YWJhc2UtcHJvdmlkZXI+PC9y
ZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5XaWxoaXRlPC9BdXRob3I+PFllYXI+MjAxMjwvWWVhcj48
UmVjTnVtPjQ1NDA8L1JlY051bT48RGlzcGxheVRleHQ+KExvd3J5IGV0IGFsLiwgMjAxMzsgTW9l
ZCwgMjAwMDsgV2lsaGl0ZSAmYW1wOyBGb25nLCAyMDEyKTwvRGlzcGxheVRleHQ+PHJlY29yZD48
cmVjLW51bWJlcj40NTQwPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBk
Yi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40NTQwPC9rZXk+PC9m
b3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBl
Pjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XaWxoaXRlLCBBLjwvYXV0aG9yPjxhdXRo
b3I+Rm9uZywgRS48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0
bGU+Q29lcmNpdmUgY2l0YXRpb24gaW4gYWNhZGVtaWMgcHVibGlzaGluZzwvdGl0bGU+PHNlY29u
ZGFyeS10aXRsZT5TY2llbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjU0Mi01
NDM8L3BhZ2VzPjx2b2x1bWU+MzM1PC92b2x1bWU+PG51bWJlcj42MDY4PC9udW1iZXI+PGRhdGVz
Pjx5ZWFyPjIwMTI8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD
aXRlPjxBdXRob3I+TW9lZDwvQXV0aG9yPjxZZWFyPjIwMDA8L1llYXI+PFJlY051bT40NjA5PC9S
ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj40NjA5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+
PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVw
Ij40NjA5PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs
ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Nb2VkLCBILjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5CaWJsaW9tZXRy
aWMgaW5kaWNhdG9ycyByZWZsZWN0IHB1YmxpY2F0aW9uIGFuZCBtYW5hZ2VtZW50IHN0cmF0ZWdp
ZXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NpZW50b21ldHJpY3M8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+MzIzLTM0NjwvcGFnZXM+PHZvbHVtZT40Nzwvdm9sdW1lPjxudW1i
ZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDAwPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxz
PjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkxvd3J5PC9BdXRob3I+PFllYXI+MjAxMzwv
WWVhcj48UmVjTnVtPjUwNjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjUwNjE8L3JlYy1u
dW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3
dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjUwNjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUg
bmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9y
cz48YXV0aG9yPkxvd3J5LCBQYXVsIEJlbmphbWluPC9hdXRob3I+PGF1dGhvcj5Nb29keSwgR3Jl
Z29yeSBELjwvYXV0aG9yPjxhdXRob3I+R2Fza2luLCBKYW1lczwvYXV0aG9yPjxhdXRob3I+R2Fs
bGV0dGEsIERlbm5pcyBGLjwvYXV0aG9yPjxhdXRob3I+SHVtcGhlcnlzLCBTZWFuIEwuPC9hdXRo
b3I+PGF1dGhvcj5CYXJsb3csIEpvcmRhbiBCLjwvYXV0aG9yPjxhdXRob3I+V2lsc29uLCBEYXZp
ZCBXLjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5FdmFs
dWF0aW5nIGpvdXJuYWwgcXVhbGl0eSBhbmQgdGhlIEFzc29jaWF0aW9uIGZvciBJbmZvcm1hdGlv
biBTeXN0ZW1zIFNlbmlvciBTY2hvbGFycyZhcG9zOyBqb3VybmFsIGJhc2tldCB2aWEgYmlibGlv
bWV0cmljIG1lYXN1cmVzOiBEbyBleHBlcnQgam91cm5hbCBhc3Nlc3NtZW50cyBhZGQgdmFsdWU/
PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk1JUyBxdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48
L3RpdGxlcz48cGFnZXM+OTkzLUEyMTwvcGFnZXM+PHZvbHVtZT4zNzwvdm9sdW1lPjxudW1iZXI+
NDwvbnVtYmVyPjxrZXl3b3Jkcz48a2V5d29yZD5JTkZPUk1BVElPTiBzdG9yYWdlICZhbXA7IHJl
dHJpZXZhbCBzeXN0ZW1zPC9rZXl3b3JkPjxrZXl3b3JkPk1BTkFHRU1FTlQgaW5mb3JtYXRpb24g
c3lzdGVtczwva2V5d29yZD48a2V5d29yZD5TQ0lFTlRPTUVUUklDUzwva2V5d29yZD48a2V5d29y
ZD5JTVBBQ1QgZmFjdG9yIChDaXRhdGlvbiBhbmFseXNpcyk8L2tleXdvcmQ+PGtleXdvcmQ+UEVS
SU9ESUNBTFM8L2tleXdvcmQ+PGtleXdvcmQ+Sk9VUk5BTCBwcm9kdWN0aXZpdHk8L2tleXdvcmQ+
PGtleXdvcmQ+QklCTElPTUVUUklDUzwva2V5d29yZD48a2V5d29yZD5BSVMgU2VuaW9yIFNjaG9s
YXJzIGJhc2tldCBvZiBqb3VybmFsczwva2V5d29yZD48a2V5d29yZD5jb21wb3NpdGUgcmFua2lu
ZyBvciByYXRpbmc8L2tleXdvcmQ+PGtleXdvcmQ+ZXhwZXJ0IG9waW5pb248L2tleXdvcmQ+PGtl
eXdvcmQ+aC1pbmRleDwva2V5d29yZD48a2V5d29yZD5pbXBhY3QgZmFjdG9yPC9rZXl3b3JkPjxr
ZXl3b3JkPkluZm9ybWF0aW9uIHN5c3RlbXMgam91cm5hbCByYW5raW5nczwva2V5d29yZD48a2V5
d29yZD5qb3VybmFsIHF1YWxpdHk8L2tleXdvcmQ+PGtleXdvcmQ+bm9tb2xvZ2llcyBmb3IgZGlz
c2VtaW5hdGlvbiBvZiBzY2llbnRpZmljIGtub3dsZWRnZTwva2V5d29yZD48a2V5d29yZD5zZWxm
LWNpdGF0aW9uPC9rZXl3b3JkPjxrZXl3b3JkPlNlblMtNjwva2V5d29yZD48a2V5d29yZD5TZW5T
LTg8L2tleXdvcmQ+PGtleXdvcmQ+c29jaWFsIG5ldHdvcmsgYW5hbHlzaXM8L2tleXdvcmQ+PC9r
ZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+TUlTIFF1
YXJ0ZXJseTwvcHVibGlzaGVyPjxpc2JuPjAyNzY3NzgzPC9pc2JuPjxhY2Nlc3Npb24tbnVtPjkx
OTA2MjQxPC9hY2Nlc3Npb24tbnVtPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vc2Vh
cmNoLmVic2NvaG9zdC5jb20vbG9naW4uYXNweD9kaXJlY3Q9dHJ1ZSZhbXA7ZGI9YnRoJmFtcDtB
Tj05MTkwNjI0MSZhbXA7c2l0ZT1laG9zdC1saXZlPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxz
PjxyZW1vdGUtZGF0YWJhc2UtbmFtZT5idGg8L3JlbW90ZS1kYXRhYmFzZS1uYW1lPjxyZW1vdGUt
ZGF0YWJhc2UtcHJvdmlkZXI+RUJTQ09ob3N0PC9yZW1vdGUtZGF0YWJhc2UtcHJvdmlkZXI+PC9y
ZWNvcmQ+PC9DaXRlPjwvRW5kTm90ZT4A
ADDIN EN.CITE.DATA (Lowry et al., 2013; Moed, 2000; Wilhite & Fong, 2012). The h-indexThis is a relatively new indicator proposed by Hirsch ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2005</Year><RecNum>2418</RecNum><DisplayText>(2005)</DisplayText><record><rec-number>2418</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2418</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hirsch, J.</author></authors></contributors><titles><title>An index to quantify an individual's scientific research output</title><secondary-title>Proceedings of the National Academy of Sciences of the United States of America</secondary-title></titles><pages>16569-16572</pages><volume>102</volume><number>46</number><dates><year>2005</year></dates><urls></urls></record></Cite></EndNote>(2005) that has generated a huge amount of interest. It can be used for journals, individual researchers, or departments. We will only summarise the main advantages and disadvantages, for more detailed reviews see PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Db3N0YXM8L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxS
ZWNOdW0+MjY3MzwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQWxvbnNvIGV0IGFsLiwgMjAwOTsgQm9y
bm1hbiAmYW1wOyBEYW5pZWwsIDIwMDU7IENvc3RhcyAmYW1wOyBCb3Jkb25zLCAyMDA3OyBHbMOk
bnplbCwgMjAwNjsgTm9ycmlzICZhbXA7IE9wcGVuaGVpbSwgMjAxMCk8L0Rpc3BsYXlUZXh0Pjxy
ZWNvcmQ+PHJlYy1udW1iZXI+MjY3MzwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBw
PSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+MjY3Mzwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9y
ZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q29zdGFzLCBSLjwvYXV0aG9y
PjxhdXRob3I+Qm9yZG9ucywgTS48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRp
dGxlcz48dGl0bGU+VGhlIGgtaW5kZXg6IEFkdmFudGFnZXMsIGxpbWl0YXRpb25zIGFuZCBpdHMg
cmVsYXRpb24gd2l0aCBvdGhlciBiaWJsaW9tZXRyaWMgaW5kaWNhdG9ycyBhdCB0aGUgbWljcm8g
bGV2ZWw8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBJbmZvcm1ldHJpY3M8L3Nl
Y29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MTkzLTIwMzwvcGFnZXM+PHZvbHVtZT4xPC92
b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMDc8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNv
cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+QWxvbnNvPC9BdXRob3I+PFllYXI+MjAwOTwvWWVhcj48
UmVjTnVtPjM4MTg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM4MTg8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2
ZWN3ZTJyMncyYWRmZXAiPjM4MTg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i
Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkFsb25zbywgUy48L2F1dGhvcj48YXV0aG9yPkNhYnJlcml6bywgRi4gSi48L2F1dGhvcj48
YXV0aG9yPkhlcnJlcmEtVmllZG1hLCBFLjwvYXV0aG9yPjxhdXRob3I+SGVycmVyYSwgRi48L2F1
dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+aC1JbmRleDogQSBy
ZXZpZXcgZm9jdXNlZCBpbiBpdHMgdmFyaWFudHMsIGNvbXB1dGF0aW9uIGFuZCBzdGFuZGFyZGl6
YXRpb24gZm9yIGRpZmZlcmVudCBzY2llbnRpZmljIGZpZWxkczwvdGl0bGU+PHNlY29uZGFyeS10
aXRsZT5Kb3VybmFsIG9mIEluZm9ybWV0cmljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxw
YWdlcz4yNzMtMjg5PC9wYWdlcz48dm9sdW1lPjM8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48
a2V5d29yZHM+PGtleXdvcmQ+aC1JbmRleDwva2V5d29yZD48a2V5d29yZD5CaWJsaW9tZXRyaWMg
aW5kaWNhdG9yczwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDA5PC95ZWFyPjwv
ZGF0ZXM+PGlzYm4+MTc1MS0xNTc3PC9pc2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRw
Oi8vd3d3LnNjaWVuY2VkaXJlY3QuY29tL3NjaWVuY2UvYXJ0aWNsZS9waWkvUzE3NTExNTc3MDkw
MDAzMzk8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVt
Pmh0dHA6Ly9keC5kb2kub3JnLzEwLjEwMTYvai5qb2kuMjAwOS4wNC4wMDE8L2VsZWN0cm9uaWMt
cmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkdsw6RuemVsPC9BdXRo
b3I+PFllYXI+MjAwNjwvWWVhcj48UmVjTnVtPjI0NTQ8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt
YmVyPjI0NTQ8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJw
OTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjI0NTQ8L2tleT48L2ZvcmVpZ24t
a2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkdsw6RuemVsLCBXLjwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5PbiB0aGUgaC1pbmRleCAtIGEgbWF0aGVtYXRp
Y2FsIGFwcHJvYWNoIHRvIGEgbmV3IG1lYXN1cmUgb2YgcHVibGljYXRpb24gYWN0aXZpdHkgYW5k
IGNpdGF0aW9uIGltcGFjdDwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TY2llbnRvbWV0cmljczwv
c2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4zMTUtMzIxPC9wYWdlcz48dm9sdW1lPjY3
PC92b2x1bWU+PG51bWJlcj4yPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDY8L3llYXI+PC9kYXRl
cz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Qm9ybm1hbjwvQXV0
aG9yPjxZZWFyPjIwMDU8L1llYXI+PFJlY051bT4yNDQ2PC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj4yNDQ2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
cDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4yNDQ2PC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Cb3JubWFuLCBMLjwvYXV0aG9yPjxhdXRob3I+RGFu
aWVsLCBILi1ELjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs
ZT5Eb2VzIHRoZSBoLWluZGV4IGZvciByYW5raW5nIG9mIHNjaWVudGlzdHMgcmVhbGx5IHdvcms/
PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNjaWVudG9tZXRyaWNzPC9zZWNvbmRhcnktdGl0bGU+
PC90aXRsZXM+PHBhZ2VzPjM5MS0zOTI8L3BhZ2VzPjx2b2x1bWU+NjU8L3ZvbHVtZT48bnVtYmVy
PjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwNTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48
L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Ob3JyaXM8L0F1dGhvcj48WWVhcj4yMDEwPC9Z
ZWFyPjxSZWNOdW0+NDgzNTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDgzNTwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0
c3JwYXZlY3dlMnIydzJhZGZlcCI+NDgzNTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+Tm9ycmlzLCBNLjwvYXV0aG9yPjxhdXRob3I+T3BwZW5oZWltLCBDLjwvYXV0aG9y
PjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UaGUgaOKAkGluZGV4OiBh
IGJyb2FkIHJldmlldyBvZiBhIG5ldyBiaWJsaW9tZXRyaWMgaW5kaWNhdG9ybnVsbDwvdGl0bGU+
PHNlY29uZGFyeS10aXRsZT5Kb3VybmFsIG9mIERvY3VtZW50YXRpb248L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+NjgxLTcwNTwvcGFnZXM+PHZvbHVtZT42Njwvdm9sdW1lPjxudW1i
ZXI+NTwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDEwPC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+MjAx
MC8wOS8wNzwvZGF0ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjxwdWJsaXNoZXI+RW1lcmFsZDwvcHVi
bGlzaGVyPjxpc2JuPjAwMjItMDQxODwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0
cDovL2R4LmRvaS5vcmcvMTAuMTEwOC8wMDIyMDQxMTAxMTA2Njc5MDwvdXJsPjwvcmVsYXRlZC11
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTEwOC8wMDIyMDQxMTAxMTA2
Njc5MDwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PGFjY2Vzcy1kYXRlPjIwMTUvMTEvMDU8L2Fj
Y2Vzcy1kYXRlPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Db3N0YXM8L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxS
ZWNOdW0+MjY3MzwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQWxvbnNvIGV0IGFsLiwgMjAwOTsgQm9y
bm1hbiAmYW1wOyBEYW5pZWwsIDIwMDU7IENvc3RhcyAmYW1wOyBCb3Jkb25zLCAyMDA3OyBHbMOk
bnplbCwgMjAwNjsgTm9ycmlzICZhbXA7IE9wcGVuaGVpbSwgMjAxMCk8L0Rpc3BsYXlUZXh0Pjxy
ZWNvcmQ+PHJlYy1udW1iZXI+MjY3MzwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBw
PSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+MjY3Mzwv
a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9y
ZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q29zdGFzLCBSLjwvYXV0aG9y
PjxhdXRob3I+Qm9yZG9ucywgTS48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRp
dGxlcz48dGl0bGU+VGhlIGgtaW5kZXg6IEFkdmFudGFnZXMsIGxpbWl0YXRpb25zIGFuZCBpdHMg
cmVsYXRpb24gd2l0aCBvdGhlciBiaWJsaW9tZXRyaWMgaW5kaWNhdG9ycyBhdCB0aGUgbWljcm8g
bGV2ZWw8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBJbmZvcm1ldHJpY3M8L3Nl
Y29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MTkzLTIwMzwvcGFnZXM+PHZvbHVtZT4xPC92
b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMDc8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNv
cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+QWxvbnNvPC9BdXRob3I+PFllYXI+MjAwOTwvWWVhcj48
UmVjTnVtPjM4MTg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjM4MTg8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2
ZWN3ZTJyMncyYWRmZXAiPjM4MTg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i
Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkFsb25zbywgUy48L2F1dGhvcj48YXV0aG9yPkNhYnJlcml6bywgRi4gSi48L2F1dGhvcj48
YXV0aG9yPkhlcnJlcmEtVmllZG1hLCBFLjwvYXV0aG9yPjxhdXRob3I+SGVycmVyYSwgRi48L2F1
dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+aC1JbmRleDogQSBy
ZXZpZXcgZm9jdXNlZCBpbiBpdHMgdmFyaWFudHMsIGNvbXB1dGF0aW9uIGFuZCBzdGFuZGFyZGl6
YXRpb24gZm9yIGRpZmZlcmVudCBzY2llbnRpZmljIGZpZWxkczwvdGl0bGU+PHNlY29uZGFyeS10
aXRsZT5Kb3VybmFsIG9mIEluZm9ybWV0cmljczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxw
YWdlcz4yNzMtMjg5PC9wYWdlcz48dm9sdW1lPjM8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48
a2V5d29yZHM+PGtleXdvcmQ+aC1JbmRleDwva2V5d29yZD48a2V5d29yZD5CaWJsaW9tZXRyaWMg
aW5kaWNhdG9yczwva2V5d29yZD48L2tleXdvcmRzPjxkYXRlcz48eWVhcj4yMDA5PC95ZWFyPjwv
ZGF0ZXM+PGlzYm4+MTc1MS0xNTc3PC9pc2JuPjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRw
Oi8vd3d3LnNjaWVuY2VkaXJlY3QuY29tL3NjaWVuY2UvYXJ0aWNsZS9waWkvUzE3NTExNTc3MDkw
MDAzMzk8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMtcmVzb3VyY2UtbnVt
Pmh0dHA6Ly9keC5kb2kub3JnLzEwLjEwMTYvai5qb2kuMjAwOS4wNC4wMDE8L2VsZWN0cm9uaWMt
cmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkdsw6RuemVsPC9BdXRo
b3I+PFllYXI+MjAwNjwvWWVhcj48UmVjTnVtPjI0NTQ8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt
YmVyPjI0NTQ8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJw
OTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjI0NTQ8L2tleT48L2ZvcmVpZ24t
a2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkdsw6RuemVsLCBXLjwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5PbiB0aGUgaC1pbmRleCAtIGEgbWF0aGVtYXRp
Y2FsIGFwcHJvYWNoIHRvIGEgbmV3IG1lYXN1cmUgb2YgcHVibGljYXRpb24gYWN0aXZpdHkgYW5k
IGNpdGF0aW9uIGltcGFjdDwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TY2llbnRvbWV0cmljczwv
c2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4zMTUtMzIxPC9wYWdlcz48dm9sdW1lPjY3
PC92b2x1bWU+PG51bWJlcj4yPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMDY8L3llYXI+PC9kYXRl
cz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Qm9ybm1hbjwvQXV0
aG9yPjxZZWFyPjIwMDU8L1llYXI+PFJlY051bT4yNDQ2PC9SZWNOdW0+PHJlY29yZD48cmVjLW51
bWJlcj4yNDQ2PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i
cDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4yNDQ2PC9rZXk+PC9mb3JlaWdu
LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250
cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Cb3JubWFuLCBMLjwvYXV0aG9yPjxhdXRob3I+RGFu
aWVsLCBILi1ELjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs
ZT5Eb2VzIHRoZSBoLWluZGV4IGZvciByYW5raW5nIG9mIHNjaWVudGlzdHMgcmVhbGx5IHdvcms/
PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNjaWVudG9tZXRyaWNzPC9zZWNvbmRhcnktdGl0bGU+
PC90aXRsZXM+PHBhZ2VzPjM5MS0zOTI8L3BhZ2VzPjx2b2x1bWU+NjU8L3ZvbHVtZT48bnVtYmVy
PjM8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwNTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48
L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Ob3JyaXM8L0F1dGhvcj48WWVhcj4yMDEwPC9Z
ZWFyPjxSZWNOdW0+NDgzNTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDgzNTwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0
c3JwYXZlY3dlMnIydzJhZGZlcCI+NDgzNTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+Tm9ycmlzLCBNLjwvYXV0aG9yPjxhdXRob3I+T3BwZW5oZWltLCBDLjwvYXV0aG9y
PjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UaGUgaOKAkGluZGV4OiBh
IGJyb2FkIHJldmlldyBvZiBhIG5ldyBiaWJsaW9tZXRyaWMgaW5kaWNhdG9ybnVsbDwvdGl0bGU+
PHNlY29uZGFyeS10aXRsZT5Kb3VybmFsIG9mIERvY3VtZW50YXRpb248L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+NjgxLTcwNTwvcGFnZXM+PHZvbHVtZT42Njwvdm9sdW1lPjxudW1i
ZXI+NTwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDEwPC95ZWFyPjxwdWItZGF0ZXM+PGRhdGU+MjAx
MC8wOS8wNzwvZGF0ZT48L3B1Yi1kYXRlcz48L2RhdGVzPjxwdWJsaXNoZXI+RW1lcmFsZDwvcHVi
bGlzaGVyPjxpc2JuPjAwMjItMDQxODwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0
cDovL2R4LmRvaS5vcmcvMTAuMTEwOC8wMDIyMDQxMTAxMTA2Njc5MDwvdXJsPjwvcmVsYXRlZC11
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+MTAuMTEwOC8wMDIyMDQxMTAxMTA2
Njc5MDwvZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+PGFjY2Vzcy1kYXRlPjIwMTUvMTEvMDU8L2Fj
Y2Vzcy1kYXRlPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE.DATA (Alonso et al., 2009; Bornman & Daniel, 2005; Costas & Bordons, 2007; Gl?nzel, 2006; Norris & Oppenheim, 2010) and for mathematical properties see Gl?nzel ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2006</Year><RecNum>2454</RecNum><DisplayText>(2006)</DisplayText><record><rec-number>2454</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2454</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gl?nzel, W.</author></authors></contributors><titles><title>On the h-index - a mathematical approach to a new measure of publication activity and citation impact</title><secondary-title>Scientometrics</secondary-title></titles><pages>315-321</pages><volume>67</volume><number>2</number><dates><year>2006</year></dates><urls></urls></record></Cite></EndNote>(2006) and Franceschini and Maisano ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2010</Year><RecNum>4455</RecNum><DisplayText>(2010)</DisplayText><record><rec-number>4455</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4455</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Franceschini, Fiorenzo</author><author>Maisano, Domenico A.</author></authors></contributors><titles><title>Analysis of the Hirsch index’s operational properties</title><secondary-title>European Journal of Operational Research</secondary-title></titles><periodical><full-title>European Journal of Operational Research</full-title></periodical><pages>494-504</pages><volume>203</volume><number>2</number><keywords><keyword>Hirsch index</keyword><keyword>Performance evaluation</keyword><keyword>Citations</keyword><keyword>Ranking</keyword><keyword>Indicator properties</keyword><keyword>Operational properties</keyword><keyword>Publication year</keyword></keywords><dates><year>2010</year></dates><isbn>0377-2217</isbn><urls><related-urls><url>;(2010).The h index is defined as: “a scientist has index h if h of his or her Np papers have at least h citations each and the other (Np – h) papers have <= h citations each” Hirsch ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2005</Year><RecNum>2418</RecNum><Suffix>`, p. 16569</Suffix><DisplayText>(2005, p. 16569)</DisplayText><record><rec-number>2418</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2418</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hirsch, J.</author></authors></contributors><titles><title>An index to quantify an individual's scientific research output</title><secondary-title>Proceedings of the National Academy of Sciences of the United States of America</secondary-title></titles><pages>16569-16572</pages><volume>102</volume><number>46</number><dates><year>2005</year></dates><urls></urls></record></Cite></EndNote>(2005, p. 16569). Thus h is the top h papers of a collection that all have at least h citations. The novel property of h is that in one number it summarises both impact, in terms of citations, and productivity in terms of number of papers. It thus lies somewhere between CPP, which ignores productivity, and total cites which is heavily dependent on productivity. The h papers are generally called the h-core. The h-index ignores all papers outside the h-core, and also ignores the actual number of citations received by the h-core papers. The strengths of the h-index are: It combines both productivity and impact in a single measure that is easily understood and very intuitive.It is easily calculated just knowing the number of citations either from WoS, Scopus or Google Scholar (GS). Indeed, all three now routinely calculate it.It can be applied at different levels – researcher, journal or department.It is objective and a good comparator within a discipline where citation rates are similar.It is robust to poor data since it ignores the lower down papers where the problems usually occur. This is particularly important if using GS.However, many limitations have been identified including some that affect all citation based measures (e.g., the problem of different scientific areas, and ensuring correctness of data), and a range of modifications have been suggested ADDIN EN.CITE <EndNote><Cite><Author>Bornmann</Author><Year>2008</Year><RecNum>3568</RecNum><DisplayText>(Bornmann et al., 2008)</DisplayText><record><rec-number>3568</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3568</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bornmann, L.</author><author>Mutz, R.</author><author>Daniel, H. D.</author></authors></contributors><titles><title>Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>830-837</pages><volume>59</volume><number>5</number><dates><year>2008</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(Bornmann et al., 2008). The metric is insensitive to the actual number of citations so two journals could have the same h-index but very different total citations. The g-index ADDIN EN.CITE <EndNote><Cite><Author>Egghe</Author><Year>2006</Year><RecNum>2450</RecNum><DisplayText>(Egghe, 2006)</DisplayText><record><rec-number>2450</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2450</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Egghe, L.</author></authors></contributors><titles><title>Theory and practice of the g-index</title><secondary-title>Scientometrics</secondary-title></titles><pages>131-152</pages><volume>69</volume><number>1</number><dates><year>2006</year></dates><urls></urls></record></Cite></EndNote>(Egghe, 2006) has been suggested as a way of compensating for this. The h-index is strictly increasing and strongly related to the time the publications have existed. This biases it against newer journals. It is possible to time-limit the h-index, for example Google Metrics uses a 5-year h-index ADDIN EN.CITE <EndNote><Cite><Author>Jin</Author><Year>2007</Year><RecNum>4453</RecNum><DisplayText>(Jin et al., 2007)</DisplayText><record><rec-number>4453</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4453</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Jin, B.</author><author>Liang, L.</author><author>Rousseau, R.</author><author>Egghe, L.</author></authors></contributors><titles><title>The R- and AR-indices: Complementing the h-index</title><secondary-title>Chinese Science Bulletin</secondary-title></titles><pages>855-863</pages><volume>52</volume><number>6</number><dates><year>2007</year></dates><urls></urls></record></Cite></EndNote>(Jin et al., 2007). The h-index is field dependent and so should be normalised in some way. Iglesias and Pecharroman ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2007</Year><RecNum>3571</RecNum><DisplayText>(2007)</DisplayText><record><rec-number>3571</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3571</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Iglesias, J.</author><author>Pecharromán, C.</author></authors></contributors><titles><title>Scaling the h-index for different scientific ISI fields</title><secondary-title>Scientometrics</secondary-title></titles><pages>303-320</pages><volume>73</volume><number>3</number><keywords><keyword>Computer Science</keyword></keywords><dates><year>2007</year></dates><publisher>Akadémiai Kiadó, co-published with Springer Science+Business Media B.V., Formerly Kluwer Academic Publishers B.V.</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(2007) constructed a table or normalisation factors for 21 different scientific fields. The h-index is dependent on or limited by the total number of publications and this is a disadvantage for journals which are highly cited but for a relatively small number of publications ADDIN EN.CITE <EndNote><Cite><Author>Costas</Author><Year>2007</Year><RecNum>2673</RecNum><DisplayText>(Costas & Bordons, 2007)</DisplayText><record><rec-number>2673</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2673</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Costas, R.</author><author>Bordons, M.</author></authors></contributors><titles><title>The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>193-203</pages><volume>1</volume><dates><year>2007</year></dates><urls></urls></record></Cite></EndNote>(Costas & Bordons, 2007). It will thus tend to favour journals that publish many papers against those with a small number of high quality papers. This can be seen clearly in the empirical results.There have been many comparisons of the h-index with other indicators PEVuZE5vdGU+PENpdGU+PEF1dGhvcj52YW4gUmFhbjwvQXV0aG9yPjxZZWFyPjIwMDU8L1llYXI+
PFJlY051bT4yNDQ3PC9SZWNOdW0+PERpc3BsYXlUZXh0PihCb3JubWFubiAmYW1wOyBEYW5pZWws
IDIwMDc7IExlaG1hbm4gZXQgYWwuLCAyMDA2OyB2YW4gUmFhbiwgMjAwNWEpPC9EaXNwbGF5VGV4
dD48cmVjb3JkPjxyZWMtbnVtYmVyPjI0NDc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjI0
NDc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPnZhbiBSYWFuLCBBLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Db21wYXJpc29u
IG9mIHRoZSBoaXJzY2gtaW5kZXggd2l0aCBzdGFuZGFyZCBiaWJsaW9tZXRyaWMgaW5kaWNhdG9y
cyBhbmQgd2l0aCBwZWVyIGp1ZGdlbWVudCBmb3IgMTQ3IGNoZW1pc3RyeSByZXNlYXJjaCBncm91
cHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NpZW50b21ldHJpY3M8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+NDkxLTUwMjwvcGFnZXM+PHZvbHVtZT42Nzwvdm9sdW1lPjxudW1i
ZXI+MzwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDA1PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxz
PjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkxlaG1hbm48L0F1dGhvcj48WWVhcj4yMDA2
PC9ZZWFyPjxSZWNOdW0+NDQ1ODwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQ1ODwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQ1ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+TGVobWFubiwgU3VuZTwvYXV0aG9yPjxhdXRob3I+SmFja3NvbiwgQW5kcmV3
IEQuPC9hdXRob3I+PGF1dGhvcj5MYXV0cnVwLCBCZW5ueSBFLjwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5NZWFzdXJlcyBmb3IgbWVhc3VyZXM8L3RpdGxl
PjxzZWNvbmRhcnktdGl0bGU+TmF0dXJlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2Vz
PjEwMDMtMTAwNDwvcGFnZXM+PHZvbHVtZT40NDQ8L3ZvbHVtZT48bnVtYmVyPjcxMjI8L251bWJl
cj48ZGF0ZXM+PHllYXI+MjAwNjwveWVhcj48L2RhdGVzPjxpc2JuPjAwMjgtMDgzNjwvaXNibj48
d29yay10eXBlPjEwLjEwMzgvNDQ0MTAwM2E8L3dvcmstdHlwZT48dXJscz48cmVsYXRlZC11cmxz
Pjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAzOC80NDQxMDAzYTwvdXJsPjwvcmVsYXRlZC11
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+aHR0cDovL3d3dy5uYXR1cmUuY29t
L25hdHVyZS9qb3VybmFsL3Y0NDQvbjcxMjIvc3VwcGluZm8vNDQ0MTAwM2FfUzEuaHRtbDwvZWxl
Y3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Qm9ybm1h
bm48L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxSZWNOdW0+NDQ2MDwvUmVjTnVtPjxyZWNvcmQ+
PHJlYy1udW1iZXI+NDQ2MDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIg
ZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQ2MDwva2V5Pjwv
Zm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlw
ZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Qm9ybm1hbm4sIEx1dHo8L2F1dGhvcj48
YXV0aG9yPkRhbmllbCwgSGFucy1EaWV0ZXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRv
cnM+PHRpdGxlcz48dGl0bGU+V2hhdCBkbyB3ZSBrbm93IGFib3V0IHRoZSBoIGluZGV4PzwvdGl0
bGU+PHNlY29uZGFyeS10aXRsZT5Kb3VybmFsIG9mIHRoZSBBbWVyaWNhbiBTb2NpZXR5IGZvciBJ
bmZvcm1hdGlvbiBTY2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRs
ZXM+PHBhZ2VzPjEzODEtMTM4NTwvcGFnZXM+PHZvbHVtZT41ODwvdm9sdW1lPjxudW1iZXI+OTwv
bnVtYmVyPjxkYXRlcz48eWVhcj4yMDA3PC95ZWFyPjwvZGF0ZXM+PHB1Ymxpc2hlcj5XaWxleSBT
dWJzY3JpcHRpb24gU2VydmljZXMsIEluYy4sIEEgV2lsZXkgQ29tcGFueTwvcHVibGlzaGVyPjxp
c2JuPjE1MzItMjg5MDwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRv
aS5vcmcvMTAuMTAwMi9hc2kuMjA2MDk8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0
cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwMDIvYXNpLjIwNjA5PC9lbGVjdHJvbmljLXJlc291cmNl
LW51bT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj52YW4gUmFhbjwvQXV0aG9yPjxZZWFyPjIwMDU8L1llYXI+
PFJlY051bT4yNDQ3PC9SZWNOdW0+PERpc3BsYXlUZXh0PihCb3JubWFubiAmYW1wOyBEYW5pZWws
IDIwMDc7IExlaG1hbm4gZXQgYWwuLCAyMDA2OyB2YW4gUmFhbiwgMjAwNWEpPC9EaXNwbGF5VGV4
dD48cmVjb3JkPjxyZWMtbnVtYmVyPjI0NDc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjI0
NDc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPnZhbiBSYWFuLCBBLjwv
YXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Db21wYXJpc29u
IG9mIHRoZSBoaXJzY2gtaW5kZXggd2l0aCBzdGFuZGFyZCBiaWJsaW9tZXRyaWMgaW5kaWNhdG9y
cyBhbmQgd2l0aCBwZWVyIGp1ZGdlbWVudCBmb3IgMTQ3IGNoZW1pc3RyeSByZXNlYXJjaCBncm91
cHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NpZW50b21ldHJpY3M8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+NDkxLTUwMjwvcGFnZXM+PHZvbHVtZT42Nzwvdm9sdW1lPjxudW1i
ZXI+MzwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDA1PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxz
PjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkxlaG1hbm48L0F1dGhvcj48WWVhcj4yMDA2
PC9ZZWFyPjxSZWNOdW0+NDQ1ODwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDQ1ODwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQ1ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+TGVobWFubiwgU3VuZTwvYXV0aG9yPjxhdXRob3I+SmFja3NvbiwgQW5kcmV3
IEQuPC9hdXRob3I+PGF1dGhvcj5MYXV0cnVwLCBCZW5ueSBFLjwvYXV0aG9yPjwvYXV0aG9ycz48
L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5NZWFzdXJlcyBmb3IgbWVhc3VyZXM8L3RpdGxl
PjxzZWNvbmRhcnktdGl0bGU+TmF0dXJlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2Vz
PjEwMDMtMTAwNDwvcGFnZXM+PHZvbHVtZT40NDQ8L3ZvbHVtZT48bnVtYmVyPjcxMjI8L251bWJl
cj48ZGF0ZXM+PHllYXI+MjAwNjwveWVhcj48L2RhdGVzPjxpc2JuPjAwMjgtMDgzNjwvaXNibj48
d29yay10eXBlPjEwLjEwMzgvNDQ0MTAwM2E8L3dvcmstdHlwZT48dXJscz48cmVsYXRlZC11cmxz
Pjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTAzOC80NDQxMDAzYTwvdXJsPjwvcmVsYXRlZC11
cmxzPjwvdXJscz48ZWxlY3Ryb25pYy1yZXNvdXJjZS1udW0+aHR0cDovL3d3dy5uYXR1cmUuY29t
L25hdHVyZS9qb3VybmFsL3Y0NDQvbjcxMjIvc3VwcGluZm8vNDQ0MTAwM2FfUzEuaHRtbDwvZWxl
Y3Ryb25pYy1yZXNvdXJjZS1udW0+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Qm9ybm1h
bm48L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxSZWNOdW0+NDQ2MDwvUmVjTnVtPjxyZWNvcmQ+
PHJlYy1udW1iZXI+NDQ2MDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIg
ZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+NDQ2MDwva2V5Pjwv
Zm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlw
ZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Qm9ybm1hbm4sIEx1dHo8L2F1dGhvcj48
YXV0aG9yPkRhbmllbCwgSGFucy1EaWV0ZXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRv
cnM+PHRpdGxlcz48dGl0bGU+V2hhdCBkbyB3ZSBrbm93IGFib3V0IHRoZSBoIGluZGV4PzwvdGl0
bGU+PHNlY29uZGFyeS10aXRsZT5Kb3VybmFsIG9mIHRoZSBBbWVyaWNhbiBTb2NpZXR5IGZvciBJ
bmZvcm1hdGlvbiBTY2llbmNlIGFuZCBUZWNobm9sb2d5PC9zZWNvbmRhcnktdGl0bGU+PC90aXRs
ZXM+PHBhZ2VzPjEzODEtMTM4NTwvcGFnZXM+PHZvbHVtZT41ODwvdm9sdW1lPjxudW1iZXI+OTwv
bnVtYmVyPjxkYXRlcz48eWVhcj4yMDA3PC95ZWFyPjwvZGF0ZXM+PHB1Ymxpc2hlcj5XaWxleSBT
dWJzY3JpcHRpb24gU2VydmljZXMsIEluYy4sIEEgV2lsZXkgQ29tcGFueTwvcHVibGlzaGVyPjxp
c2JuPjE1MzItMjg5MDwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRv
aS5vcmcvMTAuMTAwMi9hc2kuMjA2MDk8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0
cm9uaWMtcmVzb3VyY2UtbnVtPjEwLjEwMDIvYXNpLjIwNjA5PC9lbGVjdHJvbmljLXJlc291cmNl
LW51bT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (Bornmann & Daniel, 2007; Lehmann et al., 2006; van Raan, 2005a). Generally, such comparisons show that the h-index is highly correlated with other bibliometric indicators, but more so with measures of productivity such as number of papers and total number of citations, rather than with citations per paper which is more a measure of pure impact PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Db3N0YXM8L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxS
ZWNOdW0+MjY3MzwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQWxvbnNvIGV0IGFsLiwgMjAwOTsgQ29z
dGFzICZhbXA7IEJvcmRvbnMsIDIwMDc7IFRvZGVzY2hpbmksIDIwMTEpPC9EaXNwbGF5VGV4dD48
cmVjb3JkPjxyZWMtbnVtYmVyPjI2NzM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFw
cD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjI2NzM8
L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwv
cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkNvc3RhcywgUi48L2F1dGhv
cj48YXV0aG9yPkJvcmRvbnMsIE0uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPlRoZSBoLWluZGV4OiBBZHZhbnRhZ2VzLCBsaW1pdGF0aW9ucyBhbmQgaXRz
IHJlbGF0aW9uIHdpdGggb3RoZXIgYmlibGlvbWV0cmljIGluZGljYXRvcnMgYXQgdGhlIG1pY3Jv
IGxldmVsPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2YgSW5mb3JtZXRyaWNzPC9z
ZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE5My0yMDM8L3BhZ2VzPjx2b2x1bWU+MTwv
dm9sdW1lPjxkYXRlcz48eWVhcj4yMDA3PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlRvZGVzY2hpbmk8L0F1dGhvcj48WWVhcj4yMDExPC9Z
ZWFyPjxSZWNOdW0+MzY5MzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzY5MzwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0
c3JwYXZlY3dlMnIydzJhZGZlcCI+MzY5Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+VG9kZXNjaGluaSwgUi48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+VGhlIGotaW5kZXg6IGEgbmV3IGJpYmxpb21ldHJpYyBpbmRleCBhbmQg
bXVsaXR2YXJpYXRlIGNvbXBhcmlzb25zIGJldHdlZW4gb3RoZXIgYmlibGlvbWV0cmljIGluZGlj
ZXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NpZW50b21ldHJpY3M8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+NjIxLTYzOTwvcGFnZXM+PHZvbHVtZT44Nzwvdm9sdW1lPjxkYXRl
cz48eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48
Q2l0ZT48QXV0aG9yPkFsb25zbzwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT4zODE4
PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zODE4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFk
ZmVwIj4zODE4PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0
aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5BbG9uc28s
IFMuPC9hdXRob3I+PGF1dGhvcj5DYWJyZXJpem8sIEYuIEouPC9hdXRob3I+PGF1dGhvcj5IZXJy
ZXJhLVZpZWRtYSwgRS48L2F1dGhvcj48YXV0aG9yPkhlcnJlcmEsIEYuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPmgtSW5kZXg6IEEgcmV2aWV3IGZvY3Vz
ZWQgaW4gaXRzIHZhcmlhbnRzLCBjb21wdXRhdGlvbiBhbmQgc3RhbmRhcmRpemF0aW9uIGZvciBk
aWZmZXJlbnQgc2NpZW50aWZpYyBmaWVsZHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5h
bCBvZiBJbmZvcm1ldHJpY3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MjczLTI4
OTwvcGFnZXM+PHZvbHVtZT4zPC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGtleXdvcmRzPjxr
ZXl3b3JkPmgtSW5kZXg8L2tleXdvcmQ+PGtleXdvcmQ+QmlibGlvbWV0cmljIGluZGljYXRvcnM8
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjxpc2Ju
PjE3NTEtMTU3NzwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5zY2ll
bmNlZGlyZWN0LmNvbS9zY2llbmNlL2FydGljbGUvcGlpL1MxNzUxMTU3NzA5MDAwMzM5PC91cmw+
PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT5odHRwOi8vZHgu
ZG9pLm9yZy8xMC4xMDE2L2ouam9pLjIwMDkuMDQuMDAxPC9lbGVjdHJvbmljLXJlc291cmNlLW51
bT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Db3N0YXM8L0F1dGhvcj48WWVhcj4yMDA3PC9ZZWFyPjxS
ZWNOdW0+MjY3MzwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oQWxvbnNvIGV0IGFsLiwgMjAwOTsgQ29z
dGFzICZhbXA7IEJvcmRvbnMsIDIwMDc7IFRvZGVzY2hpbmksIDIwMTEpPC9EaXNwbGF5VGV4dD48
cmVjb3JkPjxyZWMtbnVtYmVyPjI2NzM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFw
cD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjI2NzM8
L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwv
cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkNvc3RhcywgUi48L2F1dGhv
cj48YXV0aG9yPkJvcmRvbnMsIE0uPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0
aXRsZXM+PHRpdGxlPlRoZSBoLWluZGV4OiBBZHZhbnRhZ2VzLCBsaW1pdGF0aW9ucyBhbmQgaXRz
IHJlbGF0aW9uIHdpdGggb3RoZXIgYmlibGlvbWV0cmljIGluZGljYXRvcnMgYXQgdGhlIG1pY3Jv
IGxldmVsPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2YgSW5mb3JtZXRyaWNzPC9z
ZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjE5My0yMDM8L3BhZ2VzPjx2b2x1bWU+MTwv
dm9sdW1lPjxkYXRlcz48eWVhcj4yMDA3PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlRvZGVzY2hpbmk8L0F1dGhvcj48WWVhcj4yMDExPC9Z
ZWFyPjxSZWNOdW0+MzY5MzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzY5MzwvcmVjLW51
bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0
c3JwYXZlY3dlMnIydzJhZGZlcCI+MzY5Mzwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu
YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz
PjxhdXRob3I+VG9kZXNjaGluaSwgUi48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+VGhlIGotaW5kZXg6IGEgbmV3IGJpYmxpb21ldHJpYyBpbmRleCBhbmQg
bXVsaXR2YXJpYXRlIGNvbXBhcmlzb25zIGJldHdlZW4gb3RoZXIgYmlibGlvbWV0cmljIGluZGlj
ZXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NpZW50b21ldHJpY3M8L3NlY29uZGFyeS10aXRs
ZT48L3RpdGxlcz48cGFnZXM+NjIxLTYzOTwvcGFnZXM+PHZvbHVtZT44Nzwvdm9sdW1lPjxkYXRl
cz48eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48
Q2l0ZT48QXV0aG9yPkFsb25zbzwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT4zODE4
PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4zODE4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl
eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFk
ZmVwIj4zODE4PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0
aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5BbG9uc28s
IFMuPC9hdXRob3I+PGF1dGhvcj5DYWJyZXJpem8sIEYuIEouPC9hdXRob3I+PGF1dGhvcj5IZXJy
ZXJhLVZpZWRtYSwgRS48L2F1dGhvcj48YXV0aG9yPkhlcnJlcmEsIEYuPC9hdXRob3I+PC9hdXRo
b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPmgtSW5kZXg6IEEgcmV2aWV3IGZvY3Vz
ZWQgaW4gaXRzIHZhcmlhbnRzLCBjb21wdXRhdGlvbiBhbmQgc3RhbmRhcmRpemF0aW9uIGZvciBk
aWZmZXJlbnQgc2NpZW50aWZpYyBmaWVsZHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5h
bCBvZiBJbmZvcm1ldHJpY3M8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+MjczLTI4
OTwvcGFnZXM+PHZvbHVtZT4zPC92b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGtleXdvcmRzPjxr
ZXl3b3JkPmgtSW5kZXg8L2tleXdvcmQ+PGtleXdvcmQ+QmlibGlvbWV0cmljIGluZGljYXRvcnM8
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjxpc2Ju
PjE3NTEtMTU3NzwvaXNibj48dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3d3dy5zY2ll
bmNlZGlyZWN0LmNvbS9zY2llbmNlL2FydGljbGUvcGlpL1MxNzUxMTU3NzA5MDAwMzM5PC91cmw+
PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51bT5odHRwOi8vZHgu
ZG9pLm9yZy8xMC4xMDE2L2ouam9pLjIwMDkuMDQuMDAxPC9lbGVjdHJvbmljLXJlc291cmNlLW51
bT48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Alonso et al., 2009; Costas & Bordons, 2007; Todeschini, 2011). There have been several studies of the use of the h-index in business and management fields such as information systems ADDIN EN.CITE <EndNote><Cite><Author>Oppenheim</Author><Year>2007</Year><RecNum>2420</RecNum><DisplayText>(Oppenheim, 2007; Truex III et al., 2009)</DisplayText><record><rec-number>2420</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2420</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Oppenheim, C.</author></authors></contributors><titles><title>Using the h-index to rank influential British researchers in Information Science and Librarianship</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>297-301</pages><volume>58</volume><number>2</number><dates><year>2007</year></dates><urls></urls></record></Cite><Cite><Author>Truex III</Author><Year>2009</Year><RecNum>4461</RecNum><record><rec-number>4461</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4461</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Truex III, D.</author><author>Cuellar, M.</author><author>Takeda, H.</author></authors></contributors><titles><title>Assessing scholarly influence: Using the Hirsch indices to reframe the discourse</title><secondary-title>Journal of the Association for Information Systems</secondary-title></titles><pages>560-594</pages><volume>10</volume><number>7</number><dates><year>2009</year></dates><urls><related-urls><url>;(Oppenheim, 2007; Truex III et al., 2009), management science ADDIN EN.CITE <EndNote><Cite><Author>Mingers</Author><Year>2008</Year><RecNum>2527</RecNum><DisplayText>(Mingers, 2008; Mingers et al., 2012)</DisplayText><record><rec-number>2527</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2527</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author></authors></contributors><titles><title>Measuring the research contribution of management academics using the Hirsch-index</title><secondary-title>Journal Operational Research Society</secondary-title></titles><pages>1143-1153</pages><volume>60</volume><number>8</number><dates><year>2008</year></dates><urls></urls></record></Cite><Cite><Author>Mingers</Author><Year>2012</Year><RecNum>3773</RecNum><record><rec-number>3773</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3773</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Macri, F.</author><author>Petrovici, D.</author></authors></contributors><titles><title>Using the h-index to measure the quality of journals in the field of Business and Management</title><secondary-title>Information Processing & Management</secondary-title></titles><pages>234-241</pages><volume>48</volume><number>2</number><keywords><keyword>Citations</keyword><keyword>Google Scholar</keyword><keyword>h-Index</keyword><keyword>Impact factor</keyword><keyword>Journal quality</keyword><keyword>Research impact</keyword><keyword>Web of Science</keyword></keywords><dates><year>2012</year></dates><isbn>0306-4573</isbn><urls><related-urls><url>;(Mingers, 2008; Mingers et al., 2012), consumer research ADDIN EN.CITE <EndNote><Cite><Author>Saad</Author><Year>2006</Year><RecNum>2460</RecNum><DisplayText>(Saad, 2006)</DisplayText><record><rec-number>2460</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2460</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Saad, G.</author></authors></contributors><titles><title>Exploring the h-index at the author and journal levels using bibliometric data of productive consumer scholars and business-related journals respectively</title><secondary-title>Scientometrics</secondary-title></titles><pages>117-120</pages><volume>69</volume><number>1</number><dates><year>2006</year></dates><urls></urls></record></Cite></EndNote>(Saad, 2006), marketing ADDIN EN.CITE <EndNote><Cite><Author>Moussa</Author><Year>2010</Year><RecNum>3650</RecNum><DisplayText>(Moussa & Touzani, 2010)</DisplayText><record><rec-number>3650</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3650</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Moussa, S.</author><author>Touzani, M.</author></authors></contributors><titles><title>Ranking marketing journals using the Google Scholar-based hg-index</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>107-117</pages><volume>4</volume><dates><year>2010</year></dates><urls></urls></record></Cite></EndNote>(Moussa & Touzani, 2010) and business ADDIN EN.CITE <EndNote><Cite><Author>Harzing</Author><Year>2009</Year><RecNum>3466</RecNum><DisplayText>(Harzing & Van der Wal, 2009)</DisplayText><record><rec-number>3466</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3466</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Harzing, A.-W.</author><author>Van der Wal, R.</author></authors></contributors><titles><title>A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business?</title><secondary-title>J. Am. Soc. for Information Science and Technology</secondary-title></titles><pages>41-46</pages><volume>60</volume><number>1</number><dates><year>2009</year></dates><urls></urls></record></Cite></EndNote>(Harzing & Van der Wal, 2009). NormalisationOne of the main principles of bibliometric analysis is that citations indicators from different academic fields should not be compared directly with one another because of the major differences in citation density across fields. It is also desirable to consider differences in publication type, for example, journals with large number of review papers, which are highly cited, or editorials or book reviews which generate citations but which might not be counted as papers. In this paper we will discuss three approaches to normalisation – field normalisation, percentile normalisation and citing-side or source normalisation – for empirical analysis see Waltman and Marx ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2015</Year><RecNum>4843</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>4843</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4843</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bornmann, L.</author><author>Marx, W.</author></authors></contributors><titles><title>Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgement of experts?</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>408-418</pages><volume>9</volume><number>2</number><keywords><keyword>Computer Science</keyword></keywords><dates><year>2015</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(2015) and Waltman and van Eck ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2013</Year><RecNum>4288</RecNum><DisplayText>(2013)</DisplayText><record><rec-number>4288</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4288</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Waltman, L.</author><author>van Eck, N.</author></authors></contributors><titles><title>A systematic empirical comparison of different approaches for normalizing citation impact indicators</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>833-849</pages><volume>7</volume><number>4</number><keywords><keyword>Computer Science</keyword></keywords><dates><year>2013</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(2013).Field normalisationField normalisation means comparing the number of citations for a paper or journal, whether in absolute or average form, with the expected number of citations within the appropriate research field. For example, van Leeuwen and Moed ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2002</Year><RecNum>4841</RecNum><DisplayText>(2002)</DisplayText><record><rec-number>4841</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4841</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>van Leeuwen, T.</author><author>Moed, H.</author></authors></contributors><titles><title>Development and application of journal impact measures in the Dutch science system</title><secondary-title>Scientometrics</secondary-title><alt-title>Scientometrics</alt-title></titles><pages>249-266</pages><volume>53</volume><number>2</number><dates><year>2002</year><pub-dates><date>2002/02/01</date></pub-dates></dates><publisher>Kluwer Academic Publishers</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(2002) developed a citation impact indicator that normalises for field, publication year, and document type. This works by comparing the number of citations received by a paper with the mean number of citations of similar papers across all journals in the field. The main problem is determining an appropriate field, and corresponding journals, for each paper. This is generally implemented within WoS and the WoS field lists are used. This approach forms the basis of the well-established methodology for evaluating research centres developed by the Centre for Science and Technology Studies (CWTS) at Leiden University known as the crown indicator or Leiden Ranking Methodology (LRM) ADDIN EN.CITE <EndNote><Cite><Author>van Raan</Author><Year>2005</Year><RecNum>3694</RecNum><DisplayText>(Moed, 2010c; van Raan, 2005c)</DisplayText><record><rec-number>3694</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3694</key></foreign-keys><ref-type name="Book Section">5</ref-type><contributors><authors><author>van Raan, A.</author></authors><secondary-authors><author>Moed, H.</author><author>Glenzel, W.</author><author>Schmoch, U.</author></secondary-authors></contributors><titles><title>Measuring science: Capita selectaof current main issues</title><secondary-title>Handbook of Quantitative Science and Technology Research</secondary-title></titles><volume>19-50</volume><dates><year>2005</year></dates><pub-location>New York</pub-location><publisher>Springer</publisher><urls></urls></record></Cite><Cite><Author>Moed</Author><Year>2010</Year><RecNum>4842</RecNum><record><rec-number>4842</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4842</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Moed, Henk F</author></authors></contributors><titles><title>CWTS crown indicator measures citation impact of a research group's publication oeuvre</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>436-438</pages><volume>4</volume><number>3</number><dates><year>2010</year></dates><urls></urls></record></Cite></EndNote>(Moed, 2010c; van Raan, 2005c). The problems with this approach are that the WoS field categories are ad hoc, with no systemic basis ADDIN EN.CITE <EndNote><Cite><Author>Mingers</Author><Year>2015</Year><RecNum>4448</RecNum><DisplayText>(Leydesdorff & Bornmann, 2014; Mingers & Leydesdorff, 2015a)</DisplayText><record><rec-number>4448</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4448</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Leydesdorff, L.</author></authors></contributors><titles><title>Identifying research fields within business and management: A journal cross-citation analysis</title><secondary-title>Journal of the Operational Research Society</secondary-title></titles><periodical><full-title>Journal of the Operational Research Society</full-title></periodical><pages>1370-1384</pages><volume>66</volume><number>8</number><edition>13/10/2014</edition><dates><year>2015</year></dates><urls></urls><electronic-resource-num> 10.1057/jors.2014.113</electronic-resource-num></record></Cite><Cite><Author>Leydesdorff</Author><Year>2014</Year><RecNum>4487</RecNum><record><rec-number>4487</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4487</key></foreign-keys><ref-type name="Electronic Article">43</ref-type><contributors><authors><author>Leydesdorff, Loet</author><author>Bornmann, Lutz</author></authors></contributors><titles><title>The Operationalization of "Fields" as WoS Subject Categories (WCs) in Evaluative Bibliometrics: The cases of "Library and Information Science" and "Science & Technology Studies"</title><secondary-title>arXiv preprint arXiv:1407.7849</secondary-title></titles><num-vols>1407.7849</num-vols><dates><year>2014</year></dates><publisher>arXiv</publisher><urls><related-urls><url>;(Leydesdorff & Bornmann, 2014; Mingers & Leydesdorff, 2015a) and that it is difficult to cope with inter-disciplinary papers or journals ADDIN EN.CITE <EndNote><Cite><Author>Rafols</Author><Year>2012</Year><RecNum>3823</RecNum><DisplayText>(Rafols et al., 2012)</DisplayText><record><rec-number>3823</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3823</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Rafols, I.</author><author>Leydesdorff, L.</author><author>O’Hare, A.</author><author>Nightingale, P.</author><author>Stirling, A. </author></authors></contributors><titles><title>How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business & management</title><secondary-title>Research Policy</secondary-title></titles><pages>1262-1282</pages><volume>41</volume><number>7</number><dates><year>2012</year></dates><urls></urls></record></Cite></EndNote>(Rafols et al., 2012) .This form of normalisation is particularly difficult with Google Scholar data as there are no field lists provided in GS. One attempt at normalising GS data has been made ADDIN EN.CITE <EndNote><Cite><Author>Bornmann</Author><Year>2016</Year><RecNum>5099</RecNum><DisplayText>(Bornmann et al., 2016)</DisplayText><record><rec-number>5099</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">5099</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bornmann, Lutz</author><author>Thor, Andreas</author><author>Marx, Werner</author><author>Schier, Hermann</author></authors></contributors><titles><title>The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute</title><secondary-title>Journal of the Association for Information Science and Technology</secondary-title></titles><pages>n/a-n/a</pages><keywords><keyword>bibliometrics</keyword></keywords><dates><year>2016</year></dates><isbn>2330-1643</isbn><urls><related-urls><url>;(Bornmann et al., 2016) but it is very time-consuming and messy, and the results are not that reliable.Citing-side or source normalisationAn alternative method, originally suggested by Zitt and Small ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2008</Year><RecNum>4289</RecNum><DisplayText>(2008)</DisplayText><record><rec-number>4289</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4289</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Zitt, Michel</author><author>Small, Henry</author></authors></contributors><titles><title>Modifying the journal impact factor by fractional citation weighting: The audience factor</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>1856-1860</pages><volume>59</volume><number>11</number><dates><year>2008</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(2008) in their “audience factor”, is to consider the source of citations – that is the reference lists of citing papers. The assumption is that high density fields will have large reference lists and low density fields short ones. This approach is also known as “citing-side approach” ADDIN EN.CITE <EndNote><Cite><Author>Zitt</Author><Year>2011</Year><RecNum>4291</RecNum><DisplayText>(Zitt, 2011)</DisplayText><record><rec-number>4291</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4291</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Zitt, M.</author></authors></contributors><titles><title>Behind citing-side normalization of citations: some properties of the journal impact factor</title><secondary-title>Scientometrics</secondary-title><alt-title>Scientometrics</alt-title></titles><pages>329-344</pages><volume>89</volume><number>1</number><keywords><keyword>Impact factor</keyword><keyword>Citation</keyword><keyword>Citation normalization</keyword><keyword>Citing-side normalization</keyword><keyword>Source-level normalization</keyword></keywords><dates><year>2011</year><pub-dates><date>2011/10/01</date></pub-dates></dates><publisher>Springer Netherlands</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(Zitt, 2011), fractional counting of citations ADDIN EN.CITE <EndNote><Cite><Author>Leydesdorff</Author><Year>2010</Year><RecNum>4274</RecNum><DisplayText>(Leydesdorff & Opthof, 2010)</DisplayText><record><rec-number>4274</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4274</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Leydesdorff, Loet</author><author>Opthof, Tobias</author></authors></contributors><titles><title>Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>2365-2369</pages><volume>61</volume><number>11</number><dates><year>2010</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(Leydesdorff & Opthof, 2010) and a priori normalisation ADDIN EN.CITE <EndNote><Cite><Author>Gl?nzel</Author><Year>2011</Year><RecNum>4485</RecNum><DisplayText>(Gl?nzel et al., 2011)</DisplayText><record><rec-number>4485</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4485</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gl?nzel, Wolfgang</author><author>Schubert, András</author><author>Thijs, Bart</author><author>Debackere, Koenraad</author></authors></contributors><titles><title>A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking</title><secondary-title>Scientometrics</secondary-title></titles><pages>415-424</pages><volume>87</volume><number>2</number><dates><year>2011</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(Gl?nzel et al., 2011). It is the basis of the SNIP metric to be discussed later ADDIN EN.CITE <EndNote><Cite><Author>Moed</Author><Year>2010</Year><RecNum>4275</RecNum><DisplayText>(Moed, 2010b)</DisplayText><record><rec-number>4275</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4275</key></foreign-keys><ref-type name="Electronic Article">43</ref-type><contributors><authors><author>Moed, H.</author></authors></contributors><titles><title>The Source-Normalized Impact per Paper (SNIP) is a valid and sophisticated indicator of journal citation impact</title><secondary-title>arXiv preprint</secondary-title></titles><num-vols>arXiv:1005.4906, 2010</num-vols><dates><year>2010</year></dates><publisher></publisher><urls><related-urls><url>;(Moed, 2010b).This approach is different in that the reference set of journals is not defined in advance according to a WoS category, but is rather determined at the time as the set of all papers or journals that have cited the journal in question. Each evaluated journal will therefore have its own specific set of citing journals thus avoiding problems with outdated and ad hoc WoS categories. The disadvantage is that the journals are not being compared against the same benchmark set.Second generation indicatorsIn recent years several new, and more complex, indicators have been developed to take into account concerns about normalisation and the relative prestige of citing journals. Some of these indicators are specific to particular data sources, e.g., the Eigenfactor in WoS, and SNIP and SJR in ScopusIndicators measuring the prestige of citations: Eigenfactor and SJRThe idea of these indicators is that having a paper cited in a very high-quality or prestigious journal such as Nature or Science is worth more than a citation in an obscure journal. The indicators all work on a recursive algorithm similar to that of Google’s PageRank for web pages. The first such was developed by Pinsky and Narin ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>1976</Year><RecNum>4473</RecNum><DisplayText>(1976)</DisplayText><record><rec-number>4473</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4473</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Pinski, Gabriel</author><author>Narin, Francis</author></authors></contributors><titles><title>Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics</title><secondary-title>Information Processing & Management</secondary-title></titles><pages>297-312</pages><volume>12</volume><number>5</number><dates><year>1976</year></dates><isbn>0306-4573</isbn><urls></urls></record></Cite></EndNote>(1976) but that had calculation problems. Since then, Page et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>1999</Year><RecNum>4474</RecNum><DisplayText>(1999)</DisplayText><record><rec-number>4474</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4474</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Page, Lawrence</author><author>Brin, Sergey</author><author>Motwani, Rajeev</author><author>Winograd, Terry</author></authors></contributors><titles><title>The PageRank citation ranking: Bringing order to the web</title></titles><dates><year>1999</year></dates><urls></urls></record></Cite></EndNote>(1999) and Ma ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2008</Year><RecNum>2674</RecNum><DisplayText>(2008)</DisplayText><record><rec-number>2674</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">2674</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Ma, N.</author><author>Guan, J.</author><author>Zhao, Y.</author></authors></contributors><titles><title>Bringing PageRank to the citation analysis</title><secondary-title>Information Processing and Management</secondary-title></titles><pages>800-810</pages><volume>44</volume><dates><year>2008</year></dates><urls></urls></record></Cite></EndNote>(2008) have an algorithm based directly on PageRank but adapted to citations; Bergstrom ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2007</Year><RecNum>4475</RecNum><DisplayText>(2007)</DisplayText><record><rec-number>4475</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4475</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bergstrom, Carl</author></authors></contributors><titles><title>Measuring the value and prestige of scholarly journals</title><secondary-title>College and Research Library News</secondary-title></titles><periodical><full-title>College and Research Library News</full-title></periodical><pages>3146</pages><volume>68</volume><number>5</number><dates><year>2007</year></dates><urls></urls></record></Cite></EndNote>(2007) has developed the Eigenfactor which is implemented in WoS; and Gonzalez-Pereira et al ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2010</Year><RecNum>4444</RecNum><DisplayText>(2010)</DisplayText><record><rec-number>4444</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4444</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>González-Pereira, Borja</author><author>Guerrero-Bote, Vicente P.</author><author>Moya-Anegón, Félix</author></authors></contributors><titles><title>A new approach to the metric of journals’ scientific prestige: The SJR indicator</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>379-391</pages><volume>4</volume><number>3</number><keywords><keyword>SJR indicator</keyword><keyword>Academic journals</keyword><keyword>Journal prestige</keyword><keyword>Eigenvector centrality</keyword><keyword>Citation networks</keyword></keywords><dates><year>2010</year></dates><isbn>1751-1577</isbn><urls><related-urls><url>;(2010) have developed SCImago Journal Rank (SJR) which is implemented in Scopus. We will use the latter two. The Eigenfactor essentially measures the relative frequency of occurrence of each journal in the network of citations, and uses this as a measure of prestige. It explicitly excludes journal self-citations unlike most other indicators. Its values tend to be very small, for example the largest in the management field is Management Science with a value of 0.03 while the 20th is 0.008, figures which are not easily interpreted. The Eigenfactor is based on the total number of citations and so is affected by the total number of papers published by a journal. A related metric, also in WoS, is the Article Influence Score (AIS) which is the Eigenfactor divided by the proportion of papers in the database belonging to a particular journal over five years. This is therefore similar to a 5-year JIF but normalised so that a value of 1.0 shows that the journal has average influence; values greater than 1.0 show greater influence. The SJR works iteratively in a similar way to the Eigenfactor but its value is normalised by the total number of citations in the citing journal for the year in question. It works in two stages: firstly calculating an un-normalised value iteratively based on three components – a fixed amount for being included in Scopus, a value dependent on the number of citations received, and the prestige of the sources. There are a number of seemingly-arbitrary weightings in the formula. This value is then normalised by the number of published articles and adjusted to give an “easy-to-use” value. The currently implemented version of SJR in Scopus has a further refinement ADDIN EN.CITE <EndNote><Cite><Author>Guerrero-Bote</Author><Year>2012</Year><RecNum>4445</RecNum><DisplayText>(Guerrero-Bote & Moya-Anegón, 2012)</DisplayText><record><rec-number>4445</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4445</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Guerrero-Bote, Vicente P.</author><author>Moya-Anegón, Félix</author></authors></contributors><titles><title>A further step forward in measuring journals’ scientific prestige: The SJR2 indicator</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>674-688</pages><volume>6</volume><number>4</number><keywords><keyword>SJR2 indicator</keyword><keyword>Academic journals</keyword><keyword>Journal prestige</keyword><keyword>Eigenvector centrality</keyword><keyword>Citation networks</keyword></keywords><dates><year>2012</year></dates><isbn>1751-1577</isbn><urls><related-urls><url>;(Guerrero-Bote & Moya-Anegón, 2012) in that the “relatedness” of the citing journal is also taken into account. A major problem with this metric (and with SNIP) is that its results are not reproducible outside of its actual production, for example by other researchers.There are several limitations of these 2nd generation measures: the values for “prestige” are difficult to interpret as they are not a mean citation value but only make sense in comparison with others; they are still not normalised for field ADDIN EN.CITE <EndNote><Cite><Author>Lancho-Barrantes</Author><Year>2010</Year><RecNum>4478</RecNum><DisplayText>(Lancho-Barrantes et al., 2010)</DisplayText><record><rec-number>4478</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4478</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Lancho-Barrantes, Bárbara S</author><author>Guerrero-Bote, Vicente P</author><author>Moya-Anegón, Félix</author></authors></contributors><titles><title>What lies behind the averages and significance of citation indicators in different disciplines?</title><secondary-title>Journal of Information Science</secondary-title></titles><pages>371-382</pages><volume>36</volume><number>3</number><dates><year>2010</year></dates><isbn>0165-5515</isbn><urls></urls></record></Cite></EndNote>(Lancho-Barrantes et al., 2010); and the fields themselves are open to disagreement ADDIN EN.CITE <EndNote><Cite><Author>Mingers</Author><Year>2015</Year><RecNum>4448</RecNum><DisplayText>(Mingers & Leydesdorff, 2015a)</DisplayText><record><rec-number>4448</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4448</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Leydesdorff, L.</author></authors></contributors><titles><title>Identifying research fields within business and management: A journal cross-citation analysis</title><secondary-title>Journal of the Operational Research Society</secondary-title></titles><periodical><full-title>Journal of the Operational Research Society</full-title></periodical><pages>1370-1384</pages><volume>66</volume><number>8</number><edition>13/10/2014</edition><dates><year>2015</year></dates><urls></urls><electronic-resource-num> 10.1057/jors.2014.113</electronic-resource-num></record></Cite></EndNote>(Mingers & Leydesdorff, 2015a).Source-normalised indicator: SNIPAnother 2nd generation metric is SNIP ADDIN EN.CITE <EndNote><Cite><Author>Moed</Author><Year>2010</Year><RecNum>4276</RecNum><DisplayText>(Moed, 2010a)</DisplayText><record><rec-number>4276</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4276</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Moed, H.</author></authors></contributors><titles><title>Measuring contextual citation impact of scientific journals</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>265-277</pages><volume>4</volume><number>3</number><dates><year>2010</year></dates><urls></urls></record></Cite></EndNote>(Moed, 2010a) – source normalised impact per paper - which is only available within Scopus. This normalises for different fields using the citing-side form of normalisation. It firstly calculates a 3-year IPP (effectively a 3-year JIF). It then calculates the “database citation potential” DCP for the particular journal by finding all the papers in year n that cite papers from the journal in the preceding ten years and calculating the mean of the number of references in those papers to papers within the database – i.e., Scopus. Next, the DCP for all journals in the database is calculated and the median of these values noted. The DCP for the journal is then divided by the median to relativise it to journals as a whole creating a relative DCP (RDCP). If this value is above 1 then the field has greater citation potential; if it is less than 1 the field has lower citation potential. Finally, SNIP = IPP/RDCP. If the field is high density then RDCP will be above 1 and the IPP will be reduced and vice versa if the field is low density. The currently implemented version of SNIP has two changes ADDIN EN.CITE <EndNote><Cite><Author>Waltman</Author><Year>2013</Year><RecNum>4278</RecNum><DisplayText>(Waltman et al., 2013)</DisplayText><record><rec-number>4278</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4278</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Waltman, L.</author><author>van Eck, N.</author><author>van Leeuwen, T.</author><author>Visser, M.</author></authors></contributors><titles><title>Some modifications to the SNIP journal impact indicator</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>272-285</pages><volume>7</volume><number>2</number><keywords><keyword>Computer Science</keyword></keywords><dates><year>2013</year></dates><publisher>.</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(Waltman et al., 2013): the DCP is calculated using the harmonic mean rather than the arithmetic mean, and the relativisation of the DCP is now dropped. This is an innovative measure both because it normalises for both number of publications and field, and because the set of reference journals are specific to each journal rather than being defined beforehand somewhat arbitrarily. Moed ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2010</Year><RecNum>4276</RecNum><DisplayText>(2010a)</DisplayText><record><rec-number>4276</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4276</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Moed, H.</author></authors></contributors><titles><title>Measuring contextual citation impact of scientific journals</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>265-277</pages><volume>4</volume><number>3</number><dates><year>2010</year></dates><urls></urls></record></Cite></EndNote>(2010a) presents empirical evidence from the sciences that the subject normalisation does work even at the level of pairs of journals in the same field. But, it is complex and rather opaque and criticisms have been levelled by Leydesdorff and Opthof ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2010</Year><RecNum>4274</RecNum><DisplayText>(2010)</DisplayText><record><rec-number>4274</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4274</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Leydesdorff, Loet</author><author>Opthof, Tobias</author></authors></contributors><titles><title>Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>2365-2369</pages><volume>61</volume><number>11</number><dates><year>2010</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(2010) and Mingers ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2014</Year><RecNum>4294</RecNum><DisplayText>(2014)</DisplayText><record><rec-number>4294</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4294</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author></authors></contributors><titles><title>Problems with SNIP</title><secondary-title>Journal of Informetrics</secondary-title></titles><pages>890-894</pages><volume>8</volume><number>4</number><dates><year>2014</year></dates><urls></urls><electronic-resource-num>10.1016/j.joi.2014.09.004</electronic-resource-num></record></Cite></EndNote>(2014).Percentile-based indicator – I3This approach aims to overcome the statistical problems of using means with highly skewed data. This uses WoS field categories to establish percentile ranks (PR) in terms of the number of citations necessary for a paper to be in the top 1%, 5%, 10% … of papers published in the field ADDIN EN.CITE <EndNote><Cite><Author>Leydesdorff</Author><Year>2012</Year><RecNum>4490</RecNum><DisplayText>(Leydesdorff, 2012)</DisplayText><record><rec-number>4490</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4490</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Leydesdorff, L.</author></authors></contributors><titles><title>Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most highly cited papers</title><secondary-title>Scientometrics</secondary-title></titles><pages>355-365</pages><volume>92</volume><dates><year>2012</year></dates><urls></urls></record></Cite></EndNote>(Leydesdorff, 2012). The target set of papers for a journal are then all evaluated to see which PR they fall into and the proportions falling into each one are calculated. These can then be compared so that, for example, a journal with 5% of its papers having more citations than the top 1% in its field is above average.Based on this form of normalisation, a metric has been developed as an alternative to the journal impact factor (JIF) called I3 ADDIN EN.CITE <EndNote><Cite><Author>Leydesdorff</Author><Year>2011</Year><RecNum>4492</RecNum><DisplayText>(Leydesdorff & Bornmann, 2011)</DisplayText><record><rec-number>4492</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4492</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Leydesdorff, Loet</author><author>Bornmann, Lutz</author></authors></contributors><titles><title>Integrated impact indicators compared with impact factors: An alternative research design with policy implications</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>2133-2146</pages><volume>62</volume><number>11</number><dates><year>2011</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(Leydesdorff & Bornmann, 2011). Instead of multiplying the percentile ranks by the proportion of papers in each class, they are multiplied by the actual numbers of papers in each class thus giving a measure that combines productivity with citation impact. This indicator is not available in any of the databases and so we will not be able to include it in the empirical investigation.Table 1 summarises the main characteristics and the advantages and disadvantages of the indicators. It also shows typical values for a high density field (cell biology) and the management field. Methodology and DataWe wished to compare the various indicators empirically on a sample of business and management journals and then compare the results with the ABS journal ranking. One of the problems is that the indicators are not all available from the same source – the JIF and Eigenfactor come from the WoS; the h-index from the Scimago website; and the SJR and SNIP from Scopus. Clearly this introduces problem of consistency as the databases do not cover the same set of journals and therefore have differences in the numbers of citations ADDIN EN.CITE <EndNote><Cite><Author>Leydesdorff</Author><Year>2014</Year><RecNum>5097</RecNum><DisplayText>(Leydesdorff et al., 2014)</DisplayText><record><rec-number>5097</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">5097</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Leydesdorff, Loet</author><author>de Moya-Anegón, Félix</author><author>de Nooy, Wouter</author></authors></contributors><titles><title>Aggregated journal-journal citation relations in Scopus and Web-of-Science matched and compared in terms of networks, maps, and interactive overlays</title><secondary-title>arXiv preprint arXiv:1404.2505</secondary-title></titles><dates><year>2014</year></dates><urls></urls></record></Cite></EndNote>(Leydesdorff et al., 2014). However, from a practical viewpoint in terms of using these indicators either by themselves or as part of creating a journal list such as ABS, we have to accept what is available and so it is the data as it stands that we have analysed.Data was collected from these three sources for the years 2012 and 2013. The analysis we present is based on 2013 but checks with 2012 showed no major inconsistencies. The data was validated, especially in terms of ensuring consistency of journal title and ISSN, and a small number of outliers were removed. The full dataset contains the following variables – note that there are many less journals classified as business and management in WoS. This meant that the statistical analyses were often restricted by the smaller WoS size.MetricDescriptionAdvantagesDisadvantagesMaximum values for:a) cell biologyb) managementNormalises for:No of papersFieldPrestigeImpact factor (JIF) (WoS)Mean citations per paper over a 2 or 5 year window. Normalised to number of papers. Counts citations equallyWell-known, easy to calculate and understand.Not normalised to discipline; short time span; concerns about data and manipulationFrom WoSa) 36.5b) 7.8YNNEigenfactor and article influence score (AIS)(WoS)Based on PageRank, measures citations in terms of the prestige of citing journal. Not normalised to discipline or number of papers. Correlated with total citations. Ignores self-citations. AI is normalised to number of papers, so is like a JIF5-yr windowThe AI is normalised to number of papers. A value of 1.0 shows average influence across all journalsVery small values, difficult to interpret, Eigenfactor not normalisedFrom WoSEigenfactor:a)0.599b)0.03 AI:a) 22.2b) 6.56NYNNYYSJR and SJR2(Scopus)Based on citation prestige but also includes a size normalisation factor.SJR2 also allows for the closeness of the citing journal. 3-year windowComplex calculations and not easy to interpret. Not field normalisedNormalised number of papers but not to field so comparable to JIF. Most sophisticated indicatorFrom Scimagoa) 22.43b)15.3YNYh-index(Scimago website and Google Indicators)The h papers of a journal that have at least h citations. Can have any window – Google indicators uses 5-yearEasy to calculate and understand. Robust to poor dataNot normalised to number of papers or field.Not pure impact but includes volumeFrom Google Indicatorsh5:a) 223b) 72NNNSNIPRevised SNIP(Scopus)Citations per paper normalised to the relative database citation potential, that is the mean number of references in the papers that cite the journalNormalises both to number of papers and field.Does not consider citation prestige. Complex and difficult to check. Revised version is sensitive to variability of number of referencesFrom CWTSa) 7.9b) 6.19YYNI3Combines the distribution of citation percentiles with respect to a reference set with the number of papers in each percentile classNormalises across fields. Does not use the mean but is based on percentiles which is better for skewed dataNeeds reference sets based on pre-defined categories such as WoSNot knownNYNTable 1 Characteristics of Indicators for Measuring Journal Impact (Based on ADDIN EN.CITE <EndNote><Cite><Author>Mingers</Author><Year>2015</Year><RecNum>4717</RecNum><DisplayText>(Mingers & Leydesdorff, 2015b)</DisplayText><record><rec-number>4717</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4717</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Leydesdorff, L.</author></authors></contributors><titles><title>A review of theory and practice in scientometrics</title><secondary-title>European Journal of Operational Research</secondary-title></titles><periodical><full-title>European Journal of Operational Research</full-title></periodical><pages>1-19</pages><volume>246</volume><number>1</number><dates><year>2015</year></dates><urls></urls><electronic-resource-num>10.1016/j.ejor.2015.04.002</electronic-resource-num></record></Cite></EndNote>(Mingers & Leydesdorff, 2015b) Table 2)Table 2 shows the descriptive statistics with the following points of note:The different values of N reflect the different sources, i.e., WoS or Scopus or Scimago.There are widely different values from a mean of 0.00317 for the Eigenfactor to 129.8 for 3-year total citations.The variables, especially those involving un-normalised values of citations, are all highly skewed (the critical point is generally taken to be [6/N]1/2).All of the variables except the immediacy index are some form of citation metric. We have not included the citation half-life as it is time-based. Variable N Mean StDev Median Maximum SkewnessH index 1279 16.884 23.381 7.000 182.000 2.933-year Total Cites 1279 129.80 330.18 33.00 6028.00 8.04IF 426 1.2751 1.0790 0.9875 7.8170 2.22Total cites 426 1619 3047 696 26370 4.485-Year IF 426 1.6532 1.6004 1.2780 10.1540 1.92Immediacy Index 426 0.2433 0.2674 0.1570 2.7140 3.09Eigenfactor Score 426 0.003173 0.006325 0.001320 0.063680 5.90Article Influence Score 426 0.7644 1.0954 0.4535 9.0960 3.91SNIP 1022 0.8588 0.7743 0.6545 5.9210 2.08IPP 1022 0.9828 1.1323 0.6215 8.7220 2.67SJR 1134 0.7026 1.3002 0.2975 18.4400 6.03Table 2 Descriptive Statistics for the Journal IndicatorsThe next step was to look at the correlations between the various indicators.Figure 1 Correlation Plots between IndicatorsWe can see that several combinations, especially the various versions of the impact factor, have strong linear relationships as we would expect. We can therefore calculate the Pearson’s correlation coefficient as in Table 3. Tot.Cit IF 5-IF Immed. Eigen. AIS H-Ind. SNIP SJRIF 0.5355-Year IF 0.529 0.920Immediacy Ind. 0.353 0.600 0.551Eigenfactor 0.734 0.559 0.596 0.352Article Infl. 0.371 0.740 0.824 0.413 0.719H index 0.743 0.715 0.764 0.468 0.737 0.657SNIP 0.601 0.853 0.857 0.523 0.615 0.745 0.800SJR 0.553 0.806 0.835 0.462 0.780 0.906 0.779 0.807IPP 0.639 0.917 0.908 0.552 0.559 0.649 0.814 0.924 0.827Table 3. Correlation Coefficients (above 0.9 are greyed, below 0.5 are italicised)As expected, all of the coefficients are statistically significant at the 0.01 level. Those based on citations per paper (IF, 5-year IF, IPP) have very high correlations as do article influence and SJR, and SNIP and IPP. The latter one is interesting as SNIP is a normalised version of IPP but this shows that either the normalization does not work well, or that that journals show similar referencing patterns. Also of note is that the immediacy index has the lowest correlations with the other indicators. We should be careful, however, not to conclude from high correlations that the indicators are all measuring the same thing – West et al ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2010</Year><RecNum>3647</RecNum><DisplayText>(2010)</DisplayText><record><rec-number>3647</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3647</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>West, J.</author><author>Bergstrom, T.</author><author>Bergstrom, C.</author></authors></contributors><titles><title>Big Macs and Eigenfactor scores: Don't let correlation coefficients fool you</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>1800-1807</pages><volume>61</volume><number>9</number><dates><year>2010</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(2010) have shown that even in a sample where the Eigenfactor and total citations were correlated at the 0.995 level there were still significant differences in journal rankings between them.Analysis of the ResultsPrincipal ComponentsGiven the interesting pattern of correlations, it is useful to conduct a principal components analysis to look at the relationships between the variables. Figure 2 is a plot of the first two component loadings. PC1 Does not discriminate well between the indicators although those normalised for number of papers have higher values. PC2 distinguishes clearly between these types of indicators with those un-normalised having positive values. We have not included the immediacy index as this is something of an outlier as it is a very short-term 1-year JIF and so may not be appropriate for business and management where citation rates are slow in comparison with science. Figure 2 Plot of Principal Component Loadings 1 and 2From this plot we can see that the indicators fall into several groups plus some outliers. The main group are all those which normalise citations for the number of papers, that is JIF, 5-JIF, IPP, SNIP and SJR. At this level of aggregation the fact that SJR allows for prestige and SNIP for field effect does not seem evident. Close to this group is the article influence score (AIS) which is the Eigenfactor normalised for the number of papers. Towards the top are the 3-year total cites and also the Eigenfactor, neither of which normalise for papers. The h-index comes between these two groups which seems appropriate. There are two further points of note. First, SJR is closer to the other non-prestige indicators than it is to AIS which is the other prestige based metric, and it is far away from the Eigenfactor suggesting perhaps that it is not measuring prestige in the same way. Second, SNIP is very close to the impact factors suggesting that the source normalization is not having much effect. This could be because the citation practices within business and management do not differ greatly but there is evidence against this – Mingers and Leydesdorff ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2015</Year><RecNum>4448</RecNum><DisplayText>(2015a)</DisplayText><record><rec-number>4448</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4448</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Mingers, J.</author><author>Leydesdorff, L.</author></authors></contributors><titles><title>Identifying research fields within business and management: A journal cross-citation analysis</title><secondary-title>Journal of the Operational Research Society</secondary-title></titles><periodical><full-title>Journal of the Operational Research Society</full-title></periodical><pages>1370-1384</pages><volume>66</volume><number>8</number><edition>13/10/2014</edition><dates><year>2015</year></dates><urls></urls><electronic-resource-num> 10.1057/jors.2014.113</electronic-resource-num></record></Cite></EndNote>(2015a), in an analysis of journal cross-citation rates, identified six different sub-fields where the citation rates differed significantly, from a CPP of 32.5 in marketing, IB, strategy and IS, to 11.8 in operational research and operations management.Figure 3 is a plot of PC2 vs PC3. Here, we can see a clear differentiation between those that take prestige into account (SJR, Eigenfactor and AIS) and those that do not. This is perhaps the clearest categorisation of the indicators: NE, total cites and no prestige; SE, cites per paper and no prestige; SW, cites per paper and prestige; and NW, total cites and prestige.Figure 3 Plot of Principal Component Loadings 2 and 3Overall, the empirical results suggest that the theoretical differences between indicators can be detected at an aggregate level in the empirical results. We now turn to the practical results in terms of the rankings of journals using these indicators.Rank ordering of journalsThe questions to be considered in this section are the extent to which the different indicators rank order journals differently, and whether these differences reflect the theoretical differences described above. Presenting the results is difficult as we are comparing so many indicators at the same time - most studies compare only one or two. For example, Leydesdorff, and Bornmann ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2011</Year><RecNum>4492</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>4492</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4492</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Leydesdorff, Loet</author><author>Bornmann, Lutz</author></authors></contributors><titles><title>Integrated impact indicators compared with impact factors: An alternative research design with policy implications</title><secondary-title>Journal of the American Society for Information Science and Technology</secondary-title></titles><pages>2133-2146</pages><volume>62</volume><number>11</number><dates><year>2011</year></dates><publisher>Wiley Subscription Services, Inc., A Wiley Company</publisher><isbn>1532-2890</isbn><urls><related-urls><url>;(2011) compared the integrated impact indicator (a percentile based metric) with the journal impact factor concluding that the percentile approach had advantages. Falagas et al ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2008</Year><RecNum>4480</RecNum><DisplayText>(2008)</DisplayText><record><rec-number>4480</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4480</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Falagas, Matthew E</author><author>Kouranos, Vasilios D</author><author>Arencibia-Jorge, Ricardo</author><author>Karageorgopoulos, Drosos E</author></authors></contributors><titles><title>Comparison of SCImago journal rank indicator with journal impact factor</title><secondary-title>The FASEB Journal</secondary-title></titles><pages>2623-2628</pages><volume>22</volume><number>8</number><dates><year>2008</year></dates><isbn>0892-6638</isbn><urls></urls></record></Cite></EndNote>(2008) compared the JIF with the SJR concluding that both had advantages and disadvantages. Bollen et al ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2006</Year><RecNum>4594</RecNum><DisplayText>(2006)</DisplayText><record><rec-number>4594</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4594</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Bollen, Johan</author><author>Rodriquez, Marko A.</author><author>Van de Sompel, Herbert</author></authors></contributors><titles><title>Journal status</title><secondary-title>Scientometrics</secondary-title><alt-title>Scientometrics</alt-title></titles><pages>669-687</pages><volume>69</volume><number>3</number><dates><year>2006</year><pub-dates><date>2006/12/01</date></pub-dates></dates><publisher>Kluwer Academic Publishers</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(2006) compared JIF with a PageRank algorithm concluding that they were measuring rather different things – popularity as opposed to prestige – and suggested the Y-factor as a combination of the two. Fersht ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2009</Year><RecNum>4476</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>4476</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4476</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Fersht, Alan</author></authors></contributors><titles><title>The most influential journals: Impact Factor and Eigenfactor</title><secondary-title>Proceedings of the National Academy of Sciences</secondary-title></titles><pages>6883-6884</pages><volume>106</volume><number>17</number><dates><year>2009</year></dates><isbn>0027-8424</isbn><urls></urls></record></Cite></EndNote>(2009) discussed the JIF and the Eigenfactor and concluded that the Eigenfactor was strongly related to total citations, and that individual scientists should be judged by the h-index rather than journal indicators. In two other comparison studies, Glanzel et al ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2011</Year><RecNum>4485</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>4485</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4485</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gl?nzel, Wolfgang</author><author>Schubert, András</author><author>Thijs, Bart</author><author>Debackere, Koenraad</author></authors></contributors><titles><title>A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking</title><secondary-title>Scientometrics</secondary-title></titles><pages>415-424</pages><volume>87</volume><number>2</number><dates><year>2011</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(2011) looked at normalizing journal indicators and compared a priori normalization by using fractional counting of citations with a posteriori normalization of the JIF and suggested that both were useful approaches. .In Table 4 we have ranked the journals using the different indicators and we have also summed the ranks to give an overall ranking. The Table is ordered in terms of this summed rank. We have also shown the 2009 ABS journal list category (where available) and the field according to ABS as well as total citation over a 3 year period and total documents published per year. In using the sum of the ranks to select and order the journals, we are thereby biasing this table towards those journals that do well across the board, that is that do not do badly on any one metric. There may well be journals that do very well on the majority but do particularly badly on one or two indicators and they would not appear. We will see examples of this below.First, we can notice the following general points:The journals come from a range of different ABS sub-fields and they also have a wide range of total papers published - from 12 to 709, and citations - from 140 to 2029 so there is no obvious overall pattern.The vast majority are classified by ABS as 4* journals but five are only classified as 3* and three are not even included in ABS – International Organization, Academy of Management Perspectives and the Journal of Cleaner Production. We will show more detailed comparisons with ABS below.Standard TitleFieldABS 2009 1* to 4*Total Docs. 3-year Total CitesH index rankIF Rank5-Year IFIPP RankSNIP RankSJR RankEigen RankAI Score RankSum of RanksSum rankedJournal of FinanceFINANCE469142024761132261Academy of Management ReviewGEN MAN443987512124166372Academy of Management JournalGEN MAN4801384163813667503Journal of Management *GEN MAN4711533123545899554Journal of Financial EconomicsFINANCE4158202981313149325675Journal of MarketingMKT4481094710898111514826Review of Financial StudiesFINANCE4701626211792012213857MIS QuarterlyINFO MAN4681771105453132621878Strategic Management JournalSTRAT4186126432711171878161079Organization ScienceORG STUD4951407911162334971112010Journal of Operations ManagementOPS & TECH437916158671516503315011Personnel Psychology *HRM&EMP44446236712151917421316112Journal of Accounting and EconomicsACCOUNT431682283423101110331916813Journal of International Business StudiesIB&AREA4491026141515182824213517014Journal of Consumer ResearchMKT4811091183721222215122417115Journal of Management StudiesGEN MAN4661082262017192331202918516Journal of Business VenturingENT-SMBUS464753232224121427523420817Journal of Marketing ResearchMKT4541030194236354312101821518Journal of Organizational Behavior*ORG STUD483875222322243339293222419Research Policy*SOC SCI41491721114531302748134925420Organizational Research Methods*ORG STUD428449771814255020361725721Management ScienceOR&MANSCI416614276494558443252326222Journal of the Academy of Marketing Science*MKT452772171925213240615226723Journal of Accounting ResearchACCOUNT440377305337385621392529924Accounting ReviewACCOUNT475808316246413722273830425International Organization*29210404432653126511530426Brookings Papers on Economic Activity*ECON3121401112510101291831132627Omega*OR&MANSCI391970472440312033468632728Information Systems ResearchINFO MAN463960135928516135493733329Marketing ScienceMKT462688386365545514183033730Organizational Behavior and Human Decision ProcessesPSYCH474569293033559441282733731Entrepreneurship Theory and PracticeENT-SMBUS4113759724634263838565536532Long Range Planning*STRAT34071693682726231126139233Academy of Management Perspectives24344503538445955814040234Journal of Financial and Quantitative AnalysisFINANCE448479578975884725192042035International Journal of Management Reviews*GEN MAN33347411941261110611084542136Journal of Cleaner Production*709361752163028401031413842137Journal of Public Administration Research and Theory*PUB SEC437425893343674937624742738Journal of Labor Economics*ECON332172587863115631937844139Accounting, Organizations and SocietyACCOUNT438465486935344272796644540Table 4 Top 40 Journals Ranked by Sum of the Ranks (those with an * are not included in the FT Top-45 list)Even in this relatively consistent set of journals, i.e., the top ones, there are some very significant differences in rank. For example the h-index would rank some of them as over 50 places lower, generally those with relatively few citations and documents such as Brookings Papers and the Int. J. of Management Reviews. The Eigenfactor and IPP have similar effects.In contrast, some journals would be ranked much higher on particular indicators, e.g., Long Range Planning would be 30 places higher on SNIP and IPP and Brookings Papers would be 1st on article influence score. This is a particularly divergent journal being ranked 111th on h-index.Apart from the ABS list, another influential one is the FT Top-45 list, particularly important in the MBA market. These are often seen as the very elite journals. We have indicated which journals are not included in that list but arguably should be. Some in the FT list that arguably should not be are: Review of Accounting Studies (120th), Journal of Business Ethics (114th) and Human Resource Management (112th).We will now consider particular indicators. Table 5 shows a selection of journals that differ considerably (>=40 places) in their rank compared with the sum of ranks ranking. They are not necessarily from the top 40. We have been selective in choosing those which appear under several indicators. The numbers in brackets below are the total number of documents.H-indexIt is clear that the h-index is strongly affected by the number of documents published. The mean number of documents for higher ranked journals is 267 per year and for the lower journals is only 27 per year. The Academy of Management Annals is an interesting example because it is favoured by all the other indicators except the Eigenfactor. This journal is only published once per year and had only 6 papers in 2013, but each one was long and detailed and became highly cited. Other journals in a similar position with respect to the indicators are: Management and Organization Review (22), Human Resource Management Journal and J. of Consumer Culture. The European J. of OR (165) does well as it publishes many papers and is also well cited.IF, 5-Year IF, IPPThese indicators are all similar in normalizing for the number of papers but nothing else. They therefore favour those journals with high citations per paper. Journals favoured by these metrics are, for example, J. of Supply Chain Management (31), Management and Organization Review (22) and Human Resource Management Journal (23), the latter two in contrast to the h-index. Journals doing poorly are: J. Financial and Quantitative Analysis (48), European J. of OR (165), and J. of Conflict Resolution (44) which have relatively more papers. Of the three, the IF and 5-IF have more in common than the IPP.SNIPThis indicator normalises for the citation density in the field as well as the papers published by correcting the IPP for the length of reference lists. It is difficult to see from this data to what extent it works – it would need a large sample of journals from more diverse research fields, especially science and humanities. We can see that four of the favoured journals are the same as the IPP. In fact, there is one that is significantly different – the J. Supply Chain Management. Interestingly, in the lower journals there are two from the supply chain area and two from the business ethics area suggesting that possibly those fields have larger reference lists which has led to their IPP being reduced.SJR, Eigenfactor and article influence scoreThese indicators all include prestige although the Eigenfactor does not normalise for number of papers. We would therefore expect that SJR and AIS were quite similar and indeed they are. In Table 5 we can see several journals that are ranked highly by both, for example Academy of Mgt. Annals, Quantitative Marketing and Economics, Strategic Entrepreneurship J. and J. of Industrial Economics. Similarly with those downgraded - J. Business Research, J. Business Ethics and Int. Marketing Review. It is not possible to check, easily, whether these actually do differ in the prestige of the citing journals. The Eigenfactor, because of its lack of normalisation, grades several of these journals in the opposite direction. As noted above, metrics such as SJR and SNIP are difficult to investigate because of their lack of transparency and reproducibility.Table 4 also reveals some interesting contrasts between the indicators. For example, MIS Quarterly is ranked very highly by the impact factors and by SNIP, but much lower by SJR and AIS. This would imply that it gets a lot of citations but from relatively less prestigious journals. However, this may be because most of its citations would come from IS journals which would themselves be less prestigious than general management journals. In contrast, the J. of Financial Economics and the J. of Financial Studies are ranked higher by the prestige indicators than the pure citation ones. This could reflect the fact that several finance journals are all ranked very highly (three in the top ten) which could mean that finances journals as a whole are highly cited and so have greater prestige as a field. H-IndexIF5-IFIPPSNIPSJREigenAISJournals ranked higher than the sum of ranks by more than 40 placesMgt. ScienceEuropean J. of ORJ Business ResearchCalifornia Mgt. ReviewJ. of RetailingInt J of Proj. MgtJ. Business EthicsAcademy of Mgt. AnnalsFamily Business ReviewJ. Supply Chain Mgt.Mgt. and Organization ReviewStrategic Entrepreneurship JHuman Res. Mgt. JJ. of Industrial EconomicsInt. Mkting. ReviewJ. Economic InequalityInt J of Proj. MgtAcademy of Mgt. AnnalsResearch in Org. Beh.J. Supply Chain Mgt.Mgt. and Organization ReviewStrategic Entrepreneurship JJ. of Industrial Economics.Academy of Mgt. AnnalsJ. Supply Chain Mgt.Business HorizonsHuman Res. Mgt. JInt J of Proj. MgtAcademy of Mgt. AnnalsBusiness HorizonsACM Trans Inf. Syst.Int. J. of Proj. Mgt.J of Economic InequalityJ. Professional Issues in EngineeringHuman Res. Mgt. J J. of Consumer CultureAcademy of Mgt. AnnalsQuantitative Mkting. and EconomicsStrategic Entrepreneurship J.Human Res. Mgt. J.J. of Industrial EconomicsJ. Risk and UncertaintyQuantitative Mkting. and EconomicsEuropean J. of ORStrategic Entrepreneurship JJ. of Industrial EconomicsJ. Business Research J. Business EthicsAcademy of Mgt. AnnalsQuantitative Mkting. and EconomicsStrategic Entrepreneurship JJ. of Industrial EconomicsJ. Economic InequalityJ. of Consumer CultureInt J of Proj. MgtMathematical FinanceJ. Risk and UncertaintyJournals ranked lower than the sum of ranks by more than 40 placesAcademy of Mgt. AnnalsBusiness |HorizonsQuantitative Mkting. and EconomicsStrategic Entrepreneurship JJ. of Consumer CultureMgt. and Organization ReviewHuman Res. Mgt. JJ. of Industrial EconomicsJ. Economic InequalityJ. Financial and Quantitative AnalysisEuropean J. of ORInformation and Mgt.Int. J. Research in Mkting.Business HorizonsJ. Business ResearchJ. of Conflict ResolutionMathematical FinanceJ. Financial and Quantitative AnalysisEuropean J. of ORIMF Economic ReviewJ. of Conflict ResolutionInt J of Proj. MgHuman Res. Mgt. J Int. Mkting. ReviewMathematical FinanceJ. of Conflict ResolutionJ. Financial and Quantitative AnalysisMathematical FinanceJ. Risk and UncertaintyJ. Supply Chain Mgt.Supply Chain Mgt.Business Ethics QuarterlyJ. of Conflict ResolutionQuantitative Mkting. and EconomicsJ. Business EthicsJ. Risk and UncertaintyJ. Business ResearchJ. Business EthicsInt. Mkting. ReviewJ. Economic InequalityInt. Mkting. ReviewJ. Supply Chain MgtMgt. and Organization ReviewJ. of Consumer CultureHuman Res. Mgt. JJ. Business ResearchJ. Business EthicsInt. Mkting. ReviewEuropean J. of ORInt J of Proj. MgtTable 5 Journals that Differ in Rank Considerably Across IndicatorsSummaryReviewing the above results, we can come to the following tentative conclusions, based obviously on this particular sample of data.At first sight, through the correlation analysis, the indicators all appear to be very similar with very high correlation coefficient values. However, looking in more detail we see that in fact they differ considerably and individual journals may well change their rank position by over a hundred places from one indicator to another. This is very significant especially given the increasing concern with the quality of the journal as an (illicit) measure of the quality of a paper within it. It could easily lead to a journal (and also one of its papers) being classified as either a top journal or merely a low one (4* or a 2* in ABS terms).We can see that the theoretical differences also reveal themselves in the empirical data.The total citation metrics – h-index, total cites and Eigenfactor – favour journals that publish many papers and consequently disfavour journals publishing a few, highly-cited papers. They do not normalise for field.The mean citation metrics – IF, 5-IF, IPP, SNIP – favour journals that publish relatively few, highly-cited, papers and disfavour journals publishing a lot of papers, even if highly cited. Apart from SNIP, they do not normalise for field.There is only limited evidence, on this sample, that SNIP normalises significantly for field.There is some evidence that the prestige indicators – SJR, AIS – do have an effect but this may be field-related rather than journal-related.We do not see any one indicator as superior to the others – they all have their weaknesses and biases. However, given that they are being used and that one had to make a recommendation at this point in time, we would suggest using both the h-index, because it is transparent, easy to understand and robust to poor data especially if Google Scholar is being used; and SNIP as it aims to normalises for the number of publications and potentially the field as well. There are ways of normalising the h-index for field ADDIN EN.CITE <EndNote><Cite><Author>Iglesias</Author><Year>2007</Year><RecNum>3571</RecNum><DisplayText>(Iglesias & Pecharromán, 2007)</DisplayText><record><rec-number>3571</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">3571</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Iglesias, J.</author><author>Pecharromán, C.</author></authors></contributors><titles><title>Scaling the h-index for different scientific ISI fields</title><secondary-title>Scientometrics</secondary-title></titles><pages>303-320</pages><volume>73</volume><number>3</number><keywords><keyword>Computer Science</keyword></keywords><dates><year>2007</year></dates><publisher>Akadémiai Kiadó, co-published with Springer Science+Business Media B.V., Formerly Kluwer Academic Publishers B.V.</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(Iglesias & Pecharromán, 2007) although this is an area for further research ADDIN EN.CITE <EndNote><Cite><Author>Gl?nzel</Author><Year>2011</Year><RecNum>4485</RecNum><DisplayText>(Gl?nzel et al., 2011)</DisplayText><record><rec-number>4485</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4485</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gl?nzel, Wolfgang</author><author>Schubert, András</author><author>Thijs, Bart</author><author>Debackere, Koenraad</author></authors></contributors><titles><title>A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking</title><secondary-title>Scientometrics</secondary-title></titles><pages>415-424</pages><volume>87</volume><number>2</number><dates><year>2011</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(Gl?nzel et al., 2011).Comparing Journal Indicators with Peer Review Journal ListsIn practice, at the moment, most journal ranking is actually done through peer reviewed lists such as the ABS list, or the Australian Business Dean’s Council (ABDC) list which itself is a development of the more extensive Excellence in Research for Australia (ERA) list ADDIN EN.CITE <EndNote><Cite><Author>Hall</Author><Year>2011</Year><RecNum>4234</RecNum><DisplayText>(Hall, 2011)</DisplayText><record><rec-number>4234</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4234</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hall, C.</author></authors></contributors><titles><title>Publish and perish? Bibliometric analysis, journal ranking and the assessment of research quality in tourism</title><secondary-title>Tourism Management</secondary-title></titles><pages>16-27</pages><volume>32</volume><number>1</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(Hall, 2011), although these may include some use of bibliometric indicators in their compilation. Interestingly, the ERA ranked list was discontinued after 2010 and now all that is available is an unranked list of the journals that were submitted in the ERA. We have already demonstrated the important effects that these lists can have on universities, departments and even individual scholars despite the extensive criticism of such lists PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Oa29tbzwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJl
Y051bT4zMzI2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihBZGxlciAmYW1wOyBIYXJ6aW5nLCAyMDA5
OyBNb29zYSwgMjAxMTsgTmtvbW8sIDIwMDkpPC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVt
YmVyPjMzMjY8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJw
OTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjMzMjY8L2tleT48L2ZvcmVpZ24t
a2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPk5rb21vLCBTLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UaGUgc2VkdWN0aXZlIHBvd2VyIG9mIGFjYWRlbWlj
IGpvdXJuYWwgcmFua2luZ3M8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QWNhZGVteSBvZiBNYW5h
Z2VtZW50IExlYXJuaW5nIGFuZCBFZHVjYXRpb248L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGFnZXM+MTA2LTExMjwvcGFnZXM+PHZvbHVtZT44PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+
PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+TW9vc2E8L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+
NDg0NjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDg0NjwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIy
dzJhZGZlcCI+NDg0Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFs
IEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+TW9v
c2EsIEltYWQ8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
VGhlIGRlbWlzZSBvZiB0aGUgQVJDIGpvdXJuYWwgcmFua2luZyBzY2hlbWU6IGFuIGV4IHBvc3Qg
YW5hbHlzaXMgb2YgdGhlIGFjY291bnRpbmcgYW5kIGZpbmFuY2Ugam91cm5hbHM8L3RpdGxlPjxz
ZWNvbmRhcnktdGl0bGU+QWNjb3VudGluZyAmYW1wOyBGaW5hbmNlPC9zZWNvbmRhcnktdGl0bGU+
PC90aXRsZXM+PHBhZ2VzPjgwOS04MzY8L3BhZ2VzPjx2b2x1bWU+NTE8L3ZvbHVtZT48bnVtYmVy
PjM8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+Sm91cm5hbCByYW5raW5nPC9rZXl3b3JkPjxr
ZXl3b3JkPkFSQzwva2V5d29yZD48a2V5d29yZD5FeGNlbGxlbmNlIGluIFJlc2VhcmNoIGZvciBB
dXN0cmFsaWE8L2tleXdvcmQ+PGtleXdvcmQ+aCBpbmRleDwva2V5d29yZD48a2V5d29yZD5DaXRh
dGlvbnM8L2tleXdvcmQ+PGtleXdvcmQ+RzA8L2tleXdvcmQ+PGtleXdvcmQ+TTQxPC9rZXl3b3Jk
Pjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPkJs
YWNrd2VsbCBQdWJsaXNoaW5nIEx0ZDwvcHVibGlzaGVyPjxpc2JuPjE0NjctNjI5WDwvaXNibj48
dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTExMS9qLjE0Njct
NjI5WC4yMDExLjAwNDMyLng8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMt
cmVzb3VyY2UtbnVtPjEwLjExMTEvai4xNDY3LTYyOVguMjAxMS4wMDQzMi54PC9lbGVjdHJvbmlj
LXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5BZGxlcjwvQXV0aG9y
PjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT4zMjgzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl
cj4zMjgzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkw
ZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjgzPC9rZXk+PC9mb3JlaWduLWtl
eXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmli
dXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5BZGxlciwgTi48L2F1dGhvcj48YXV0aG9yPkhhcnppbmcs
IEEuLVcuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPldo
ZW4ga25vd2xlZGdlIHdpbnM6IFRyYW5zY2VuZGluZyB0aGUgc2Vuc2UgYW5kIG5vbnNlbnNlIG9m
IGFjYWRlbWljIHJhbmtpbmdzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFjYWRlbXkgb2YgTWFu
YWdlbWVudCBMZWFybmluZyBhbmQgRWR1Y2F0aW9uPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+
PHBhZ2VzPjcyLTk1PC9wYWdlcz48dm9sdW1lPjg8L3ZvbHVtZT48bnVtYmVyPjE8L251bWJlcj48
ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0Np
dGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Oa29tbzwvQXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJl
Y051bT4zMzI2PC9SZWNOdW0+PERpc3BsYXlUZXh0PihBZGxlciAmYW1wOyBIYXJ6aW5nLCAyMDA5
OyBNb29zYSwgMjAxMTsgTmtvbW8sIDIwMDkpPC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVt
YmVyPjMzMjY8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJw
OTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjMzMjY8L2tleT48L2ZvcmVpZ24t
a2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRy
aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPk5rb21vLCBTLjwvYXV0aG9yPjwvYXV0aG9ycz48L2Nv
bnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UaGUgc2VkdWN0aXZlIHBvd2VyIG9mIGFjYWRlbWlj
IGpvdXJuYWwgcmFua2luZ3M8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QWNhZGVteSBvZiBNYW5h
Z2VtZW50IExlYXJuaW5nIGFuZCBFZHVjYXRpb248L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48
cGFnZXM+MTA2LTExMjwvcGFnZXM+PHZvbHVtZT44PC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+
PGRhdGVzPjx5ZWFyPjIwMDk8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9D
aXRlPjxDaXRlPjxBdXRob3I+TW9vc2E8L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+
NDg0NjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+NDg0NjwvcmVjLW51bWJlcj48Zm9yZWln
bi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3JwYXZlY3dlMnIy
dzJhZGZlcCI+NDg0Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFs
IEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+TW9v
c2EsIEltYWQ8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+
VGhlIGRlbWlzZSBvZiB0aGUgQVJDIGpvdXJuYWwgcmFua2luZyBzY2hlbWU6IGFuIGV4IHBvc3Qg
YW5hbHlzaXMgb2YgdGhlIGFjY291bnRpbmcgYW5kIGZpbmFuY2Ugam91cm5hbHM8L3RpdGxlPjxz
ZWNvbmRhcnktdGl0bGU+QWNjb3VudGluZyAmYW1wOyBGaW5hbmNlPC9zZWNvbmRhcnktdGl0bGU+
PC90aXRsZXM+PHBhZ2VzPjgwOS04MzY8L3BhZ2VzPjx2b2x1bWU+NTE8L3ZvbHVtZT48bnVtYmVy
PjM8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+Sm91cm5hbCByYW5raW5nPC9rZXl3b3JkPjxr
ZXl3b3JkPkFSQzwva2V5d29yZD48a2V5d29yZD5FeGNlbGxlbmNlIGluIFJlc2VhcmNoIGZvciBB
dXN0cmFsaWE8L2tleXdvcmQ+PGtleXdvcmQ+aCBpbmRleDwva2V5d29yZD48a2V5d29yZD5DaXRh
dGlvbnM8L2tleXdvcmQ+PGtleXdvcmQ+RzA8L2tleXdvcmQ+PGtleXdvcmQ+TTQxPC9rZXl3b3Jk
Pjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PC9kYXRlcz48cHVibGlzaGVyPkJs
YWNrd2VsbCBQdWJsaXNoaW5nIEx0ZDwvcHVibGlzaGVyPjxpc2JuPjE0NjctNjI5WDwvaXNibj48
dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL2R4LmRvaS5vcmcvMTAuMTExMS9qLjE0Njct
NjI5WC4yMDExLjAwNDMyLng8L3VybD48L3JlbGF0ZWQtdXJscz48L3VybHM+PGVsZWN0cm9uaWMt
cmVzb3VyY2UtbnVtPjEwLjExMTEvai4xNDY3LTYyOVguMjAxMS4wMDQzMi54PC9lbGVjdHJvbmlj
LXJlc291cmNlLW51bT48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5BZGxlcjwvQXV0aG9y
PjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT4zMjgzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl
cj4zMjgzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icDkw
ZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zMjgzPC9rZXk+PC9mb3JlaWduLWtl
eXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmli
dXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5BZGxlciwgTi48L2F1dGhvcj48YXV0aG9yPkhhcnppbmcs
IEEuLVcuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPldo
ZW4ga25vd2xlZGdlIHdpbnM6IFRyYW5zY2VuZGluZyB0aGUgc2Vuc2UgYW5kIG5vbnNlbnNlIG9m
IGFjYWRlbWljIHJhbmtpbmdzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFjYWRlbXkgb2YgTWFu
YWdlbWVudCBMZWFybmluZyBhbmQgRWR1Y2F0aW9uPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+
PHBhZ2VzPjcyLTk1PC9wYWdlcz48dm9sdW1lPjg8L3ZvbHVtZT48bnVtYmVyPjE8L251bWJlcj48
ZGF0ZXM+PHllYXI+MjAwOTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0Np
dGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Adler & Harzing, 2009; Moosa, 2011; Nkomo, 2009). There have been several studies within information systems of the use of metrics PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5TdHJhdWI8L0F1dGhvcj48WWVhcj4yMDA2PC9ZZWFyPjxS
ZWNOdW0+NTA2NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oU3RyYXViLCAyMDA2OyBTdHJhdWIgJmFt
cDsgQW5kZXJzb24sIDIwMTApPC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjUwNjU8
L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVw
d3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjUwNjU8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48
YXV0aG9ycz48YXV0aG9yPlN0cmF1YiwgRGV0bWFyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJp
YnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRoZSBWYWx1ZSBvZiBTY2llbnRvbWV0cmljIFN0dWRpZXM6
IEFuIEludHJvZHVjdGlvbiB0byBhIERlYmF0ZSBvbiBJUyBhcyBhIFJlZmVyZW5jZSBEaXNjaXBs
aW5lPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2YgdGhlIEFzc29jaWF0aW9uIGZv
ciBJbmZvcm1hdGlvbiBTeXN0ZW1zPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjI0
MS0yNDU8L3BhZ2VzPjx2b2x1bWU+Nzwvdm9sdW1lPjxudW1iZXI+NTwvbnVtYmVyPjxrZXl3b3Jk
cz48a2V5d29yZD5JTkZPUk1BVElPTiByZXNvdXJjZXM8L2tleXdvcmQ+PGtleXdvcmQ+SU5GT1JN
QVRJT04gc2NpZW5jZTwva2V5d29yZD48a2V5d29yZD5QUkVGQUNFUyAmYW1wOyBmb3Jld29yZHM8
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwNjwveWVhcj48L2RhdGVzPjxwdWJs
aXNoZXI+QXNzb2NpYXRpb24gZm9yIEluZm9ybWF0aW9uIFN5c3RlbXM8L3B1Ymxpc2hlcj48aXNi
bj4xNTM2OTMyMzwvaXNibj48YWNjZXNzaW9uLW51bT4yMTA5MjczNDwvYWNjZXNzaW9uLW51bT48
dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3NlYXJjaC5lYnNjb2hvc3QuY29tL2xvZ2lu
LmFzcHg/ZGlyZWN0PXRydWUmYW1wO2RiPWJ0aCZhbXA7QU49MjEwOTI3MzQmYW1wO3NpdGU9ZWhv
c3QtbGl2ZTwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48cmVtb3RlLWRhdGFiYXNlLW5hbWU+
YnRoPC9yZW1vdGUtZGF0YWJhc2UtbmFtZT48cmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVyPkVCU0NP
aG9zdDwvcmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVyPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0
aG9yPlN0cmF1YjwvQXV0aG9yPjxZZWFyPjIwMTA8L1llYXI+PFJlY051bT40ODUwPC9SZWNOdW0+
PHJlY29yZD48cmVjLW51bWJlcj40ODUwPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh
cHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40ODUw
PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9Ik5ld3NwYXBlciBBcnRpY2xlIj4y
MzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlN0cmF1YiwgRGV0bWFy
PC9hdXRob3I+PGF1dGhvcj5BbmRlcnNvbiwgQ2hhZDwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRy
aWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Kb3VybmFsIFF1YWxpdHkgYW5kIENpdGF0aW9uczogQ29t
bW9uIE1ldHJpY3MgYW5kIENvbnNpZGVyYXRpb25zIGFib3V0IFRoZWlyIFVzZTwvdGl0bGU+PHNl
Y29uZGFyeS10aXRsZT5NSVMgUXVhcnRlcmx5PC9zZWNvbmRhcnktdGl0bGU+PGFsdC10aXRsZT5N
SVMgUXVhcnRlcmx5PC9hbHQtdGl0bGU+PC90aXRsZXM+PHBhZ2VzPmlpaS14aWk8L3BhZ2VzPjx2
b2x1bWU+MzQ8L3ZvbHVtZT48a2V5d29yZHM+PGtleXdvcmQ+UEVFUiByZXZpZXcgKFByb2Zlc3Np
b25hbCBwZXJmb3JtYW5jZSk8L2tleXdvcmQ+PGtleXdvcmQ+U0NIT0xBUkxZIHBlcmlvZGljYWxz
IC0tIEV2YWx1YXRpb248L2tleXdvcmQ+PGtleXdvcmQ+U0NIT0xBUkxZIHB1Ymxpc2hpbmcgLS0g
RXZhbHVhdGlvbjwva2V5d29yZD48a2V5d29yZD5CSUJMSU9HUkFQSElDQUwgY2l0YXRpb25zPC9r
ZXl3b3JkPjxrZXl3b3JkPlRFQ0hOSUNBTCB3cml0aW5nPC9rZXl3b3JkPjxrZXl3b3JkPkFOTk9U
QVRJT05TICZhbXA7IGNpdGF0aW9ucyAoTGF3KTwva2V5d29yZD48a2V5d29yZD5BQ0FERU1JQyBk
aXNjb3Vyc2U8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAxMDwveWVhcj48L2Rh
dGVzPjxwdWJsaXNoZXI+TUlTIFF1YXJ0ZXJseTwvcHVibGlzaGVyPjxpc2JuPjAyNzY3NzgzPC9p
c2JuPjx3b3JrLXR5cGU+RWRpdG9yaWFsPC93b3JrLXR5cGU+PHVybHM+PHJlbGF0ZWQtdXJscz48
dXJsPmh0dHA6Ly9zZWFyY2guZWJzY29ob3N0LmNvbS9sb2dpbi5hc3B4P2RpcmVjdD10cnVlJmFt
cDtkYj1idGgmYW1wO0FOPTQ4NDc2NTM5JmFtcDtzaXRlPWVob3N0LWxpdmU8L3VybD48L3JlbGF0
ZWQtdXJscz48L3VybHM+PHJlbW90ZS1kYXRhYmFzZS1uYW1lPmJ0aDwvcmVtb3RlLWRhdGFiYXNl
LW5hbWU+PHJlbW90ZS1kYXRhYmFzZS1wcm92aWRlcj5FQlNDT2hvc3Q8L3JlbW90ZS1kYXRhYmFz
ZS1wcm92aWRlcj48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5TdHJhdWI8L0F1dGhvcj48WWVhcj4yMDA2PC9ZZWFyPjxS
ZWNOdW0+NTA2NTwvUmVjTnVtPjxEaXNwbGF5VGV4dD4oU3RyYXViLCAyMDA2OyBTdHJhdWIgJmFt
cDsgQW5kZXJzb24sIDIwMTApPC9EaXNwbGF5VGV4dD48cmVjb3JkPjxyZWMtbnVtYmVyPjUwNjU8
L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVw
d3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjUwNjU8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVm
LXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48
YXV0aG9ycz48YXV0aG9yPlN0cmF1YiwgRGV0bWFyPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJp
YnV0b3JzPjx0aXRsZXM+PHRpdGxlPlRoZSBWYWx1ZSBvZiBTY2llbnRvbWV0cmljIFN0dWRpZXM6
IEFuIEludHJvZHVjdGlvbiB0byBhIERlYmF0ZSBvbiBJUyBhcyBhIFJlZmVyZW5jZSBEaXNjaXBs
aW5lPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2YgdGhlIEFzc29jaWF0aW9uIGZv
ciBJbmZvcm1hdGlvbiBTeXN0ZW1zPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjI0
MS0yNDU8L3BhZ2VzPjx2b2x1bWU+Nzwvdm9sdW1lPjxudW1iZXI+NTwvbnVtYmVyPjxrZXl3b3Jk
cz48a2V5d29yZD5JTkZPUk1BVElPTiByZXNvdXJjZXM8L2tleXdvcmQ+PGtleXdvcmQ+SU5GT1JN
QVRJT04gc2NpZW5jZTwva2V5d29yZD48a2V5d29yZD5QUkVGQUNFUyAmYW1wOyBmb3Jld29yZHM8
L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAwNjwveWVhcj48L2RhdGVzPjxwdWJs
aXNoZXI+QXNzb2NpYXRpb24gZm9yIEluZm9ybWF0aW9uIFN5c3RlbXM8L3B1Ymxpc2hlcj48aXNi
bj4xNTM2OTMyMzwvaXNibj48YWNjZXNzaW9uLW51bT4yMTA5MjczNDwvYWNjZXNzaW9uLW51bT48
dXJscz48cmVsYXRlZC11cmxzPjx1cmw+aHR0cDovL3NlYXJjaC5lYnNjb2hvc3QuY29tL2xvZ2lu
LmFzcHg/ZGlyZWN0PXRydWUmYW1wO2RiPWJ0aCZhbXA7QU49MjEwOTI3MzQmYW1wO3NpdGU9ZWhv
c3QtbGl2ZTwvdXJsPjwvcmVsYXRlZC11cmxzPjwvdXJscz48cmVtb3RlLWRhdGFiYXNlLW5hbWU+
YnRoPC9yZW1vdGUtZGF0YWJhc2UtbmFtZT48cmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVyPkVCU0NP
aG9zdDwvcmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVyPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0
aG9yPlN0cmF1YjwvQXV0aG9yPjxZZWFyPjIwMTA8L1llYXI+PFJlY051bT40ODUwPC9SZWNOdW0+
PHJlY29yZD48cmVjLW51bWJlcj40ODUwPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh
cHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj40ODUw
PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9Ik5ld3NwYXBlciBBcnRpY2xlIj4y
MzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlN0cmF1YiwgRGV0bWFy
PC9hdXRob3I+PGF1dGhvcj5BbmRlcnNvbiwgQ2hhZDwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRy
aWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Kb3VybmFsIFF1YWxpdHkgYW5kIENpdGF0aW9uczogQ29t
bW9uIE1ldHJpY3MgYW5kIENvbnNpZGVyYXRpb25zIGFib3V0IFRoZWlyIFVzZTwvdGl0bGU+PHNl
Y29uZGFyeS10aXRsZT5NSVMgUXVhcnRlcmx5PC9zZWNvbmRhcnktdGl0bGU+PGFsdC10aXRsZT5N
SVMgUXVhcnRlcmx5PC9hbHQtdGl0bGU+PC90aXRsZXM+PHBhZ2VzPmlpaS14aWk8L3BhZ2VzPjx2
b2x1bWU+MzQ8L3ZvbHVtZT48a2V5d29yZHM+PGtleXdvcmQ+UEVFUiByZXZpZXcgKFByb2Zlc3Np
b25hbCBwZXJmb3JtYW5jZSk8L2tleXdvcmQ+PGtleXdvcmQ+U0NIT0xBUkxZIHBlcmlvZGljYWxz
IC0tIEV2YWx1YXRpb248L2tleXdvcmQ+PGtleXdvcmQ+U0NIT0xBUkxZIHB1Ymxpc2hpbmcgLS0g
RXZhbHVhdGlvbjwva2V5d29yZD48a2V5d29yZD5CSUJMSU9HUkFQSElDQUwgY2l0YXRpb25zPC9r
ZXl3b3JkPjxrZXl3b3JkPlRFQ0hOSUNBTCB3cml0aW5nPC9rZXl3b3JkPjxrZXl3b3JkPkFOTk9U
QVRJT05TICZhbXA7IGNpdGF0aW9ucyAoTGF3KTwva2V5d29yZD48a2V5d29yZD5BQ0FERU1JQyBk
aXNjb3Vyc2U8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+MjAxMDwveWVhcj48L2Rh
dGVzPjxwdWJsaXNoZXI+TUlTIFF1YXJ0ZXJseTwvcHVibGlzaGVyPjxpc2JuPjAyNzY3NzgzPC9p
c2JuPjx3b3JrLXR5cGU+RWRpdG9yaWFsPC93b3JrLXR5cGU+PHVybHM+PHJlbGF0ZWQtdXJscz48
dXJsPmh0dHA6Ly9zZWFyY2guZWJzY29ob3N0LmNvbS9sb2dpbi5hc3B4P2RpcmVjdD10cnVlJmFt
cDtkYj1idGgmYW1wO0FOPTQ4NDc2NTM5JmFtcDtzaXRlPWVob3N0LWxpdmU8L3VybD48L3JlbGF0
ZWQtdXJscz48L3VybHM+PHJlbW90ZS1kYXRhYmFzZS1uYW1lPmJ0aDwvcmVtb3RlLWRhdGFiYXNl
LW5hbWU+PHJlbW90ZS1kYXRhYmFzZS1wcm92aWRlcj5FQlNDT2hvc3Q8L3JlbW90ZS1kYXRhYmFz
ZS1wcm92aWRlcj48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPn==
ADDIN EN.CITE.DATA (Straub, 2006; Straub & Anderson, 2010) and comparisons with expert rankings PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Mb3dyeTwvQXV0aG9yPjxZZWFyPjIwMDQ8L1llYXI+PFJl
Y051bT4yMzU1PC9SZWNOdW0+PERpc3BsYXlUZXh0PihMb3dyeSBldCBhbC4sIDIwMTM7IExvd3J5
IGV0IGFsLiwgMjAwNCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjM1NTwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+MjM1NTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+TG93cnksIFAuIEIuPC9hdXRob3I+PGF1dGhvcj5Sb21hbnMsIEQuPC9hdXRo
b3I+PGF1dGhvcj5DdXJ0aXMsIEEuIDwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48
dGl0bGVzPjx0aXRsZT5HbG9iYWwgam91cm5hbCBwcmVzdGlnZSBhbmQgc3VwcG9ydGluZyBkaXNj
aXBsaW5lczogQSBzY2llbnRvbWV0cmljIHN0dWR5IG9mIGluZm9ybWF0aW9uIHN5c3RlbXMgam91
cm5hbHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiB0aGUgQXNzb2NpYXRpb24g
Zm9yIEluZm9ybWF0aW9uIFN5c3RlbXM8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+
MjktNzU8L3BhZ2VzPjx2b2x1bWU+NTwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48
eWVhcj4yMDA0PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0
ZT48QXV0aG9yPkxvd3J5PC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48UmVjTnVtPjUwNjE8L1Jl
Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjUwNjE8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48
a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAi
PjUwNjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl
Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkxvd3J5LCBQYXVs
IEJlbmphbWluPC9hdXRob3I+PGF1dGhvcj5Nb29keSwgR3JlZ29yeSBELjwvYXV0aG9yPjxhdXRo
b3I+R2Fza2luLCBKYW1lczwvYXV0aG9yPjxhdXRob3I+R2FsbGV0dGEsIERlbm5pcyBGLjwvYXV0
aG9yPjxhdXRob3I+SHVtcGhlcnlzLCBTZWFuIEwuPC9hdXRob3I+PGF1dGhvcj5CYXJsb3csIEpv
cmRhbiBCLjwvYXV0aG9yPjxhdXRob3I+V2lsc29uLCBEYXZpZCBXLjwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5FdmFsdWF0aW5nIGpvdXJuYWwgcXVhbGl0
eSBhbmQgdGhlIEFzc29jaWF0aW9uIGZvciBJbmZvcm1hdGlvbiBTeXN0ZW1zIFNlbmlvciBTY2hv
bGFycyZhcG9zOyBqb3VybmFsIGJhc2tldCB2aWEgYmlibGlvbWV0cmljIG1lYXN1cmVzOiBEbyBl
eHBlcnQgam91cm5hbCBhc3Nlc3NtZW50cyBhZGQgdmFsdWU/PC90aXRsZT48c2Vjb25kYXJ5LXRp
dGxlPk1JUyBxdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+OTkzLUEy
MTwvcGFnZXM+PHZvbHVtZT4zNzwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxrZXl3b3Jkcz48
a2V5d29yZD5JTkZPUk1BVElPTiBzdG9yYWdlICZhbXA7IHJldHJpZXZhbCBzeXN0ZW1zPC9rZXl3
b3JkPjxrZXl3b3JkPk1BTkFHRU1FTlQgaW5mb3JtYXRpb24gc3lzdGVtczwva2V5d29yZD48a2V5
d29yZD5TQ0lFTlRPTUVUUklDUzwva2V5d29yZD48a2V5d29yZD5JTVBBQ1QgZmFjdG9yIChDaXRh
dGlvbiBhbmFseXNpcyk8L2tleXdvcmQ+PGtleXdvcmQ+UEVSSU9ESUNBTFM8L2tleXdvcmQ+PGtl
eXdvcmQ+Sk9VUk5BTCBwcm9kdWN0aXZpdHk8L2tleXdvcmQ+PGtleXdvcmQ+QklCTElPTUVUUklD
Uzwva2V5d29yZD48a2V5d29yZD5BSVMgU2VuaW9yIFNjaG9sYXJzIGJhc2tldCBvZiBqb3VybmFs
czwva2V5d29yZD48a2V5d29yZD5jb21wb3NpdGUgcmFua2luZyBvciByYXRpbmc8L2tleXdvcmQ+
PGtleXdvcmQ+ZXhwZXJ0IG9waW5pb248L2tleXdvcmQ+PGtleXdvcmQ+aC1pbmRleDwva2V5d29y
ZD48a2V5d29yZD5pbXBhY3QgZmFjdG9yPC9rZXl3b3JkPjxrZXl3b3JkPkluZm9ybWF0aW9uIHN5
c3RlbXMgam91cm5hbCByYW5raW5nczwva2V5d29yZD48a2V5d29yZD5qb3VybmFsIHF1YWxpdHk8
L2tleXdvcmQ+PGtleXdvcmQ+bm9tb2xvZ2llcyBmb3IgZGlzc2VtaW5hdGlvbiBvZiBzY2llbnRp
ZmljIGtub3dsZWRnZTwva2V5d29yZD48a2V5d29yZD5zZWxmLWNpdGF0aW9uPC9rZXl3b3JkPjxr
ZXl3b3JkPlNlblMtNjwva2V5d29yZD48a2V5d29yZD5TZW5TLTg8L2tleXdvcmQ+PGtleXdvcmQ+
c29jaWFsIG5ldHdvcmsgYW5hbHlzaXM8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+
MjAxMzwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+TUlTIFF1YXJ0ZXJseTwvcHVibGlzaGVyPjxp
c2JuPjAyNzY3NzgzPC9pc2JuPjxhY2Nlc3Npb24tbnVtPjkxOTA2MjQxPC9hY2Nlc3Npb24tbnVt
Pjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vc2VhcmNoLmVic2NvaG9zdC5jb20vbG9n
aW4uYXNweD9kaXJlY3Q9dHJ1ZSZhbXA7ZGI9YnRoJmFtcDtBTj05MTkwNjI0MSZhbXA7c2l0ZT1l
aG9zdC1saXZlPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxyZW1vdGUtZGF0YWJhc2UtbmFt
ZT5idGg8L3JlbW90ZS1kYXRhYmFzZS1uYW1lPjxyZW1vdGUtZGF0YWJhc2UtcHJvdmlkZXI+RUJT
Q09ob3N0PC9yZW1vdGUtZGF0YWJhc2UtcHJvdmlkZXI+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90
ZT5=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Mb3dyeTwvQXV0aG9yPjxZZWFyPjIwMDQ8L1llYXI+PFJl
Y051bT4yMzU1PC9SZWNOdW0+PERpc3BsYXlUZXh0PihMb3dyeSBldCBhbC4sIDIwMTM7IExvd3J5
IGV0IGFsLiwgMjAwNCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjM1NTwvcmVj
LW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2
ZXd0c3JwYXZlY3dlMnIydzJhZGZlcCI+MjM1NTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw
ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo
b3JzPjxhdXRob3I+TG93cnksIFAuIEIuPC9hdXRob3I+PGF1dGhvcj5Sb21hbnMsIEQuPC9hdXRo
b3I+PGF1dGhvcj5DdXJ0aXMsIEEuIDwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48
dGl0bGVzPjx0aXRsZT5HbG9iYWwgam91cm5hbCBwcmVzdGlnZSBhbmQgc3VwcG9ydGluZyBkaXNj
aXBsaW5lczogQSBzY2llbnRvbWV0cmljIHN0dWR5IG9mIGluZm9ybWF0aW9uIHN5c3RlbXMgam91
cm5hbHM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiB0aGUgQXNzb2NpYXRpb24g
Zm9yIEluZm9ybWF0aW9uIFN5c3RlbXM8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+
MjktNzU8L3BhZ2VzPjx2b2x1bWU+NTwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48
eWVhcj4yMDA0PC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0
ZT48QXV0aG9yPkxvd3J5PC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48UmVjTnVtPjUwNjE8L1Jl
Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjUwNjE8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48
a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAi
PjUwNjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl
Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkxvd3J5LCBQYXVs
IEJlbmphbWluPC9hdXRob3I+PGF1dGhvcj5Nb29keSwgR3JlZ29yeSBELjwvYXV0aG9yPjxhdXRo
b3I+R2Fza2luLCBKYW1lczwvYXV0aG9yPjxhdXRob3I+R2FsbGV0dGEsIERlbm5pcyBGLjwvYXV0
aG9yPjxhdXRob3I+SHVtcGhlcnlzLCBTZWFuIEwuPC9hdXRob3I+PGF1dGhvcj5CYXJsb3csIEpv
cmRhbiBCLjwvYXV0aG9yPjxhdXRob3I+V2lsc29uLCBEYXZpZCBXLjwvYXV0aG9yPjwvYXV0aG9y
cz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5FdmFsdWF0aW5nIGpvdXJuYWwgcXVhbGl0
eSBhbmQgdGhlIEFzc29jaWF0aW9uIGZvciBJbmZvcm1hdGlvbiBTeXN0ZW1zIFNlbmlvciBTY2hv
bGFycyZhcG9zOyBqb3VybmFsIGJhc2tldCB2aWEgYmlibGlvbWV0cmljIG1lYXN1cmVzOiBEbyBl
eHBlcnQgam91cm5hbCBhc3Nlc3NtZW50cyBhZGQgdmFsdWU/PC90aXRsZT48c2Vjb25kYXJ5LXRp
dGxlPk1JUyBxdWFydGVybHk8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGFnZXM+OTkzLUEy
MTwvcGFnZXM+PHZvbHVtZT4zNzwvdm9sdW1lPjxudW1iZXI+NDwvbnVtYmVyPjxrZXl3b3Jkcz48
a2V5d29yZD5JTkZPUk1BVElPTiBzdG9yYWdlICZhbXA7IHJldHJpZXZhbCBzeXN0ZW1zPC9rZXl3
b3JkPjxrZXl3b3JkPk1BTkFHRU1FTlQgaW5mb3JtYXRpb24gc3lzdGVtczwva2V5d29yZD48a2V5
d29yZD5TQ0lFTlRPTUVUUklDUzwva2V5d29yZD48a2V5d29yZD5JTVBBQ1QgZmFjdG9yIChDaXRh
dGlvbiBhbmFseXNpcyk8L2tleXdvcmQ+PGtleXdvcmQ+UEVSSU9ESUNBTFM8L2tleXdvcmQ+PGtl
eXdvcmQ+Sk9VUk5BTCBwcm9kdWN0aXZpdHk8L2tleXdvcmQ+PGtleXdvcmQ+QklCTElPTUVUUklD
Uzwva2V5d29yZD48a2V5d29yZD5BSVMgU2VuaW9yIFNjaG9sYXJzIGJhc2tldCBvZiBqb3VybmFs
czwva2V5d29yZD48a2V5d29yZD5jb21wb3NpdGUgcmFua2luZyBvciByYXRpbmc8L2tleXdvcmQ+
PGtleXdvcmQ+ZXhwZXJ0IG9waW5pb248L2tleXdvcmQ+PGtleXdvcmQ+aC1pbmRleDwva2V5d29y
ZD48a2V5d29yZD5pbXBhY3QgZmFjdG9yPC9rZXl3b3JkPjxrZXl3b3JkPkluZm9ybWF0aW9uIHN5
c3RlbXMgam91cm5hbCByYW5raW5nczwva2V5d29yZD48a2V5d29yZD5qb3VybmFsIHF1YWxpdHk8
L2tleXdvcmQ+PGtleXdvcmQ+bm9tb2xvZ2llcyBmb3IgZGlzc2VtaW5hdGlvbiBvZiBzY2llbnRp
ZmljIGtub3dsZWRnZTwva2V5d29yZD48a2V5d29yZD5zZWxmLWNpdGF0aW9uPC9rZXl3b3JkPjxr
ZXl3b3JkPlNlblMtNjwva2V5d29yZD48a2V5d29yZD5TZW5TLTg8L2tleXdvcmQ+PGtleXdvcmQ+
c29jaWFsIG5ldHdvcmsgYW5hbHlzaXM8L2tleXdvcmQ+PC9rZXl3b3Jkcz48ZGF0ZXM+PHllYXI+
MjAxMzwveWVhcj48L2RhdGVzPjxwdWJsaXNoZXI+TUlTIFF1YXJ0ZXJseTwvcHVibGlzaGVyPjxp
c2JuPjAyNzY3NzgzPC9pc2JuPjxhY2Nlc3Npb24tbnVtPjkxOTA2MjQxPC9hY2Nlc3Npb24tbnVt
Pjx1cmxzPjxyZWxhdGVkLXVybHM+PHVybD5odHRwOi8vc2VhcmNoLmVic2NvaG9zdC5jb20vbG9n
aW4uYXNweD9kaXJlY3Q9dHJ1ZSZhbXA7ZGI9YnRoJmFtcDtBTj05MTkwNjI0MSZhbXA7c2l0ZT1l
aG9zdC1saXZlPC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxyZW1vdGUtZGF0YWJhc2UtbmFt
ZT5idGg8L3JlbW90ZS1kYXRhYmFzZS1uYW1lPjxyZW1vdGUtZGF0YWJhc2UtcHJvdmlkZXI+RUJT
Q09ob3N0PC9yZW1vdGUtZGF0YWJhc2UtcHJvdmlkZXI+PC9yZWNvcmQ+PC9DaXRlPjwvRW5kTm90
ZT5=
ADDIN EN.CITE.DATA (Lowry et al., 2013; Lowry et al., 2004). Note also the San Francisco Declaration on Research Assessment (DORA, ) and the Leiden Manifesto for Research Metrics () which both set out guidelines for the proper use of metrics in evaluating research.In the UK, the ABS list is predominant despite intense criticism PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ib2VwbmVyPC9BdXRob3I+PFllYXI+MjAwOTwvWWVhcj48
UmVjTnVtPjM4MDE8L1JlY051bT48RGlzcGxheVRleHQ+KEhvZXBuZXIgJmFtcDsgVW5lcm1hbiwg
MjAwOTsgSHVzc2FpbiwgMjAxMTsgSHVzc2FpbiwgMjAxMzsgTWluZ2VycyAmYW1wOyBXaWxsbW90
dCwgMjAxMzsgTW9ycmlzIGV0IGFsLiwgMjAwOTsgV2lsbG1vdHQsIDIwMTEpPC9EaXNwbGF5VGV4
dD48cmVjb3JkPjxyZWMtbnVtYmVyPjM4MDE8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjM4
MDE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkhvZXBuZXIsIEEuPC9h
dXRob3I+PGF1dGhvcj5VbmVybWFuLCBKLiA8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRv
cnM+PHRpdGxlcz48dGl0bGU+RXhwbGljaXQgYW5kIGltcGxpY2l0IHN1YmplY3QgYmlhcyBpbiB0
aGUgQUJTIEpvdXJuYWwgUXVhbGl0eSBHdWlkZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BY2Nv
dW50aW5nIEVkdWNhdGlvbjwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4zLTE1PC9w
YWdlcz48dm9sdW1lPjIxPC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIw
MDk8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+SHVzc2FpbjwvQXV0aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051bT4zODAzPC9SZWNOdW0+
PHJlY29yZD48cmVjLW51bWJlcj4zODAzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh
cHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zODAz
PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8
L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IdXNzYWluLCBTLjwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Gb29kIGZvciB0aG91
Z2h0IG9uIHRoZSBBQlMgYWNhZGVtaWMgam91cm5hbCBxdWFsaXR5IGd1aWRlPC90aXRsZT48c2Vj
b25kYXJ5LXRpdGxlPkFjY291bnRpbmcgRWR1Y2F0aW9uPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs
ZXM+PHBhZ2VzPjU0NS01NTk8L3BhZ2VzPjx2b2x1bWU+MjA8L3ZvbHVtZT48bnVtYmVyPjY8L251
bWJlcj48ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29y
ZD48L0NpdGU+PENpdGU+PEF1dGhvcj5IdXNzYWluPC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48
UmVjTnVtPjQyMzI8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQyMzI8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2
ZWN3ZTJyMncyYWRmZXAiPjQyMzI8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i
Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkh1c3NhaW4sIFMuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPkpvdXJuYWwgRmV0aXNoaXNtIGFuZCB0aGUg4oCYU2lnbiBvZiB0aGUgNOKAmSBpbiB0
aGUgQUJTIGd1aWRlOiBBIFF1ZXN0aW9uIG9mIFRydXN0PzwvdGl0bGU+PHNlY29uZGFyeS10aXRs
ZT5Pcmdhbml6YXRpb248L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48dm9sdW1lPm9ubGluZTwv
dm9sdW1lPjxkYXRlcz48eWVhcj4yMDEzPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjxlbGVj
dHJvbmljLXJlc291cmNlLW51bT5kb2k6IDEwLjExNzcvMTM1MDUwODQxMzUwNjc2MyA8L2VsZWN0
cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk1vcnJpczwv
QXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT4zNTY5PC9SZWNOdW0+PHJlY29yZD48cmVj
LW51bWJlcj4zNTY5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1p
ZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zNTY5PC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Nb3JyaXMsIE0uPC9hdXRob3I+PGF1dGhvcj5I
YXJ2ZXksIEMuPC9hdXRob3I+PGF1dGhvcj5LZWxseSwgQS48L2F1dGhvcj48L2F1dGhvcnM+PC9j
b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Sm91cm5hbCByYW5raW5ncyBhbmQgdGhlIEFCUyBK
b3VybmFsIFF1YWxpdHkgR3VpZGU8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TWFuYWdlbWVudCBE
ZWNpc2lvbjwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4xNDQxLTE0NTE8L3BhZ2Vz
Pjx2b2x1bWU+NDc8L3ZvbHVtZT48bnVtYmVyPjk8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwOTwv
eWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5X
aWxsbW90dDwvQXV0aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051bT4zNzk5PC9SZWNOdW0+PHJl
Y29yZD48cmVjLW51bWJlcj4zNzk5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9
IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zNzk5PC9r
ZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3Jl
Zi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XaWxsbW90dCwgSHVnaDwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Kb3VybmFsIGxpc3Qg
ZmV0aXNoaXNtIGFuZCB0aGUgcGVydmVyc2lvbiBvZiBzY2hvbGFyc2hpcDogcmVhY3Rpdml0eSBh
bmQgdGhlIEFCUyBsaXN0PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk9yZ2FuaXphdGlvbjwvc2Vj
b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz40MjktNDQyPC9wYWdlcz48dm9sdW1lPjE4PC92
b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PHB1Yi1kYXRl
cz48ZGF0ZT5KdWx5IDEsIDIwMTE8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48dXJscz48cmVs
YXRlZC11cmxzPjx1cmw+aHR0cDovL29yZy5zYWdlcHViLmNvbS9jb250ZW50LzE4LzQvNDI5LmFi
c3RyYWN0PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51
bT4xMC4xMTc3LzEzNTA1MDg0MTE0MDM1MzI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk1pbmdlcnM8L0F1dGhvcj48WWVhcj4yMDEzPC9ZZWFy
PjxSZWNOdW0+MzgwMjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzgwMjwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3Jw
YXZlY3dlMnIydzJhZGZlcCI+MzgwMjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1l
PSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+TWluZ2VycywgSi48L2F1dGhvcj48YXV0aG9yPldpbGxtb3R0LCBILjwvYXV0aG9yPjwv
YXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UYXlsb3JpemluZyBidXNpbmVz
cyBzY2hvb2wgcmVzZWFyY2g6IE9uIHRoZSAmcXVvdDtvbmUgYmVzdCB3YXkmcXVvdDsgcGVyZm9y
bWF0aXZlIGVmZmVjdHMgb2Ygam91cm5hbCByYW5raW5nIGxpc3RzPC90aXRsZT48c2Vjb25kYXJ5
LXRpdGxlPkh1bWFuIFJlbGF0aW9uczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4x
MDUxLTEwNzM8L3BhZ2VzPjx2b2x1bWU+NjY8L3ZvbHVtZT48bnVtYmVyPjg8L251bWJlcj48ZGF0
ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+
PC9FbmROb3RlPgB=
ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Ib2VwbmVyPC9BdXRob3I+PFllYXI+MjAwOTwvWWVhcj48
UmVjTnVtPjM4MDE8L1JlY051bT48RGlzcGxheVRleHQ+KEhvZXBuZXIgJmFtcDsgVW5lcm1hbiwg
MjAwOTsgSHVzc2FpbiwgMjAxMTsgSHVzc2FpbiwgMjAxMzsgTWluZ2VycyAmYW1wOyBXaWxsbW90
dCwgMjAxMzsgTW9ycmlzIGV0IGFsLiwgMjAwOTsgV2lsbG1vdHQsIDIwMTEpPC9EaXNwbGF5VGV4
dD48cmVjb3JkPjxyZWMtbnVtYmVyPjM4MDE8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5
IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2ZWN3ZTJyMncyYWRmZXAiPjM4
MDE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x
NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkhvZXBuZXIsIEEuPC9h
dXRob3I+PGF1dGhvcj5VbmVybWFuLCBKLiA8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRv
cnM+PHRpdGxlcz48dGl0bGU+RXhwbGljaXQgYW5kIGltcGxpY2l0IHN1YmplY3QgYmlhcyBpbiB0
aGUgQUJTIEpvdXJuYWwgUXVhbGl0eSBHdWlkZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5BY2Nv
dW50aW5nIEVkdWNhdGlvbjwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4zLTE1PC9w
YWdlcz48dm9sdW1lPjIxPC92b2x1bWU+PG51bWJlcj4xPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIw
MDk8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRo
b3I+SHVzc2FpbjwvQXV0aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051bT4zODAzPC9SZWNOdW0+
PHJlY29yZD48cmVjLW51bWJlcj4zODAzPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh
cHA9IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zODAz
PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8
L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IdXNzYWluLCBTLjwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Gb29kIGZvciB0aG91
Z2h0IG9uIHRoZSBBQlMgYWNhZGVtaWMgam91cm5hbCBxdWFsaXR5IGd1aWRlPC90aXRsZT48c2Vj
b25kYXJ5LXRpdGxlPkFjY291bnRpbmcgRWR1Y2F0aW9uPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs
ZXM+PHBhZ2VzPjU0NS01NTk8L3BhZ2VzPjx2b2x1bWU+MjA8L3ZvbHVtZT48bnVtYmVyPjY8L251
bWJlcj48ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29y
ZD48L0NpdGU+PENpdGU+PEF1dGhvcj5IdXNzYWluPC9BdXRob3I+PFllYXI+MjAxMzwvWWVhcj48
UmVjTnVtPjQyMzI8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjQyMzI8L3JlYy1udW1iZXI+
PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJwOTBldnZ2ZHVwd3JkdmV3dHNycGF2
ZWN3ZTJyMncyYWRmZXAiPjQyMzI8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i
Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0
aG9yPkh1c3NhaW4sIFMuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+
PHRpdGxlPkpvdXJuYWwgRmV0aXNoaXNtIGFuZCB0aGUg4oCYU2lnbiBvZiB0aGUgNOKAmSBpbiB0
aGUgQUJTIGd1aWRlOiBBIFF1ZXN0aW9uIG9mIFRydXN0PzwvdGl0bGU+PHNlY29uZGFyeS10aXRs
ZT5Pcmdhbml6YXRpb248L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48dm9sdW1lPm9ubGluZTwv
dm9sdW1lPjxkYXRlcz48eWVhcj4yMDEzPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjxlbGVj
dHJvbmljLXJlc291cmNlLW51bT5kb2k6IDEwLjExNzcvMTM1MDUwODQxMzUwNjc2MyA8L2VsZWN0
cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk1vcnJpczwv
QXV0aG9yPjxZZWFyPjIwMDk8L1llYXI+PFJlY051bT4zNTY5PC9SZWNOdW0+PHJlY29yZD48cmVj
LW51bWJlcj4zNTY5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1p
ZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zNTY5PC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Nb3JyaXMsIE0uPC9hdXRob3I+PGF1dGhvcj5I
YXJ2ZXksIEMuPC9hdXRob3I+PGF1dGhvcj5LZWxseSwgQS48L2F1dGhvcj48L2F1dGhvcnM+PC9j
b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+Sm91cm5hbCByYW5raW5ncyBhbmQgdGhlIEFCUyBK
b3VybmFsIFF1YWxpdHkgR3VpZGU8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+TWFuYWdlbWVudCBE
ZWNpc2lvbjwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4xNDQxLTE0NTE8L3BhZ2Vz
Pjx2b2x1bWU+NDc8L3ZvbHVtZT48bnVtYmVyPjk8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwOTwv
eWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5X
aWxsbW90dDwvQXV0aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051bT4zNzk5PC9SZWNOdW0+PHJl
Y29yZD48cmVjLW51bWJlcj4zNzk5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9
IkVOIiBkYi1pZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj4zNzk5PC9r
ZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3Jl
Zi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XaWxsbW90dCwgSHVnaDwvYXV0
aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5Kb3VybmFsIGxpc3Qg
ZmV0aXNoaXNtIGFuZCB0aGUgcGVydmVyc2lvbiBvZiBzY2hvbGFyc2hpcDogcmVhY3Rpdml0eSBh
bmQgdGhlIEFCUyBsaXN0PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk9yZ2FuaXphdGlvbjwvc2Vj
b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz40MjktNDQyPC9wYWdlcz48dm9sdW1lPjE4PC92
b2x1bWU+PG51bWJlcj40PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PHB1Yi1kYXRl
cz48ZGF0ZT5KdWx5IDEsIDIwMTE8L2RhdGU+PC9wdWItZGF0ZXM+PC9kYXRlcz48dXJscz48cmVs
YXRlZC11cmxzPjx1cmw+aHR0cDovL29yZy5zYWdlcHViLmNvbS9jb250ZW50LzE4LzQvNDI5LmFi
c3RyYWN0PC91cmw+PC9yZWxhdGVkLXVybHM+PC91cmxzPjxlbGVjdHJvbmljLXJlc291cmNlLW51
bT4xMC4xMTc3LzEzNTA1MDg0MTE0MDM1MzI8L2VsZWN0cm9uaWMtcmVzb3VyY2UtbnVtPjwvcmVj
b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPk1pbmdlcnM8L0F1dGhvcj48WWVhcj4yMDEzPC9ZZWFy
PjxSZWNOdW0+MzgwMjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MzgwMjwvcmVjLW51bWJl
cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InA5MGV2dnZkdXB3cmR2ZXd0c3Jw
YXZlY3dlMnIydzJhZGZlcCI+MzgwMjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1l
PSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxh
dXRob3I+TWluZ2VycywgSi48L2F1dGhvcj48YXV0aG9yPldpbGxtb3R0LCBILjwvYXV0aG9yPjwv
YXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5UYXlsb3JpemluZyBidXNpbmVz
cyBzY2hvb2wgcmVzZWFyY2g6IE9uIHRoZSAmcXVvdDtvbmUgYmVzdCB3YXkmcXVvdDsgcGVyZm9y
bWF0aXZlIGVmZmVjdHMgb2Ygam91cm5hbCByYW5raW5nIGxpc3RzPC90aXRsZT48c2Vjb25kYXJ5
LXRpdGxlPkh1bWFuIFJlbGF0aW9uczwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwYWdlcz4x
MDUxLTEwNzM8L3BhZ2VzPjx2b2x1bWU+NjY8L3ZvbHVtZT48bnVtYmVyPjg8L251bWJlcj48ZGF0
ZXM+PHllYXI+MjAxMzwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+
PC9FbmROb3RlPgB=
ADDIN EN.CITE.DATA (Hoepner & Unerman, 2009; Hussain, 2011; Hussain, 2013; Mingers & Willmott, 2013; Morris et al., 2009; Willmott, 2011). The main criticisms of the ABS list are: first, that the 4* journals are dominated by traditional, US-operated, largely positivistic, journals at the expense of more eclectic and innovative European and non-US ones. Second, that the distribution of 4* journals across fields is highly unequal – 42% of psychology journals but less than 5% in fields such as operations management, operational research and information systems/management (IS/IM), and none in ethics/government and management education. Third, that the coverage of journals across fields was dominated by reference disciplines (psychology, economics and social science accounted for 30% of the list). For these reasons, it is valuable to compare rankings based on indicators with the ABS rankings.Standard TitleFieldABS 2009 1* to 4*Sum rankedBrookings Papers on Economic ActivityECON327OmegaOR&MANSCI328Long Range PlanningSTRAT333International Journal of Management ReviewsGEN MAN336Journal of Labor EconomicsECON339Journal of Human ResourcesECON341Tourism ManagementTOUR-HOSP343European Journal of Operational ResearchOR&MANSCI344Journal of Service ResearchSECTOR345Transportation Research Part E: Logistics and Transportation ReviewSECTOR347TechnovationINNOV249Journal of Information TechnologyINFO MAN350Human Resource Management ReviewHRM&EMP254Family Business ReviewENT-SMBUS255Decision Support SystemsINFO MAN356Information and ManagementINFO MAN357Journal of World BusinessIB&AREA360International Journal of Research in MarketingMKT361Journal of Management Information SystemsINFO MAN363Business Ethics QuarterlyETH-GOV366Journal of Strategic Information SystemsINFO MAN367Journal of Supply Chain ManagementOPS & TECH168Journal of Policy Analysis and ManagementPUB SEC369Supply Chain ManagementOPS & TECH370Table 6 Journals with High Indicator Values but Low ABS RankTable 6 shows journals that score highly in terms of indicators, “Sum Ranked”, but are not evaluated as 4* within ABS. The “Sum Ranked” column shows their position in the ranking of the sum of ranks. In the main these are ABS 3*, but Technovation is a 2* and the J. of Supply Chain Management only considered a 1*. In terms of the fields represented, we can see two from OR, five from IS/IM, and two from operations management which backs up the criticisms mentioned above.Standard TitleFieldABS 2009 1* to 4*SNIP RankSum rankedBusiness HistoryBUS HIST4433367IEEE Transactions on Engineering ManagementINFO MAN4267179Industrial RelationsHRM&EMP4251159British Journal of Industrial RelationsHRM&EMP4138148Public Administration ReviewPUB SEC4241138International Journal of Industrial OrganizationECON4165135Work, Employment and SocietyHRM&EMP4123124Review of Accounting StudiesACCOUNT4177120Human Resource ManagementHRM&EMP4196112Journal of Risk and UncertaintySOC SCI4231107Journal of RetailingMKT413096British Journal of ManagementGEN MAN411679Journal of Product Innovation ManagementINNOV46764Journal of Vocational BehaviorHRM&EMP411759Annals of Tourism ResearchTOUR-HOSP47058Human RelationsORG STUD49052Leadership QuarterlyORG STUD411051Journal of Occupational and Organizational PsychologyPSYCH49548Organization StudiesORG STUD410542Accounting, Organizations and SocietyACCOUNT44240Journal of Public Administration Research and TheoryPUB SEC44938Journal of Financial and Quantitative AnalysisFINANCE44735Entrepreneurship Theory and PracticeENT-SMBUS43832Organizational Behavior and Human Decision ProcessesPSYCH49431Table 7 Journals with Low Indicator Values but High ABS RankTable 7 shows the opposite, namely journals ranked as 4* in ABS but being relatively lowly ranked in terms of indicators. This Table does not show a preponderance of US journals or dominance of fields like Psychology. In fact, most of the journals are non-US and several come from a particular field, HR (5) and organization studies (3). To check whether this was a problem with low density fields the Table also show the SNIP rank (which aims to correct for this) but in the main the SNIP ranks are even worse. Business History is a particular outlier being ranked only 367th. FieldMean ABS scoreRank of ABS scoreMean Sum RankedPSYCH Average2.712110.14OR&MANSCI Average2.751190.50PUB SEC Average2.604202.10INFO MAN Average2.3510232.60HRM&EMP Average2.643239.79GEN MAN Average2.417259.33OPS MGT & TECH Average1.8819262.31ORG STUD Average2.389269.38SOC SCI Average2.2911272.53ECON Average2.2812272.80INNOV Average2.0016286.25FINANCE Average2.505293.73ENT-SMBUS Average2.2113294.13TOUR-HOSP Average1.9018299.80ETH-GOV Average1.8020301.90STRAT Average2.2014310.60SECTOR Average1.5523350.36MKT Average1.9517353.88ACCOUNT Average2.418391.18LAW Average2.0015404.00IB&AREA Average1.7221405.22MGDEV&ED Average1.7122440.93BUS HIST Average2.436523.43Table 8 Comparing Fields in ABS and Indicator RankingIn Table 8 we look at the fields. The second and fourth columns show the mean scores per field for the ABS grade and the sum ranked respectively. The Table is sorted in terms of sum ranked but the third column shows the ranks of the ABS mean. The rank correlation is 0.61. We can see that fields like information management and operations management do poorly in ABS while business history, accounting and finance do relatively well. This result agrees with other research concerning these fields. Templeton and Lewis ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2015</Year><RecNum>5098</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>5098</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">5098</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Templeton, Gary F</author><author>Lewis, Bruce R</author></authors></contributors><titles><title>Fairness in the institutional valuation of business journals</title><secondary-title>MIS Quarterly</secondary-title></titles><pages>523-539</pages><volume>39</volume><number>3</number><dates><year>2015</year></dates><isbn>0276-7783</isbn><urls></urls></record></Cite></EndNote>(2015) compared the prestige of B&M journals within AACSB business schools in terms of how highly they were valued, based on surveys of the Schools, compared with their citation performance based on a range of metrics (similar to ours). They found that information systems especially, but also operations management and quantitative methods were undervalued in comparison with the citation performance of the subjects’ journals. Valacich et al ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Year>2006</Year><RecNum>5060</RecNum><DisplayText>(2006)</DisplayText><record><rec-number>5060</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">5060</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Valacich, Joseph S.</author><author>Fuller, Mark A.</author><author>Schneider, Christoph</author><author>Dennis, Alan R.</author></authors></contributors><titles><title>Publication Opportunities in Premier Business Outlets: How Level Is the Playing Field?</title><secondary-title>Information Systems Research</secondary-title></titles><pages>107-125</pages><volume>17</volume><number>2</number><keywords><keyword>ACCOUNTING</keyword><keyword>FINANCE</keyword><keyword>MANAGEMENT</keyword><keyword>MARKETING</keyword><keyword>INFORMATION resources management</keyword><keyword>AUTHORS</keyword><keyword>PERIODICALS</keyword><keyword>academic discipline</keyword><keyword>academic promotion</keyword><keyword>publication</keyword><keyword>research journals</keyword><keyword>scientometrics</keyword><keyword>tenure</keyword></keywords><dates><year>2006</year></dates><publisher>INFORMS: Institute for Operations Research</publisher><isbn>10477047</isbn><accession-num>21656408</accession-num><urls><related-urls><url>;(2006) found that the publications opportunities in top journals was limited for IS researchers, i.e., there were relatively few IS journals considered to be elite, and they published relatively few papers. Lowry et al PEVuZE5vdGU+PENpdGUgRXhjbHVkZUF1dGg9IjEiPjxZZWFyPjIwMTM8L1llYXI+PFJlY051bT41
MDYxPC9SZWNOdW0+PERpc3BsYXlUZXh0PigyMDEzKTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVj
LW51bWJlcj41MDYxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1p
ZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj41MDYxPC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Mb3dyeSwgUGF1bCBCZW5qYW1pbjwvYXV0aG9y
PjxhdXRob3I+TW9vZHksIEdyZWdvcnkgRC48L2F1dGhvcj48YXV0aG9yPkdhc2tpbiwgSmFtZXM8
L2F1dGhvcj48YXV0aG9yPkdhbGxldHRhLCBEZW5uaXMgRi48L2F1dGhvcj48YXV0aG9yPkh1bXBo
ZXJ5cywgU2VhbiBMLjwvYXV0aG9yPjxhdXRob3I+QmFybG93LCBKb3JkYW4gQi48L2F1dGhvcj48
YXV0aG9yPldpbHNvbiwgRGF2aWQgVy48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+RXZhbHVhdGluZyBqb3VybmFsIHF1YWxpdHkgYW5kIHRoZSBBc3NvY2lh
dGlvbiBmb3IgSW5mb3JtYXRpb24gU3lzdGVtcyBTZW5pb3IgU2Nob2xhcnMmYXBvczsgam91cm5h
bCBiYXNrZXQgdmlhIGJpYmxpb21ldHJpYyBtZWFzdXJlczogRG8gZXhwZXJ0IGpvdXJuYWwgYXNz
ZXNzbWVudHMgYWRkIHZhbHVlPzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5NSVMgcXVhcnRlcmx5
PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjk5My1BMjE8L3BhZ2VzPjx2b2x1bWU+
Mzc8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+SU5GT1JNQVRJ
T04gc3RvcmFnZSAmYW1wOyByZXRyaWV2YWwgc3lzdGVtczwva2V5d29yZD48a2V5d29yZD5NQU5B
R0VNRU5UIGluZm9ybWF0aW9uIHN5c3RlbXM8L2tleXdvcmQ+PGtleXdvcmQ+U0NJRU5UT01FVFJJ
Q1M8L2tleXdvcmQ+PGtleXdvcmQ+SU1QQUNUIGZhY3RvciAoQ2l0YXRpb24gYW5hbHlzaXMpPC9r
ZXl3b3JkPjxrZXl3b3JkPlBFUklPRElDQUxTPC9rZXl3b3JkPjxrZXl3b3JkPkpPVVJOQUwgcHJv
ZHVjdGl2aXR5PC9rZXl3b3JkPjxrZXl3b3JkPkJJQkxJT01FVFJJQ1M8L2tleXdvcmQ+PGtleXdv
cmQ+QUlTIFNlbmlvciBTY2hvbGFycyBiYXNrZXQgb2Ygam91cm5hbHM8L2tleXdvcmQ+PGtleXdv
cmQ+Y29tcG9zaXRlIHJhbmtpbmcgb3IgcmF0aW5nPC9rZXl3b3JkPjxrZXl3b3JkPmV4cGVydCBv
cGluaW9uPC9rZXl3b3JkPjxrZXl3b3JkPmgtaW5kZXg8L2tleXdvcmQ+PGtleXdvcmQ+aW1wYWN0
IGZhY3Rvcjwva2V5d29yZD48a2V5d29yZD5JbmZvcm1hdGlvbiBzeXN0ZW1zIGpvdXJuYWwgcmFu
a2luZ3M8L2tleXdvcmQ+PGtleXdvcmQ+am91cm5hbCBxdWFsaXR5PC9rZXl3b3JkPjxrZXl3b3Jk
Pm5vbW9sb2dpZXMgZm9yIGRpc3NlbWluYXRpb24gb2Ygc2NpZW50aWZpYyBrbm93bGVkZ2U8L2tl
eXdvcmQ+PGtleXdvcmQ+c2VsZi1jaXRhdGlvbjwva2V5d29yZD48a2V5d29yZD5TZW5TLTY8L2tl
eXdvcmQ+PGtleXdvcmQ+U2VuUy04PC9rZXl3b3JkPjxrZXl3b3JkPnNvY2lhbCBuZXR3b3JrIGFu
YWx5c2lzPC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTM8L3llYXI+PC9kYXRl
cz48cHVibGlzaGVyPk1JUyBRdWFydGVybHk8L3B1Ymxpc2hlcj48aXNibj4wMjc2Nzc4MzwvaXNi
bj48YWNjZXNzaW9uLW51bT45MTkwNjI0MTwvYWNjZXNzaW9uLW51bT48dXJscz48cmVsYXRlZC11
cmxzPjx1cmw+aHR0cDovL3NlYXJjaC5lYnNjb2hvc3QuY29tL2xvZ2luLmFzcHg/ZGlyZWN0PXRy
dWUmYW1wO2RiPWJ0aCZhbXA7QU49OTE5MDYyNDEmYW1wO3NpdGU9ZWhvc3QtbGl2ZTwvdXJsPjwv
cmVsYXRlZC11cmxzPjwvdXJscz48cmVtb3RlLWRhdGFiYXNlLW5hbWU+YnRoPC9yZW1vdGUtZGF0
YWJhc2UtbmFtZT48cmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVyPkVCU0NPaG9zdDwvcmVtb3RlLWRh
dGFiYXNlLXByb3ZpZGVyPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE PEVuZE5vdGU+PENpdGUgRXhjbHVkZUF1dGg9IjEiPjxZZWFyPjIwMTM8L1llYXI+PFJlY051bT41
MDYxPC9SZWNOdW0+PERpc3BsYXlUZXh0PigyMDEzKTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVj
LW51bWJlcj41MDYxPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1p
ZD0icDkwZXZ2dmR1cHdyZHZld3RzcnBhdmVjd2UycjJ3MmFkZmVwIj41MDYxPC9rZXk+PC9mb3Jl
aWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxj
b250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Mb3dyeSwgUGF1bCBCZW5qYW1pbjwvYXV0aG9y
PjxhdXRob3I+TW9vZHksIEdyZWdvcnkgRC48L2F1dGhvcj48YXV0aG9yPkdhc2tpbiwgSmFtZXM8
L2F1dGhvcj48YXV0aG9yPkdhbGxldHRhLCBEZW5uaXMgRi48L2F1dGhvcj48YXV0aG9yPkh1bXBo
ZXJ5cywgU2VhbiBMLjwvYXV0aG9yPjxhdXRob3I+QmFybG93LCBKb3JkYW4gQi48L2F1dGhvcj48
YXV0aG9yPldpbHNvbiwgRGF2aWQgVy48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+
PHRpdGxlcz48dGl0bGU+RXZhbHVhdGluZyBqb3VybmFsIHF1YWxpdHkgYW5kIHRoZSBBc3NvY2lh
dGlvbiBmb3IgSW5mb3JtYXRpb24gU3lzdGVtcyBTZW5pb3IgU2Nob2xhcnMmYXBvczsgam91cm5h
bCBiYXNrZXQgdmlhIGJpYmxpb21ldHJpYyBtZWFzdXJlczogRG8gZXhwZXJ0IGpvdXJuYWwgYXNz
ZXNzbWVudHMgYWRkIHZhbHVlPzwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5NSVMgcXVhcnRlcmx5
PC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjk5My1BMjE8L3BhZ2VzPjx2b2x1bWU+
Mzc8L3ZvbHVtZT48bnVtYmVyPjQ8L251bWJlcj48a2V5d29yZHM+PGtleXdvcmQ+SU5GT1JNQVRJ
T04gc3RvcmFnZSAmYW1wOyByZXRyaWV2YWwgc3lzdGVtczwva2V5d29yZD48a2V5d29yZD5NQU5B
R0VNRU5UIGluZm9ybWF0aW9uIHN5c3RlbXM8L2tleXdvcmQ+PGtleXdvcmQ+U0NJRU5UT01FVFJJ
Q1M8L2tleXdvcmQ+PGtleXdvcmQ+SU1QQUNUIGZhY3RvciAoQ2l0YXRpb24gYW5hbHlzaXMpPC9r
ZXl3b3JkPjxrZXl3b3JkPlBFUklPRElDQUxTPC9rZXl3b3JkPjxrZXl3b3JkPkpPVVJOQUwgcHJv
ZHVjdGl2aXR5PC9rZXl3b3JkPjxrZXl3b3JkPkJJQkxJT01FVFJJQ1M8L2tleXdvcmQ+PGtleXdv
cmQ+QUlTIFNlbmlvciBTY2hvbGFycyBiYXNrZXQgb2Ygam91cm5hbHM8L2tleXdvcmQ+PGtleXdv
cmQ+Y29tcG9zaXRlIHJhbmtpbmcgb3IgcmF0aW5nPC9rZXl3b3JkPjxrZXl3b3JkPmV4cGVydCBv
cGluaW9uPC9rZXl3b3JkPjxrZXl3b3JkPmgtaW5kZXg8L2tleXdvcmQ+PGtleXdvcmQ+aW1wYWN0
IGZhY3Rvcjwva2V5d29yZD48a2V5d29yZD5JbmZvcm1hdGlvbiBzeXN0ZW1zIGpvdXJuYWwgcmFu
a2luZ3M8L2tleXdvcmQ+PGtleXdvcmQ+am91cm5hbCBxdWFsaXR5PC9rZXl3b3JkPjxrZXl3b3Jk
Pm5vbW9sb2dpZXMgZm9yIGRpc3NlbWluYXRpb24gb2Ygc2NpZW50aWZpYyBrbm93bGVkZ2U8L2tl
eXdvcmQ+PGtleXdvcmQ+c2VsZi1jaXRhdGlvbjwva2V5d29yZD48a2V5d29yZD5TZW5TLTY8L2tl
eXdvcmQ+PGtleXdvcmQ+U2VuUy04PC9rZXl3b3JkPjxrZXl3b3JkPnNvY2lhbCBuZXR3b3JrIGFu
YWx5c2lzPC9rZXl3b3JkPjwva2V5d29yZHM+PGRhdGVzPjx5ZWFyPjIwMTM8L3llYXI+PC9kYXRl
cz48cHVibGlzaGVyPk1JUyBRdWFydGVybHk8L3B1Ymxpc2hlcj48aXNibj4wMjc2Nzc4MzwvaXNi
bj48YWNjZXNzaW9uLW51bT45MTkwNjI0MTwvYWNjZXNzaW9uLW51bT48dXJscz48cmVsYXRlZC11
cmxzPjx1cmw+aHR0cDovL3NlYXJjaC5lYnNjb2hvc3QuY29tL2xvZ2luLmFzcHg/ZGlyZWN0PXRy
dWUmYW1wO2RiPWJ0aCZhbXA7QU49OTE5MDYyNDEmYW1wO3NpdGU9ZWhvc3QtbGl2ZTwvdXJsPjwv
cmVsYXRlZC11cmxzPjwvdXJscz48cmVtb3RlLWRhdGFiYXNlLW5hbWU+YnRoPC9yZW1vdGUtZGF0
YWJhc2UtbmFtZT48cmVtb3RlLWRhdGFiYXNlLXByb3ZpZGVyPkVCU0NPaG9zdDwvcmVtb3RlLWRh
dGFiYXNlLXByb3ZpZGVyPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+
ADDIN EN.CITE.DATA (2013) compared expert opinion (from a survey) and bibliometric measures for IS journals only and found that the results were extremely similar so that suggested that metrics should be used instead in the future. The overall conclusions of this section is that there are systematic discrepancies between the ABS list and rankings based on citation indicators which leads to questions about what justification there is for these differences. ConclusionsThis paper has considered the main journal impact indicators that are currently available through citation databases as these are the primary ones that are used in practice for decisions about journal ranking lists, destinations for research papers, jobs, promotions, and submissions to research evaluation programmes. There are several general issues to be noted in terms of the appropriate use of these metrics. First, citation data is always highly skewed and this calls into question the validity of measures based on parameters such as the mean, especially for the evaluation of individual cases of journals or researchers. Second, there is the ecological fallacy of making judgements about individuals on the bases of whole population characteristics. For example, judging the quality of individual papers purely in terms of the journal they are published in, or judging individuals in terms of particular journals when they publish across a range, potentially in different fields. Third, there is the whole issue of using the number of citations, especially in the form of short term impact factors, as a measure of journal quality anyway, certainly in the social sciences.In comparing these particular metrics, we have found that, at first sight, they appear to be highly correlated and that this may lead users to believe that they are in fact very similar in their results and that it does not, perhaps, matter too much which ones are used. But, in fact, these correlations mask significant differences between them, both theoretically and empirically, and these differences can have major effects on the rankings of individual journals. Given the extensive use of journal ranking lists and journal metrics in research evaluation, and the consequences this can have on departments and individuals, it is important that these effects are recognised and factored into any decisions being made. The differences occur because of the inevitable biases in any form of metric dependent on the particular underlying assumptions and manner of its calculation. The main theoretical differences between the indicators are: whether they normalise for the number of papers generating the citations, and the subject area or field; whether they take into consideration the prestige of the citing journals; whether they are affected by skewed data; whether they are transparent, easily interpretable and robust to poor data; whether they are reliant on a particular proprietary database; and whether they are transparent and reproducible by other researchers. These theoretical differences were largely corroborated in the empirical comparisons. We also compared rankings formed on the basis of the citation metrics with a well-known journal list that is used extensively within research assessments. Many instances were found where journals that performed well in terms of citations were ranked relatively lowly and journals that were ranked highly had little citation impact. In terms of practical recommendations, we do not feel that any one indicator stands out as superior at this time, they all have their limitations. Equally, however, peer review and expert journal lists are subjective and biased in many ways. We feel therefore that the best approach is to employ several metrics along with peer review if it is really felt necessary to produce ranked lists of journals but then to exercise great caution in inferring from the general high-level results down to the performance of individuals. If we were to recommend any metrics, we would suggest SNIP, which normalises for papers and also field (although this was not very evident on our data), and the h-index which is transparent, easy to understand, and robust to poor data thus being especially useful with Google Scholar. This particular study does have significant limitations: it was conducted only within one disciplinary field, business and management, although that is a very diverse field which displays many of the characteristics of social science as a whole. It was also limited in terms of the number of journals that could actually be included in the final analyses because of limitations in some of the data sources. We feel that further research is needed, particular in the following areas: i) Large scale tests of different forms of normalisation, both citing- and cited-side, and a priori (that is, adjusting the citations before an indicator is calculated) and a posteriori (adjusting the indicator after it is calculated) ADDIN EN.CITE <EndNote><Cite><Author>Gl?nzel</Author><Year>2011</Year><RecNum>4485</RecNum><DisplayText>(Gl?nzel et al., 2011)</DisplayText><record><rec-number>4485</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4485</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Gl?nzel, Wolfgang</author><author>Schubert, András</author><author>Thijs, Bart</author><author>Debackere, Koenraad</author></authors></contributors><titles><title>A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking</title><secondary-title>Scientometrics</secondary-title></titles><pages>415-424</pages><volume>87</volume><number>2</number><dates><year>2011</year></dates><isbn>0138-9130</isbn><urls></urls></record></Cite></EndNote>(Gl?nzel et al., 2011). ii) Investigating ways of normalizing Google Scholar data and improving its quality. iii) Investigating the possibilities of creating weighted aggregated indices that might overcome the limitations of any particular one ADDIN EN.CITE <EndNote><Cite><Author>Ennas</Author><Year>2015</Year><RecNum>4825</RecNum><DisplayText>(Ennas et al., 2015)</DisplayText><record><rec-number>4825</rec-number><foreign-keys><key app="EN" db-id="p90evvvdupwrdvewtsrpavecwe2r2w2adfep">4825</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Ennas, Gianfranco</author><author>Biggio, Battista</author><author>Di Guardo, MariaChiara</author></authors></contributors><titles><title>Data-driven journal meta-ranking in business and management</title><secondary-title>Scientometrics</secondary-title><alt-title>Scientometrics</alt-title></titles><pages>1-19</pages><keywords><keyword>Journal quality evaluation</keyword><keyword>Combining journal ranking indices</keyword><keyword>Meta-indices</keyword></keywords><dates><year>2015</year><pub-dates><date>2015/09/28</date></pub-dates></dates><publisher>Springer Netherlands</publisher><isbn>0138-9130</isbn><urls><related-urls><url>;(Ennas et al., 2015). iv) Investigating indicators that are not currently supported by WoS or Scopus such as I3 which avoids the problem of skewness. References ADDIN EN.REFLIST Aad, G., Abajyan, T., Abbott, B., Abdallah, J., Abdel Khalek, S., Abdelalim, A. A., . . . Zwalinski, L. (2012). Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC. Physics Letters B, 716(1), 1-29.Abramo, G., & D’Angelo, C. (2011). Evaluating research: from informed peer review to bibliometric. Scientometrics, 87(3), 499-514.Adler, N., & Harzing, A.-W. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning and Education, 8(1), 72-95.Adriaanse, L., & Rensleigh, C. (2013). Web of Science, Scopus and Google Scholar. The Electronic Library, 31(6), 727-744.Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273-289.Amara, N., & Landry, R. (2012). Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science. Scientometrics, 93(3), 553-581.Association of Business Schools. (2010, 1/3/2010). Academic journal quality guide, from , C. (2007). Measuring the value and prestige of scholarly journals. College and Research Library News, 68(5), 3146.Bollen, J., Rodriquez, M. A., & Van de Sompel, H. (2006). Journal status. Scientometrics, 69(3), 669-687.Bornman, L., & Daniel, H.-D. (2005). Does the h-index for ranking of scientists really work? Scientometrics, 65(3), 391-392.Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381-1385.Bornmann, L., Leydesdorff, L., & Mutz, R. (2013). The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits. Journal of Informetrics, 7(1), 158-165.Bornmann, L., & Marx, W. (2015). Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgement of experts? Journal of Informetrics, 9(2), 408-418.Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830-837.Bornmann, L., Thor, A., Marx, W., & Schier, H. (2016). The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute. Journal of the Association for Information Science and Technology, n/a-n/a.Brumback, R. (2008). Worshipping false idols: the impact factor dilemma. Journal Child Neurology, 23, 365-367.Campanario, J. M. (2011). Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window. Scientometrics, 87(1), 189-204.Chartered Association of Business Schools. (2015). Academic journal guide 2015, from , R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level. Journal of Informetrics, 1, 193-203.Cronin, B. (2001). Hyperauthorship: A postmodern perversion or evidence of a structural shift in scholarly communication practices? Journal of the American Society for Information Science and Technology, 52(7), 558-569.Cronin, B., & Sugimoto, C. (Eds.). (2014). Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. London: MIT Press.Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69(1), 131-152.Ennas, G., Biggio, B., & Di Guardo, M. (2015). Data-driven journal meta-ranking in business and management. Scientometrics, 1-19.Evidence Ltd. (2007). The use of bibliometrics to measure research quality in UK higher education institutions. Leeds: Evidence Ltd. , M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of SCImago journal rank indicator with journal impact factor. The FASEB Journal, 22(8), 2623-2628.Fersht, A. (2009). The most influential journals: Impact Factor and Eigenfactor. Proceedings of the National Academy of Sciences, 106(17), 6883-6884.Franceschet, M. (2010). A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar. Scientometrics, 83(1), 243-258.Franceschini, F., & Maisano, D. A. (2010). Analysis of the Hirsch index’s operational properties. European Journal of Operational Research, 203(2), 494-504.García-Pérez, M. A. (2010). Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology. Journal of the American Society for Information Science and Technology, 61(10), 2070-2085.Garfield, E. (1955). Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas. Science, 122(3159), 108-111.Garfield, E., & Sher, I. H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195-201.Giles, L., & Khabsa, M. (2014). The Number of scholarly documents on the Web. PLoS ONE, 9(5), e93949.Gl?nzel, W. (2006). On the h-index - a mathematical approach to a new measure of publication activity and citation impact. Scientometrics, 67(2), 315-321.Gl?nzel, W., & Moed, H. K. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171-193.Gl?nzel, W., Schubert, A., Thijs, B., & Debackere, K. (2011). A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking. Scientometrics, 87(2), 415-424.González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of Informetrics, 4(3), 379-391.Gross, P., & Gross, E. (1927). College libraries and chemical education. Science 66(385-389).Guerrero-Bote, V. P., & Moya-Anegón, F. (2012). A further step forward in measuring journals’ scientific prestige: The SJR2 indicator. Journal of Informetrics, 6(4), 674-688.Hall, C. (2011). Publish and perish? Bibliometric analysis, journal ranking and the assessment of research quality in tourism. Tourism Management, 32(1), 16-27.Harzing, A.-W. (2009). Journal Quality List 55th. Retrieved January, 2015, from , A.-W., & van der Wal, R. (2008). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8, 61-73.Harzing, A.-W., & Van der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business? J. Am. Soc. for Information Science and Technology, 60(1), 41-46.HEFCE. (2008). Counting what is measured or measuring what counts: HEFCE. , D. (2012). Performance-based university research funding systems. Research policy, 41(2), 251-261.Hirsch, J. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569-16572.Hoepner, A., & Unerman, J. (2009). Explicit and implicit subject bias in the ABS Journal Quality Guide. Accounting Education, 21(1), 3-15.Hussain, S. (2011). Food for thought on the ABS academic journal quality guide. Accounting Education, 20(6), 545-559.Hussain, S. (2013). Journal Fetishism and the ‘Sign of the 4’ in the ABS guide: A Question of Trust? Organization, online.Iglesias, J., & Pecharromán, C. (2007). Scaling the h-index for different scientific ISI fields. Scientometrics, 73(3), 303-320.Jacso, P. (2005). As we may search - Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9), 1537-1547.Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855-863.Lancho-Barrantes, B. S., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). What lies behind the averages and significance of citation indicators in different disciplines? Journal of Information Science, 36(3), 371-382.Larivière, V., Archambault, ?., Gingras, Y., & Vignola-Gagné, ?. (2006). The place of serials in referencing practices: Comparing natural sciences and engineering with social sciences and humanities. Journal of the American Society for Information Science and Technology, 57(8), 997-1004.Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2006). Measures for measures. [10.1038/4441003a]. Nature, 444(7122), 1003-1004.Leydesdorff, L. (2012). Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most highly cited papers. Scientometrics, 92, 355-365.Leydesdorff, L., & Bornmann, L. (2011). Integrated impact indicators compared with impact factors: An alternative research design with policy implications. Journal of the American Society for Information Science and Technology, 62(11), 2133-2146.Leydesdorff, L., & Bornmann, L. (2014). The Operationalization of "Fields" as WoS Subject Categories (WCs) in Evaluative Bibliometrics: The cases of "Library and Information Science" and "Science & Technology Studies". arXiv preprint arXiv:1407.7849. Retrieved from , L., Bornmann, L., Mutz, R., & Opthof, T. (2011). Turning the tables on citation analysis one more time: Principles for comparing sets of documents. Journal of the American Society for Information Science and Technology, 62(7), 1370-1381.Leydesdorff, L., de Moya-Anegón, F., & de Nooy, W. (2014). Aggregated journal-journal citation relations in Scopus and Web-of-Science matched and compared in terms of networks, maps, and interactive overlays. arXiv preprint arXiv:1404.2505.Leydesdorff, L., & Milojevi?, S. (2015). The Citation Impact of German Sociology Journals: Some Problems with the Use of Scientometric Indicators in Journal and Research Evaluations. Soziale Welt, 66(2), 193-204. , 66(2), 193-204.Leydesdorff, L., & Opthof, T. (2010). Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61(11), 2365-2369.Lowry, P. B., Moody, G. D., Gaskin, J., Galletta, D. F., Humpherys, S. L., Barlow, J. B., & Wilson, D. W. (2013). Evaluating journal quality and the Association for Information Systems Senior Scholars' journal basket via bibliometric measures: Do expert journal assessments add value? MIS quarterly, 37(4), 993-A921.Lowry, P. B., Romans, D., & Curtis, A. (2004). Global journal prestige and supporting disciplines: A scientometric study of information systems journals. Journal of the Association for Information Systems, 5(2), 29-75.Ma, N., Guan, J., & Zhao, Y. (2008). Bringing PageRank to the citation analysis. Information Processing and Management, 44, 800-810.Mahdi, S., D'Este, P., & Neely, A. (2008). Citation counts: Are they good predictors of RAE scores? London: AIM Research.Meho, L., & Rogers, Y. (2008). Citation counting, citation ranking, and h-index of human-computer interaction researchers: A comparison of Scopus and Web of Science. Journal American Society for Information Science and Technology, 59(11), 1711-1726.Meho, L., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science, Scopus and Google Scholar. Journal American Society for Information Science and Technology, 58(13), 2105-2125.Mingers, J. (2008). Measuring the research contribution of management academics using the Hirsch-index. Journal Operational Research Society, 60(8), 1143-1153.Mingers, J. (2014). Problems with SNIP. Journal of Informetrics, 8(4), 890-894.Mingers, J., & Leydesdorff, L. (2015a). Identifying research fields within business and management: A journal cross-citation analysis. Journal of the Operational Research Society, 66(8), 1370-1384.Mingers, J., & Leydesdorff, L. (2015b). A review of theory and practice in scientometrics. European Journal of Operational Research, 246(1), 1-19.Mingers, J., & Lipitakis, E. (2010). Counting the citations: A comparison of Web of Science and Google Scholar in the field of management. Scientometrics 85(2), 613-625.Mingers, J., Macri, F., & Petrovici, D. (2012). Using the h-index to measure the quality of journals in the field of Business and Management. Information Processing & Management, 48(2), 234-241.Mingers, J., & Willmott, H. (2013). Taylorizing business school research: On the "one best way" performative effects of journal ranking lists. Human Relations, 66(8), 1051-1073.Moed, H. (2000). Bibliometric indicators reflect publication and management strategies. Scientometrics, 47(2), 323-346.Moed, H. (2007). The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review. Science and Public Policy, 34(8), 575-583.Moed, H. (2010a). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265-277.Moed, H. (2010b). The Source-Normalized Impact per Paper (SNIP) is a valid and sophisticated indicator of journal citation impact. arXiv preprint. Retrieved from , H. (2015). Comprehensive indicator comparisons intelligible to non-experts: The case of two SNIP versions. Retrieved from , H., De Bruin, R., & Van Leeuwen, T. (1995). New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications. Scientometrics, 33(3), 381-422.Moed, H., & Visser, M. (2008). Appraisal of Citation Data Sources. Leiden: Centre for Science and Technology Studies, Leiden University. , H. F. (2010c). CWTS crown indicator measures citation impact of a research group's publication oeuvre. Journal of Informetrics, 4(3), 436-438.Moosa, I. (2011). The demise of the ARC journal ranking scheme: an ex post analysis of the accounting and finance journals. Accounting & Finance, 51(3), 809-836.Morris, M., Harvey, C., & Kelly, A. (2009). Journal rankings and the ABS Journal Quality Guide. Management Decision, 47(9), 1441-1451.Moussa, S., & Touzani, M. (2010). Ranking marketing journals using the Google Scholar-based hg-index. Journal of Informetrics, 4, 107-117.Nkomo, S. (2009). The seductive power of academic journal rankings. Academy of Management Learning and Education, 8(1), 106-112.Norris, M., & Oppenheim, C. (2010). The h‐index: a broad review of a new bibliometric indicatornull. Journal of Documentation, 66(5), 681-705.Northcott, D., & Linacre, S. (2010). Producing spaces for academic discourse:The impact of research assessment exercises and journal quality rankings. Australian Accounting Review, 20(1), 38-54.Oppenheim, C. (2007). Using the h-index to rank influential British researchers in Information Science and Librarianship. Journal of the American Society for Information Science and Technology, 58(2), 297-301.Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: Bringing order to the web.Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing & Management, 12(5), 297-312.Pislyakov, V. (2009). Comparing two “thermometers”: Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. Scientometrics, 79(3), 541-550.Rafols, I., Leydesdorff, L., O’Hare, A., Nightingale, P., & Stirling, A. (2012). How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business & management. Research Policy, 41(7), 1262-1282.Rebora, G., & Turri, M. (2013). The UK and Italian research assessment exercises face to face. Research policy, 42(9), 1657-1666.Saad, G. (2006). Exploring the h-index at the author and journal levels using bibliometric data of productive consumer scholars and business-related journals respectively. Scientometrics, 69(1), 117-120.Seglen, P. O. (1992). The skewness of science. Journal of the American Society for Information Science, 43(9), 628-638.Straub, D. (2006). The Value of Scientometric Studies: An Introduction to a Debate on IS as a Reference Discipline. Journal of the Association for Information Systems, 7(5), 241-245.Straub, D., & Anderson, C. (2010). Journal Quality and Citations: Common Metrics and Considerations about Their Use, Editorial, MIS Quarterly, pp. iii-xii. Retrieved from , G. F., & Lewis, B. R. (2015). Fairness in the institutional valuation of business journals. MIS Quarterly, 39(3), 523-539.Todeschini, R. (2011). The j-index: a new bibliometric index and mulitvariate comparisons between other bibliometric indices. Scientometrics, 87, 621-639.Truex III, D., Cuellar, M., & Takeda, H. (2009). Assessing scholarly influence: Using the Hirsch indices to reframe the discourse. Journal of the Association for Information Systems, 10(7), 560-594.Valacich, J. S., Fuller, M. A., Schneider, C., & Dennis, A. R. (2006). Publication Opportunities in Premier Business Outlets: How Level Is the Playing Field? Information Systems Research, 17(2), 107-125.van Leeuwen, T., & Moed, H. (2002). Development and application of journal impact measures in the Dutch science system. Scientometrics, 53(2), 249-266.van Raan, A. (2003). The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments. Technology Assessment - Theory and Practice, 1(12), 20-29.van Raan, A. (2005a). Comparison of the hirsch-index with standard bibliometric indicators and with peer judgement for 147 chemistry research groups. Scientometrics, 67(3), 491-502.van Raan, A. (2005b). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143.van Raan, A. (2005c). Measuring science: Capita selectaof current main issues. In H. Moed, W. Glenzel & U. Schmoch (Eds.), Handbook of Quantitative Science and Technology Research (Vol. 19-50). New York: Springer.Walker, J., Salter, A., & Salandra, R. (2015). Initial Findings from the Survey of UK Business Academics: Henley Business School. Waltman, L. (2015). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365-391.Waltman, L., & van Eck, N. (2013). A systematic empirical comparison of different approaches for normalizing citation impact indicators. Journal of Informetrics, 7(4), 833-849.Waltman, L., van Eck, N., van Leeuwen, T., & Visser, M. (2013). Some modifications to the SNIP journal impact indicator. Journal of Informetrics, 7(2), 272-285.West, J., Bergstrom, T., & Bergstrom, C. (2010). Big Macs and Eigenfactor scores: Don't let correlation coefficients fool you. Journal of the American Society for Information Science and Technology, 61(9), 1800-1807.Wilhite, A., & Fong, E. (2012). Coercive citation in academic publishing. Science, 335(6068), 542-543.Willmott, H. (2011). Journal list fetishism and the perversion of scholarship: reactivity and the ABS list. Organization, 18(4), 429-442.Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., & Franssen, T. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. London: HEFCE.Zitt, M. (2011). Behind citing-side normalization of citations: some properties of the journal impact factor. Scientometrics, 89(1), 329-344.Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856-1860. ................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- review of journal citation indicators uk
- on the relationship between the undisclosed number of
- refer to the link below for specific information on
- in text citations rmit university rmit university
- in text citations
- university of portland
- the nature of a scientific paper hendrix college
- abstracts for citations veterans affairs
Related searches
- journal citation reports 2019
- 2018 journal citation reports
- journal citation reports
- impact factor 2018 journal citation reports
- journal citation report
- journal citation reports 2019 excel
- journal citation reports 2020
- journal citation reports 2018 pdf
- journal citation reports 2019 clarivate
- journal citation reports 2019 pdf
- journal citation reports thomson reuters
- journal citation reports impact factor