Improving meta-analyses in sport and exercise science



SPORTSCIENCE · Perspectives / Research ResourcesImproving Meta-analyses in Sport and Exercise ScienceWill G HopkinsSportscience 22, 11-17, 2018 (2018/metaflaws.htm)Institute of Sport Exercise & Active Living, Victoria University, Melbourne, Australia. Email. Reviewer: Alan M Batterham, School of Health and Social Care, University of Teesside, Middlesbrough, UK.Flaws in recent meta-analyses in our disciplines include extraction of inappropriate measures or units from published studies, inadequate documentation of the standard errors used to weight the effects, deficiencies in accounting for heterogeneity in the effect between studies, inappropriate treatment of multiple measures from the same studies, absent or deficient analysis of publication bias, and failure to interpret the clinical or practical importance of the meta-analyzed mean and heterogeneity. More attention to these flaws by authors, reviewers and editors will raise the quality of meta-analyses accepted for publication. Until then researchers and practitioners should use published meta-analyses with caution, especially those lacking adequate analysis of heterogeneity: use such analyses primarily as convenient reference lists to find studies from which to draw conclusions about the effect in settings similar to your own. KEYWORDS: magnitude-based inference, mixed models, random-effect meta-regression, systematic reviews.Reprint?pdf?· Reprint?docx TOC \o "1-3" \h \z \u Introduction PAGEREF _Toc503810991 \h 11Expressing Effects in the Same Metric PAGEREF _Toc503810992 \h 12Dealing with Standard Errors PAGEREF _Toc503810993 \h 13Accounting for Heterogeneity PAGEREF _Toc503810994 \h 14Coping with Repeated Measurement PAGEREF _Toc503810995 \h 15Publication Bias and Outlier Studies PAGEREF _Toc503810996 \h 15Interpreting Magnitudes PAGEREF _Toc503810997 \h 16Conclusion PAGEREF _Toc503810998 \h 16References PAGEREF _Toc503810999 \h 17See also the article/slideshow on meta-analysis. The article/slideshow on the replication crisis also summarizes meta-analysis.IntroductionAs a reviewer of meta-analyses for several journals, I have noticed a worrying trend: authors of submitted manuscripts often justify their flawed analyses by citing published articles with similar flaws. My aim in this article is to identify flaws in several recent meta-analyses in the hope of raising the quality of submitted and published meta-analyses in our disciplines. Numerous flaws in a recent meta-analysis by Coquart et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(2016a) provided the stimulus for writing the article. I performed a search for meta-analyses published since 2011 over a wide range of topics to find more examples of errors and omissions PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IENodSBl

dCBhbC4sIDIwMTY7IEh1bWUgZXQgYWwuLCAyMDE1OyBKb3NlZnNzb24gZXQgYWwuLCAyMDE0OyBQ

ZXRlcnNvbiBldCBhbC4sIDIwMTE7IFNhbHZhZG9yIGV0IGFsLiwgMjAxNjsgU29vbXJvIGV0IGFs

LiwgMjAxNTsgVG9tbGluc29uIGV0IGFsLiwgMjAxNTsgV3JpZ2h0IGV0IGFsLiwgMjAxNSk8L0Rp

c3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5

cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5

cngiPjIwPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs

ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CdXJkZW4sIFJp

Y2hhcmQgSjwvYXV0aG9yPjxhdXRob3I+TW9ydG9uLCBLYXRpZTwvYXV0aG9yPjxhdXRob3I+Umlj

aGFyZHMsIFRvYnk8L2F1dGhvcj48YXV0aG9yPldoeXRlLCBHcmVnb3J5IFA8L2F1dGhvcj48YXV0

aG9yPlBlZGxhciwgQ2hhcmxlcyBSPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0

aXRsZXM+PHRpdGxlPklzIGlyb24gdHJlYXRtZW50IGJlbmVmaWNpYWwgaW4sIGlyb24tZGVmaWNp

ZW50IGJ1dCBub24tYW5hZW1pYyAoSUROQSkgZW5kdXJhbmNlIGF0aGxldGVzPyBBIG1ldGEtYW5h

bHlzaXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QnJpdGlzaCBqb3VybmFsIG9mIHNwb3J0cyBt

ZWRpY2luZTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxl

PkJyaXRpc2ggSm91cm5hbCBvZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5C

ci4gSi4gU3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPkJyIEogU3BvcnRzIE1lZDwvYWJici0y

PjwvcGVyaW9kaWNhbD48cGFnZXM+MTM4OS0xMzk3PC9wYWdlcz48dm9sdW1lPjQ5PC92b2x1bWU+

PGRhdGVzPjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4xNDczLTA0ODA8L2lzYm4+PHVy

bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNodTwvQXV0aG9yPjxZZWFy

PjIwMTY8L1llYXI+PFJlY051bT4yMzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjM8L3Jl

Yy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6

Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl

IG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhv

cnM+PGF1dGhvcj5DaHUsIEFubmE8L2F1dGhvcj48YXV0aG9yPlBldG9jeiwgUGV0ZXI8L2F1dGhv

cj48YXV0aG9yPlNhbW1hbiwgU2FtaXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+

PHRpdGxlcz48dGl0bGU+SW1tZWRpYXRlIGVmZmVjdHMgb2YgYWVyb2JpYyBleGVyY2lzZSBvbiBw

bGFzbWEvc2VydW0gemluYyBsZXZlbHM6IGEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFy

eS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9ydHMgYW5kIGV4ZXJjaXNlPC9zZWNv

bmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TWVkaWNpbmUgYW5k

IFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVsbC10aXRsZT48YWJici0xPk1lZC4g

U2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5NZWQgU2NpIFNwb3J0IEV4ZXJjaXNl

PC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz43MjYtNzMz

PC9wYWdlcz48dm9sdW1lPjQ4PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMTY8L3llYXI+PC9kYXRl

cz48aXNibj4wMTk1LTkxMzE8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0

ZT48QXV0aG9yPkh1bWU8L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+MTI8L1JlY051

bT48cmVjb3JkPjxyZWMtbnVtYmVyPjEyPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh

cHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4xMjwv

a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9y

ZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SHVtZSwgUGF0cmlhIEE8L2F1

dGhvcj48YXV0aG9yPkxvcmltZXIsIEFubmEgVjwvYXV0aG9yPjxhdXRob3I+R3JpZmZpdGhzLCBQ

ZXRlciBDPC9hdXRob3I+PGF1dGhvcj5DYXJsc29uLCBJc2FhYzwvYXV0aG9yPjxhdXRob3I+TGFt

b250LCBNaWtlPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl

PlJlY3JlYXRpb25hbCBzbm93LXNwb3J0cyBpbmp1cnkgcmlzayBmYWN0b3JzIGFuZCBjb3VudGVy

bWVhc3VyZXM6IGEgbWV0YS1hbmFseXNpcyByZXZpZXcgYW5kIEhhZGRvbiBtYXRyaXggZXZhbHVh

dGlvbjwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3NlY29uZGFyeS10

aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L2Z1

bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3BvcnQgTWVkPC9h

YmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+MTE3NS0xMTkwPC9w

YWdlcz48dm9sdW1lPjQ1PC92b2x1bWU+PG51bWJlcj44PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIw

MTU8L3llYXI+PC9kYXRlcz48aXNibj4wMTEyLTE2NDI8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVj

b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkpvc2Vmc3NvbjwvQXV0aG9yPjxZZWFyPjIwMTQ8L1ll

YXI+PFJlY051bT4xNzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTc8L3JlYy1udW1iZXI+

PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdz

Zng5YXo5eDB2ejA5cngiPjE3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9Ikpv

dXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhv

cj5Kb3NlZnNzb24sIFRvcmJqw7ZybjwvYXV0aG9yPjxhdXRob3I+TGluZHdhbGwsIE1hZ251czwv

YXV0aG9yPjxhdXRob3I+QXJjaGVyLCBUcmV2b3I8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmli

dXRvcnM+PHRpdGxlcz48dGl0bGU+UGh5c2ljYWwgZXhlcmNpc2UgaW50ZXJ2ZW50aW9uIGluIGRl

cHJlc3NpdmUgZGlzb3JkZXJzOiBNZXRh4oCQYW5hbHlzaXMgYW5kIHN5c3RlbWF0aWMgcmV2aWV3

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNjYW5kaW5hdmlhbiBqb3VybmFsIG9mIG1lZGljaW5l

IGFuZCBzY2llbmNlIGluIHNwb3J0czwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2Rp

Y2FsPjxmdWxsLXRpdGxlPlNjYW5kaW5hdmlhbiBKb3VybmFsIG9mIE1lZGljaW5lIGFuZCBTY2ll

bmNlIGluIFNwb3J0czwvZnVsbC10aXRsZT48YWJici0xPlNjYW5kLiBKLiBNZWQuIFNjaS4gU3Bv

cnRzPC9hYmJyLTE+PGFiYnItMj5TY2FuZCBKIE1lZCBTY2kgU3BvcnRzPC9hYmJyLTI+PC9wZXJp

b2RpY2FsPjxwYWdlcz4yNTktMjcyPC9wYWdlcz48dm9sdW1lPjI0PC92b2x1bWU+PG51bWJlcj4y

PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTQ8L3llYXI+PC9kYXRlcz48aXNibj4xNjAwLTA4Mzg8

L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlBldGVyc29u

PC9BdXRob3I+PFllYXI+MjAxMTwvWWVhcj48UmVjTnVtPjIyPC9SZWNOdW0+PHJlY29yZD48cmVj

LW51bWJlcj4yMjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9

InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjI8L2tleT48L2ZvcmVpZ24t

a2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRy

aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlBldGVyc29uLCBNYXJrIEQ8L2F1dGhvcj48YXV0aG9y

PlNlbiwgQW5hbmRhPC9hdXRob3I+PGF1dGhvcj5Hb3Jkb24sIFBhdWwgTTwvYXV0aG9yPjwvYXV0

aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbmZsdWVuY2Ugb2YgcmVzaXN0YW5j

ZSBleGVyY2lzZSBvbiBsZWFuIGJvZHkgbWFzcyBpbiBhZ2luZyBhZHVsdHM6IGEgbWV0YS1hbmFs

eXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9y

dHMgYW5kIGV4ZXJjaXNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1

bGwtdGl0bGU+TWVkaWNpbmUgYW5kIFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVs

bC10aXRsZT48YWJici0xPk1lZC4gU2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5N

ZWQgU2NpIFNwb3J0IEV4ZXJjaXNlPC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJp

b2RpY2FsPjxwYWdlcz4yNDk8L3BhZ2VzPjx2b2x1bWU+NDM8L3ZvbHVtZT48bnVtYmVyPjI8L251

bWJlcj48ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29y

ZD48L0NpdGU+PENpdGU+PEF1dGhvcj5TYWx2YWRvcjwvQXV0aG9yPjxZZWFyPjIwMTY8L1llYXI+

PFJlY051bT4yNDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjQ8L3JlYy1udW1iZXI+PGZv

cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5

YXo5eDB2ejA5cngiPjI0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu

YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5T

YWx2YWRvciwgQUY8L2F1dGhvcj48YXV0aG9yPkRlIEFndWlhciwgUkE8L2F1dGhvcj48YXV0aG9y

Pkxpc2JvYSwgRkQ8L2F1dGhvcj48YXV0aG9yPlBlcmVpcmEsIEtMPC9hdXRob3I+PGF1dGhvcj5D

cnV6LCBSUzwvYXV0aG9yPjxhdXRob3I+Q2FwdXRvLCBGPC9hdXRob3I+PC9hdXRob3JzPjwvY29u

dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPklzY2hlbWljIHByZWNvbmRpdGlvbmluZyBhbmQgZXhl

cmNpc2UgcGVyZm9ybWFuY2U6IGEgc3lzdGVtYXRpYyByZXZpZXcgYW5kIG1ldGEtYW5hbHlzaXM8

L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+SW50ZXJuYXRpb25hbCBqb3VybmFsIG9mIHNwb3J0cyBw

aHlzaW9sb2d5IGFuZCBwZXJmb3JtYW5jZTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJp

b2RpY2FsPjxmdWxsLXRpdGxlPkludGVybmF0aW9uYWwgSm91cm5hbCBvZiBTcG9ydHMgUGh5c2lv

bG9neSBhbmQgUGVyZm9ybWFuY2U8L2Z1bGwtdGl0bGU+PGFiYnItMT5JbnQuIEouIFNwb3J0cyBQ

aHlzaW9sLiBQZXJmb3JtLjwvYWJici0xPjxhYmJyLTI+SW50IEogU3BvcnRzIFBoeXNpb2wgUGVy

Zm9ybTwvYWJici0yPjxhYmJyLTM+SUpTUFA8L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjQt

MTQ8L3BhZ2VzPjx2b2x1bWU+MTE8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNjwveWVhcj48L2Rh

dGVzPjxpc2JuPjE1NTUtMDI2NTwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+U29vbXJvPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjE4PC9S

ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4xODwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxr

ZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+

MTg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x

NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlNvb21ybywgTmFqZWVi

dWxsYWg8L2F1dGhvcj48YXV0aG9yPlNhbmRlcnMsIFJvc3M8L2F1dGhvcj48YXV0aG9yPkhhY2tl

dHQsIERhbmllbDwvYXV0aG9yPjxhdXRob3I+SHVia2EsIFRhdGU8L2F1dGhvcj48YXV0aG9yPkVi

cmFoaW1pLCBTYWFoaWw8L2F1dGhvcj48YXV0aG9yPkZyZWVzdG9uLCBKb25hdGhhbjwvYXV0aG9y

PjxhdXRob3I+Q29ibGV5LCBTdGVwaGVuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz

Pjx0aXRsZXM+PHRpdGxlPlRoZSBlZmljYWN5IG9mIGluanVyeSBwcmV2ZW50aW9uIHByb2dyYW1z

IGluIGFkb2xlc2NlbnQgdGVhbSBzcG9ydHM6IGEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29u

ZGFyeS10aXRsZT5BbWVyaWNhbiBqb3VybmFsIG9mIHNwb3J0cyBtZWRpY2luZTwvc2Vjb25kYXJ5

LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFtZXJpY2FuIEpvdXJuYWwg

b2YgU3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxhYmJyLTE+QW0uIEouIFNwb3J0cyBNZWQu

PC9hYmJyLTE+PGFiYnItMj5BbSBKIFNwb3J0cyBNZWQ8L2FiYnItMj48L3BlcmlvZGljYWw+PHBh

Z2VzPmh0dHBzOi8vZG9pLm9yZy8xMC4xMTc3LzAzNjM1NDY1MTU2MTgzNzI8L3BhZ2VzPjxkYXRl

cz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MDM2My01NDY1PC9pc2JuPjx1cmxzPjwv

dXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Ub21saW5zb248L0F1dGhvcj48WWVh

cj4yMDE1PC9ZZWFyPjxSZWNOdW0+MjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjIxPC9y

ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4

emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4yMTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw

ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo

b3JzPjxhdXRob3I+VG9tbGluc29uLCBQZXRlciBCPC9hdXRob3I+PGF1dGhvcj5Kb3NlcGgsIENv

cmV5PC9hdXRob3I+PGF1dGhvcj5Bbmdpb2ksIE1hbnVlbGE8L2F1dGhvcj48L2F1dGhvcnM+PC9j

b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RWZmZWN0cyBvZiB2aXRhbWluIEQgc3VwcGxlbWVu

dGF0aW9uIG9uIHVwcGVyIGFuZCBsb3dlciBib2R5IG11c2NsZSBzdHJlbmd0aCBsZXZlbHMgaW4g

aGVhbHRoeSBpbmRpdmlkdWFscy4gQSBzeXN0ZW1hdGljIHJldmlldyB3aXRoIG1ldGEtYW5hbHlz

aXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBTY2llbmNlIGFuZCBNZWRpY2lu

ZSBpbiBTcG9ydDwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp

dGxlPkpvdXJuYWwgb2YgU2NpZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L2Z1bGwtdGl0bGU+

PGFiYnItMT5KLiBTY2kuIE1lZC4gU3BvcnQ8L2FiYnItMT48YWJici0yPkogU2NpIE1lZCBTcG9y

dDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFnZXM+NTc1LTU4MDwvcGFnZXM+PHZvbHVtZT4xODwv

dm9sdW1lPjxkYXRlcz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTQ0MC0yNDQwPC9p

c2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5XcmlnaHQ8L0F1

dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+MTk8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt

YmVyPjE5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlw

dng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4xOTwva2V5PjwvZm9yZWlnbi1rZXlz

PjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0

b3JzPjxhdXRob3JzPjxhdXRob3I+V3JpZ2h0LCBBbGV4aXMgQTwvYXV0aG9yPjxhdXRob3I+VGF5

bG9yLCBKZWZmcmV5IEI8L2F1dGhvcj48YXV0aG9yPkZvcmQsIEtldmluIFI8L2F1dGhvcj48YXV0

aG9yPlNpc2thLCBMaW5kc2V5PC9hdXRob3I+PGF1dGhvcj5TbW9saWdhLCBKYW1lcyBNPC9hdXRo

b3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlJpc2sgZmFjdG9ycyBh

c3NvY2lhdGVkIHdpdGggbG93ZXIgZXh0cmVtaXR5IHN0cmVzcyBmcmFjdHVyZXMgaW4gcnVubmVy

czogYSBzeXN0ZW1hdGljIHJldmlldyB3aXRoIG1ldGEtYW5hbHlzaXM8L3RpdGxlPjxzZWNvbmRh

cnktdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9mIFNwb3J0cyBNZWRpY2luZTwvc2Vjb25kYXJ5LXRp

dGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkJyaXRpc2ggSm91cm5hbCBvZiBT

cG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5Cci4gSi4gU3BvcnRzIE1lZC48L2Fi

YnItMT48YWJici0yPkJyIEogU3BvcnRzIE1lZDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFnZXM+

MTUxNy0xNTIzPC9wYWdlcz48dm9sdW1lPjQ5PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMTU8L3ll

YXI+PC9kYXRlcz48aXNibj4xNDczLTA0ODA8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwv

Q2l0ZT48L0VuZE5vdGU+AG==

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IENodSBl

dCBhbC4sIDIwMTY7IEh1bWUgZXQgYWwuLCAyMDE1OyBKb3NlZnNzb24gZXQgYWwuLCAyMDE0OyBQ

ZXRlcnNvbiBldCBhbC4sIDIwMTE7IFNhbHZhZG9yIGV0IGFsLiwgMjAxNjsgU29vbXJvIGV0IGFs

LiwgMjAxNTsgVG9tbGluc29uIGV0IGFsLiwgMjAxNTsgV3JpZ2h0IGV0IGFsLiwgMjAxNSk8L0Rp

c3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5

cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5

cngiPjIwPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs

ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CdXJkZW4sIFJp

Y2hhcmQgSjwvYXV0aG9yPjxhdXRob3I+TW9ydG9uLCBLYXRpZTwvYXV0aG9yPjxhdXRob3I+Umlj

aGFyZHMsIFRvYnk8L2F1dGhvcj48YXV0aG9yPldoeXRlLCBHcmVnb3J5IFA8L2F1dGhvcj48YXV0

aG9yPlBlZGxhciwgQ2hhcmxlcyBSPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0

aXRsZXM+PHRpdGxlPklzIGlyb24gdHJlYXRtZW50IGJlbmVmaWNpYWwgaW4sIGlyb24tZGVmaWNp

ZW50IGJ1dCBub24tYW5hZW1pYyAoSUROQSkgZW5kdXJhbmNlIGF0aGxldGVzPyBBIG1ldGEtYW5h

bHlzaXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QnJpdGlzaCBqb3VybmFsIG9mIHNwb3J0cyBt

ZWRpY2luZTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxl

PkJyaXRpc2ggSm91cm5hbCBvZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5C

ci4gSi4gU3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPkJyIEogU3BvcnRzIE1lZDwvYWJici0y

PjwvcGVyaW9kaWNhbD48cGFnZXM+MTM4OS0xMzk3PC9wYWdlcz48dm9sdW1lPjQ5PC92b2x1bWU+

PGRhdGVzPjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4xNDczLTA0ODA8L2lzYm4+PHVy

bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNodTwvQXV0aG9yPjxZZWFy

PjIwMTY8L1llYXI+PFJlY051bT4yMzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjM8L3Jl

Yy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6

Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBl

IG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhv

cnM+PGF1dGhvcj5DaHUsIEFubmE8L2F1dGhvcj48YXV0aG9yPlBldG9jeiwgUGV0ZXI8L2F1dGhv

cj48YXV0aG9yPlNhbW1hbiwgU2FtaXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+

PHRpdGxlcz48dGl0bGU+SW1tZWRpYXRlIGVmZmVjdHMgb2YgYWVyb2JpYyBleGVyY2lzZSBvbiBw

bGFzbWEvc2VydW0gemluYyBsZXZlbHM6IGEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFy

eS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9ydHMgYW5kIGV4ZXJjaXNlPC9zZWNv

bmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TWVkaWNpbmUgYW5k

IFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVsbC10aXRsZT48YWJici0xPk1lZC4g

U2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5NZWQgU2NpIFNwb3J0IEV4ZXJjaXNl

PC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz43MjYtNzMz

PC9wYWdlcz48dm9sdW1lPjQ4PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMTY8L3llYXI+PC9kYXRl

cz48aXNibj4wMTk1LTkxMzE8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0

ZT48QXV0aG9yPkh1bWU8L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+MTI8L1JlY051

bT48cmVjb3JkPjxyZWMtbnVtYmVyPjEyPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBh

cHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4xMjwv

a2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9y

ZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+SHVtZSwgUGF0cmlhIEE8L2F1

dGhvcj48YXV0aG9yPkxvcmltZXIsIEFubmEgVjwvYXV0aG9yPjxhdXRob3I+R3JpZmZpdGhzLCBQ

ZXRlciBDPC9hdXRob3I+PGF1dGhvcj5DYXJsc29uLCBJc2FhYzwvYXV0aG9yPjxhdXRob3I+TGFt

b250LCBNaWtlPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl

PlJlY3JlYXRpb25hbCBzbm93LXNwb3J0cyBpbmp1cnkgcmlzayBmYWN0b3JzIGFuZCBjb3VudGVy

bWVhc3VyZXM6IGEgbWV0YS1hbmFseXNpcyByZXZpZXcgYW5kIEhhZGRvbiBtYXRyaXggZXZhbHVh

dGlvbjwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3NlY29uZGFyeS10

aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L2Z1

bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3BvcnQgTWVkPC9h

YmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+MTE3NS0xMTkwPC9w

YWdlcz48dm9sdW1lPjQ1PC92b2x1bWU+PG51bWJlcj44PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIw

MTU8L3llYXI+PC9kYXRlcz48aXNibj4wMTEyLTE2NDI8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVj

b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkpvc2Vmc3NvbjwvQXV0aG9yPjxZZWFyPjIwMTQ8L1ll

YXI+PFJlY051bT4xNzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTc8L3JlYy1udW1iZXI+

PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdz

Zng5YXo5eDB2ejA5cngiPjE3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9Ikpv

dXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhv

cj5Kb3NlZnNzb24sIFRvcmJqw7ZybjwvYXV0aG9yPjxhdXRob3I+TGluZHdhbGwsIE1hZ251czwv

YXV0aG9yPjxhdXRob3I+QXJjaGVyLCBUcmV2b3I8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmli

dXRvcnM+PHRpdGxlcz48dGl0bGU+UGh5c2ljYWwgZXhlcmNpc2UgaW50ZXJ2ZW50aW9uIGluIGRl

cHJlc3NpdmUgZGlzb3JkZXJzOiBNZXRh4oCQYW5hbHlzaXMgYW5kIHN5c3RlbWF0aWMgcmV2aWV3

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNjYW5kaW5hdmlhbiBqb3VybmFsIG9mIG1lZGljaW5l

IGFuZCBzY2llbmNlIGluIHNwb3J0czwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2Rp

Y2FsPjxmdWxsLXRpdGxlPlNjYW5kaW5hdmlhbiBKb3VybmFsIG9mIE1lZGljaW5lIGFuZCBTY2ll

bmNlIGluIFNwb3J0czwvZnVsbC10aXRsZT48YWJici0xPlNjYW5kLiBKLiBNZWQuIFNjaS4gU3Bv

cnRzPC9hYmJyLTE+PGFiYnItMj5TY2FuZCBKIE1lZCBTY2kgU3BvcnRzPC9hYmJyLTI+PC9wZXJp

b2RpY2FsPjxwYWdlcz4yNTktMjcyPC9wYWdlcz48dm9sdW1lPjI0PC92b2x1bWU+PG51bWJlcj4y

PC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTQ8L3llYXI+PC9kYXRlcz48aXNibj4xNjAwLTA4Mzg8

L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlBldGVyc29u

PC9BdXRob3I+PFllYXI+MjAxMTwvWWVhcj48UmVjTnVtPjIyPC9SZWNOdW0+PHJlY29yZD48cmVj

LW51bWJlcj4yMjwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9

InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjI8L2tleT48L2ZvcmVpZ24t

a2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRy

aWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlBldGVyc29uLCBNYXJrIEQ8L2F1dGhvcj48YXV0aG9y

PlNlbiwgQW5hbmRhPC9hdXRob3I+PGF1dGhvcj5Hb3Jkb24sIFBhdWwgTTwvYXV0aG9yPjwvYXV0

aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbmZsdWVuY2Ugb2YgcmVzaXN0YW5j

ZSBleGVyY2lzZSBvbiBsZWFuIGJvZHkgbWFzcyBpbiBhZ2luZyBhZHVsdHM6IGEgbWV0YS1hbmFs

eXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9y

dHMgYW5kIGV4ZXJjaXNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1

bGwtdGl0bGU+TWVkaWNpbmUgYW5kIFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVs

bC10aXRsZT48YWJici0xPk1lZC4gU2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5N

ZWQgU2NpIFNwb3J0IEV4ZXJjaXNlPC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJp

b2RpY2FsPjxwYWdlcz4yNDk8L3BhZ2VzPjx2b2x1bWU+NDM8L3ZvbHVtZT48bnVtYmVyPjI8L251

bWJlcj48ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2RhdGVzPjx1cmxzPjwvdXJscz48L3JlY29y

ZD48L0NpdGU+PENpdGU+PEF1dGhvcj5TYWx2YWRvcjwvQXV0aG9yPjxZZWFyPjIwMTY8L1llYXI+

PFJlY051bT4yNDwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MjQ8L3JlYy1udW1iZXI+PGZv

cmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5

YXo5eDB2ejA5cngiPjI0PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJu

YWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5T

YWx2YWRvciwgQUY8L2F1dGhvcj48YXV0aG9yPkRlIEFndWlhciwgUkE8L2F1dGhvcj48YXV0aG9y

Pkxpc2JvYSwgRkQ8L2F1dGhvcj48YXV0aG9yPlBlcmVpcmEsIEtMPC9hdXRob3I+PGF1dGhvcj5D

cnV6LCBSUzwvYXV0aG9yPjxhdXRob3I+Q2FwdXRvLCBGPC9hdXRob3I+PC9hdXRob3JzPjwvY29u

dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPklzY2hlbWljIHByZWNvbmRpdGlvbmluZyBhbmQgZXhl

cmNpc2UgcGVyZm9ybWFuY2U6IGEgc3lzdGVtYXRpYyByZXZpZXcgYW5kIG1ldGEtYW5hbHlzaXM8

L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+SW50ZXJuYXRpb25hbCBqb3VybmFsIG9mIHNwb3J0cyBw

aHlzaW9sb2d5IGFuZCBwZXJmb3JtYW5jZTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJp

b2RpY2FsPjxmdWxsLXRpdGxlPkludGVybmF0aW9uYWwgSm91cm5hbCBvZiBTcG9ydHMgUGh5c2lv

bG9neSBhbmQgUGVyZm9ybWFuY2U8L2Z1bGwtdGl0bGU+PGFiYnItMT5JbnQuIEouIFNwb3J0cyBQ

aHlzaW9sLiBQZXJmb3JtLjwvYWJici0xPjxhYmJyLTI+SW50IEogU3BvcnRzIFBoeXNpb2wgUGVy

Zm9ybTwvYWJici0yPjxhYmJyLTM+SUpTUFA8L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjQt

MTQ8L3BhZ2VzPjx2b2x1bWU+MTE8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNjwveWVhcj48L2Rh

dGVzPjxpc2JuPjE1NTUtMDI2NTwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+U29vbXJvPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjE4PC9S

ZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4xODwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxr

ZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+

MTg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4x

NzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlNvb21ybywgTmFqZWVi

dWxsYWg8L2F1dGhvcj48YXV0aG9yPlNhbmRlcnMsIFJvc3M8L2F1dGhvcj48YXV0aG9yPkhhY2tl

dHQsIERhbmllbDwvYXV0aG9yPjxhdXRob3I+SHVia2EsIFRhdGU8L2F1dGhvcj48YXV0aG9yPkVi

cmFoaW1pLCBTYWFoaWw8L2F1dGhvcj48YXV0aG9yPkZyZWVzdG9uLCBKb25hdGhhbjwvYXV0aG9y

PjxhdXRob3I+Q29ibGV5LCBTdGVwaGVuPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3Jz

Pjx0aXRsZXM+PHRpdGxlPlRoZSBlZmljYWN5IG9mIGluanVyeSBwcmV2ZW50aW9uIHByb2dyYW1z

IGluIGFkb2xlc2NlbnQgdGVhbSBzcG9ydHM6IGEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29u

ZGFyeS10aXRsZT5BbWVyaWNhbiBqb3VybmFsIG9mIHNwb3J0cyBtZWRpY2luZTwvc2Vjb25kYXJ5

LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkFtZXJpY2FuIEpvdXJuYWwg

b2YgU3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxhYmJyLTE+QW0uIEouIFNwb3J0cyBNZWQu

PC9hYmJyLTE+PGFiYnItMj5BbSBKIFNwb3J0cyBNZWQ8L2FiYnItMj48L3BlcmlvZGljYWw+PHBh

Z2VzPmh0dHBzOi8vZG9pLm9yZy8xMC4xMTc3LzAzNjM1NDY1MTU2MTgzNzI8L3BhZ2VzPjxkYXRl

cz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MDM2My01NDY1PC9pc2JuPjx1cmxzPjwv

dXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Ub21saW5zb248L0F1dGhvcj48WWVh

cj4yMDE1PC9ZZWFyPjxSZWNOdW0+MjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjIxPC9y

ZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4

emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4yMTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw

ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo

b3JzPjxhdXRob3I+VG9tbGluc29uLCBQZXRlciBCPC9hdXRob3I+PGF1dGhvcj5Kb3NlcGgsIENv

cmV5PC9hdXRob3I+PGF1dGhvcj5Bbmdpb2ksIE1hbnVlbGE8L2F1dGhvcj48L2F1dGhvcnM+PC9j

b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RWZmZWN0cyBvZiB2aXRhbWluIEQgc3VwcGxlbWVu

dGF0aW9uIG9uIHVwcGVyIGFuZCBsb3dlciBib2R5IG11c2NsZSBzdHJlbmd0aCBsZXZlbHMgaW4g

aGVhbHRoeSBpbmRpdmlkdWFscy4gQSBzeXN0ZW1hdGljIHJldmlldyB3aXRoIG1ldGEtYW5hbHlz

aXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+Sm91cm5hbCBvZiBTY2llbmNlIGFuZCBNZWRpY2lu

ZSBpbiBTcG9ydDwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp

dGxlPkpvdXJuYWwgb2YgU2NpZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L2Z1bGwtdGl0bGU+

PGFiYnItMT5KLiBTY2kuIE1lZC4gU3BvcnQ8L2FiYnItMT48YWJici0yPkogU2NpIE1lZCBTcG9y

dDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFnZXM+NTc1LTU4MDwvcGFnZXM+PHZvbHVtZT4xODwv

dm9sdW1lPjxkYXRlcz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTQ0MC0yNDQwPC9p

c2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5XcmlnaHQ8L0F1

dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+MTk8L1JlY051bT48cmVjb3JkPjxyZWMtbnVt

YmVyPjE5PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlw

dng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4xOTwva2V5PjwvZm9yZWlnbi1rZXlz

PjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0

b3JzPjxhdXRob3JzPjxhdXRob3I+V3JpZ2h0LCBBbGV4aXMgQTwvYXV0aG9yPjxhdXRob3I+VGF5

bG9yLCBKZWZmcmV5IEI8L2F1dGhvcj48YXV0aG9yPkZvcmQsIEtldmluIFI8L2F1dGhvcj48YXV0

aG9yPlNpc2thLCBMaW5kc2V5PC9hdXRob3I+PGF1dGhvcj5TbW9saWdhLCBKYW1lcyBNPC9hdXRo

b3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPlJpc2sgZmFjdG9ycyBh

c3NvY2lhdGVkIHdpdGggbG93ZXIgZXh0cmVtaXR5IHN0cmVzcyBmcmFjdHVyZXMgaW4gcnVubmVy

czogYSBzeXN0ZW1hdGljIHJldmlldyB3aXRoIG1ldGEtYW5hbHlzaXM8L3RpdGxlPjxzZWNvbmRh

cnktdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9mIFNwb3J0cyBNZWRpY2luZTwvc2Vjb25kYXJ5LXRp

dGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkJyaXRpc2ggSm91cm5hbCBvZiBT

cG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5Cci4gSi4gU3BvcnRzIE1lZC48L2Fi

YnItMT48YWJici0yPkJyIEogU3BvcnRzIE1lZDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFnZXM+

MTUxNy0xNTIzPC9wYWdlcz48dm9sdW1lPjQ5PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMTU8L3ll

YXI+PC9kYXRlcz48aXNibj4xNDczLTA0ODA8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwv

Q2l0ZT48L0VuZE5vdGU+AG==

ADDIN EN.CITE.DATA (Burden et al., 2015; Chu et al., 2016; Hume et al., 2015; Josefsson et al., 2014; Peterson et al., 2011; Salvador et al., 2016; Soomro et al., 2015; Tomlinson et al., 2015; Wright et al., 2015). One of the meta-analyses ADDIN EN.CITE <EndNote><Cite><Author>Soomro</Author><Year>2015</Year><RecNum>18</RecNum><DisplayText>(Soomro et al., 2015)</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">18</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Soomro, Najeebullah</author><author>Sanders, Ross</author><author>Hackett, Daniel</author><author>Hubka, Tate</author><author>Ebrahimi, Saahil</author><author>Freeston, Jonathan</author><author>Cobley, Stephen</author></authors></contributors><titles><title>The eficacy of injury prevention programs in adolescent team sports: a meta-analysis</title><secondary-title>American journal of sports medicine</secondary-title></titles><periodical><full-title>American Journal of Sports Medicine</full-title><abbr-1>Am. J. Sports Med.</abbr-1><abbr-2>Am J Sports Med</abbr-2></periodical><pages>;(Soomro et al., 2015) turned out to have only one minor flaw. This article represents a summary of the flaws and advice on how to avoid them. I submitted the article to Sports Medicine in 2016, but the editor decided that he would prefer a meta-analysis of meta-analyses, something I did not have time or inclination to write. Instead, I wrote a letter to the editor about the Coquart et al. study ADDIN EN.CITE <EndNote><Cite><Author>Hopkins</Author><Year>2016</Year><RecNum>29</RecNum><DisplayText>(Hopkins, 2016)</DisplayText><record><rec-number>29</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">29</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hopkins, W G</author></authors></contributors><titles><title>Comment on: “Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study”</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>1195-1196</pages><volume>8</volume><number>46</number><dates><year>2016</year></dates><isbn>0112-1642</isbn><urls></urls></record></Cite></EndNote>(Hopkins, 2016), to which the authors responded by showing that they did not understand the difference between confidence limits and limits of agreement, by continuing to cite flawed meta-analyses as justification, and by claiming incorrectly that heterogeneity statistics are not always needed ADDIN EN.CITE <EndNote><Cite><Author>Coquart</Author><Year>2016</Year><RecNum>30</RecNum><DisplayText>(Coquart et al., 2016b)</DisplayText><record><rec-number>30</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">30</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, Jérémy B</author><author>Tabben, Montassar</author><author>Tourny, Claire</author><author>Chamari, Karim</author><author>Eston, Roger G</author></authors></contributors><titles><title>Author’s Reply to Will G. Hopkins: “Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study”</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>1197-1198</pages><volume>46</volume><number>8</number><dates><year>2016</year></dates><isbn>0112-1642</isbn><urls></urls></record></Cite></EndNote>(Coquart et al., 2016b). After setting aside this article for a year, I decided to submit it for publication in Sportscience. I have identified the flaws under subheadings that reflect the temporal order in which a meta-analysis is usually analyzed. My assertions about the wrong and right ways to do a meta-analysis are based on an article/slideshow I published in 2004, which I have updated regularly ADDIN EN.CITE <EndNote><Cite><Author>Hopkins</Author><Year>2004</Year><RecNum>1</RecNum><DisplayText>(Hopkins, 2004)</DisplayText><record><rec-number>1</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">1</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hopkins, W G</author></authors></contributors><titles><title>An introduction to meta-analysis</title><secondary-title>Sportscience</secondary-title></titles><periodical><full-title>Sportscience</full-title></periodical><pages>20-24</pages><volume>8</volume><dates><year>2004</year></dates><urls></urls></record></Cite></EndNote>(Hopkins, 2004), and on a more generic guidelines article ADDIN EN.CITE <EndNote><Cite><Author>Hopkins</Author><Year>2009</Year><RecNum>13</RecNum><DisplayText>(Hopkins et al., 2009)</DisplayText><record><rec-number>13</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">13</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hopkins, W G</author><author>Marshall, S W</author><author>Batterham, A M</author><author>Hanin, J</author></authors></contributors><titles><title>Progressive statistics for studies in sports medicine and exercise science</title><secondary-title>Medicine and Science in Sports and Exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>3-12</pages><volume>41</volume><dates><year>2009</year></dates><urls></urls></record></Cite></EndNote>(Hopkins et al., 2009). The right ways are exemplified (I hope) in recent meta-analyses I have co-authored PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Tbm93bGluZzwvQXV0aG9yPjxZZWFyPjIwMDY8L1llYXI+

PFJlY051bT41PC9SZWNOdW0+PERpc3BsYXlUZXh0PihCb25ldHRpIGFuZCBIb3BraW5zLCAyMDA5

OyBCcmFha2h1aXMgYW5kIEhvcGtpbnMsIDIwMTU7IENhcnIgZXQgYWwuLCAyMDExOyBDYXNzYXIg

ZXQgYWwuLCAyMDE3OyBNY0dyYXRoIGV0IGFsLiwgMjAxNTsgU25vd2xpbmcgYW5kIEhvcGtpbnMs

IDIwMDY7IFNvbWVydmlsbGUgZXQgYWwuLCAyMDE2OyBWYW5kZW5ib2dhZXJkZSBhbmQgSG9wa2lu

cywgMjAxMTsgV2VzdG9uIGV0IGFsLiwgMjAxNCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1u

dW1iZXI+NTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5

cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+NTwva2V5PjwvZm9yZWlnbi1rZXlz

PjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0

b3JzPjxhdXRob3JzPjxhdXRob3I+U25vd2xpbmcsIE5laWwgSjwvYXV0aG9yPjxhdXRob3I+SG9w

a2lucywgV2lsbCBHPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp

dGxlPkVmZmVjdHMgb2YgZGlmZmVyZW50IG1vZGVzIG9mIGV4ZXJjaXNlIHRyYWluaW5nIG9uIGds

dWNvc2UgY29udHJvbCBhbmQgcmlzayBmYWN0b3JzIGZvciBjb21wbGljYXRpb25zIGluIHR5cGUg

MiBkaWFiZXRpYyBwYXRpZW50cyBhIG1ldGEtYW5hbHlzaXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0

bGU+RGlhYmV0ZXMgQ2FyZTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxm

dWxsLXRpdGxlPkRpYWJldGVzIENhcmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5EaWFiZXRlcyBDYXJl

PC9hYmJyLTE+PGFiYnItMj5EaWFiZXRlcyBDYXJlPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdl

cz4yNTE4LTI1Mjc8L3BhZ2VzPjx2b2x1bWU+Mjk8L3ZvbHVtZT48bnVtYmVyPjExPC9udW1iZXI+

PGRhdGVzPjx5ZWFyPjIwMDY8L3llYXI+PC9kYXRlcz48aXNibj4wMTQ5LTU5OTI8L2lzYm4+PHVy

bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkJvbmV0dGk8L0F1dGhvcj48

WWVhcj4yMDA5PC9ZZWFyPjxSZWNOdW0+NjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+Njwv

cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoy

eHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw

ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo

b3JzPjxhdXRob3I+Qm9uZXR0aSwgRGFycmVsbCBMPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5zLCBX

aWxsIEc8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+U2Vh

LWxldmVsIGV4ZXJjaXNlIHBlcmZvcm1hbmNlIGZvbGxvd2luZyBhZGFwdGF0aW9uIHRvIGh5cG94

aWE8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0

bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxs

LXRpdGxlPjxhYmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJi

ci0yPjxhYmJyLTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjEwNy0xMjc8L3BhZ2Vz

Pjx2b2x1bWU+Mzk8L3ZvbHVtZT48bnVtYmVyPjI8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwOTwv

eWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+

PC9DaXRlPjxDaXRlPjxBdXRob3I+Q2FycjwvQXV0aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051

bT43PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj43PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl

eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnow

OXJ4Ij43PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs

ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DYXJyLCBBbWVs

aWEgSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywgV2lsbCBHPC9hdXRob3I+PGF1dGhvcj5Hb3Jl

LCBDaHJpc3RvcGhlciBKPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+

PHRpdGxlPkVmZmVjdHMgb2YgYWN1dGUgYWxrYWxvc2lzIGFuZCBhY2lkb3NpcyBvbiBwZXJmb3Jt

YW5jZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3NlY29uZGFyeS10

aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L2Z1

bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3BvcnQgTWVkPC9h

YmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+ODAxLTgxNDwvcGFn

ZXM+PHZvbHVtZT40MTwvdm9sdW1lPjxudW1iZXI+MTA8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAx

MTwveWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNv

cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2VzdG9uPC9BdXRob3I+PFllYXI+MjAxNDwvWWVhcj48

UmVjTnVtPjg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjg8L3JlYy1udW1iZXI+PGZvcmVp

Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5

eDB2ejA5cngiPjg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBB

cnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldlc3Rv

biwgTWF0dGhldzwvYXV0aG9yPjxhdXRob3I+VGF5bG9yLCBLYXRocnluIEw8L2F1dGhvcj48YXV0

aG9yPkJhdHRlcmhhbSwgQWxhbiBNPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5zLCBXaWxsIEc8L2F1

dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RWZmZWN0cyBvZiBs

b3ctdm9sdW1lIGhpZ2gtaW50ZW5zaXR5IGludGVydmFsIHRyYWluaW5nIChISVQpIG9uIGZpdG5l

c3MgaW4gYWR1bHRzOiBhIG1ldGEtYW5hbHlzaXMgb2YgY29udHJvbGxlZCBhbmQgbm9uLWNvbnRy

b2xsZWQgdHJpYWxzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvc2Vj

b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlNwb3J0cyBNZWRp

Y2luZTwvZnVsbC10aXRsZT48YWJici0xPlNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5TcG9y

dCBNZWQ8L2FiYnItMj48YWJici0zPlNNPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMDA1

LTEwMTc8L3BhZ2VzPjx2b2x1bWU+NDQ8L3ZvbHVtZT48bnVtYmVyPjc8L251bWJlcj48ZGF0ZXM+

PHllYXI+MjAxNDwveWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3Vy

bHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+VmFuZGVuYm9nYWVyZGU8L0F1dGhvcj48

WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+OTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+OTwv

cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoy

eHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+OTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw

ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo

b3JzPjxhdXRob3I+VmFuZGVuYm9nYWVyZGUsIFRvbSBKPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5z

LCBXaWxsIEc8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+

RWZmZWN0cyBvZiBhY3V0ZSBjYXJib2h5ZHJhdGUgc3VwcGxlbWVudGF0aW9uIG9uIGVuZHVyYW5j

ZSBwZXJmb3JtYW5jZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3Nl

Y29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVk

aWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3Bv

cnQgTWVkPC9hYmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+Nzcz

LTc5MjwvcGFnZXM+PHZvbHVtZT40MTwvdm9sdW1lPjxudW1iZXI+OTwvbnVtYmVyPjxkYXRlcz48

eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PGlzYm4+MDExMi0xNjQyPC9pc2JuPjx1cmxzPjwvdXJs

cz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5CcmFha2h1aXM8L0F1dGhvcj48WWVhcj4y

MDE1PC9ZZWFyPjxSZWNOdW0+MTA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjEwPC9yZWMt

bnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdl

cGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4xMDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu

YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz

PjxhdXRob3I+QnJhYWtodWlzLCBBbmRyZWEgSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywgV2ls

bCBHPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkltcGFj

dCBvZiBkaWV0YXJ5IGFudGlveGlkYW50cyBvbiBzcG9ydCBwZXJmb3JtYW5jZTogYSByZXZpZXc8

L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+

PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxsLXRp

dGxlPjxhYmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJici0y

PjxhYmJyLTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjkzOS05NTU8L3BhZ2VzPjx2

b2x1bWU+NDU8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxpc2JuPjAx

MTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+

U29tZXJ2aWxsZTwvQXV0aG9yPjxZZWFyPjIwMTY8L1llYXI+PFJlY051bT4zNDwvUmVjTnVtPjxy

ZWNvcmQ+PHJlYy1udW1iZXI+MzQ8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i

RU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjM0PC9rZXk+

PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10

eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Tb21lcnZpbGxlLCBWYXVnaGFuIFM8

L2F1dGhvcj48YXV0aG9yPkJyYWFraHVpcywgQW5kcmVhIEo8L2F1dGhvcj48YXV0aG9yPkhvcGtp

bnMsIFdpbGwgRzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs

ZT5FZmZlY3Qgb2YgZmxhdm9ub2lkcyBvbiB1cHBlciByZXNwaXJhdG9yeSB0cmFjdCBpbmZlY3Rp

b25zIGFuZCBpbW11bmUgZnVuY3Rpb246IEEgc3lzdGVtYXRpYyByZXZpZXcgYW5kIG1ldGEtYW5h

bHlzaXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QWR2YW5jZXMgaW4gTnV0cml0aW9uPC9zZWNv

bmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjQ4OC00OTc8L3BhZ2VzPjx2b2x1bWU+Nzwvdm9s

dW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MjE1Ni01Mzc2PC9pc2Ju

Pjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5NY0dyYXRoPC9BdXRo

b3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjMzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl

cj4zMzwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4

NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MzM8L2tleT48L2ZvcmVpZ24ta2V5cz48

cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y

cz48YXV0aG9ycz48YXV0aG9yPk1jR3JhdGgsIEwgSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywg

VyBHPC9hdXRob3I+PGF1dGhvcj5IaW5ja3NvbiwgRSBBPC9hdXRob3I+PC9hdXRob3JzPjwvY29u

dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc29jaWF0aW9ucyBvZiBvYmplY3RpdmVseSBtZWFz

dXJlZCBidWlsdC1lbnZpcm9ubWVudCBhdHRyaWJ1dGVzIHdpdGggeW91dGggbW9kZXJhdGXigJN2

aWdvcm91cyBwaHlzaWNhbCBhY3Rpdml0eTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1h

bmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3NlY29uZGFy

eS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVkaWNpbmU8

L2Z1bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3BvcnQgTWVk

PC9hYmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+ODQxLTg2NTwv

cGFnZXM+PHZvbHVtZT40NTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+

PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNhc3NhcjwvQXV0aG9y

PjxZZWFyPjIwMTc8L1llYXI+PFJlY051bT4zMjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+

MzI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVm

MHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjMyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJl

Zi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+

PGF1dGhvcnM+PGF1dGhvcj5DYXNzYXIsIFM8L2F1dGhvcj48YXV0aG9yPk1pc3NvLCBNIEw8L2F1

dGhvcj48YXV0aG9yPkhvcGtpbnMsIFcgRzwvYXV0aG9yPjxhdXRob3I+U2hhdywgQyBTPC9hdXRo

b3I+PGF1dGhvcj5UZWVkZSwgSCBKPC9hdXRob3I+PGF1dGhvcj5TdGVwdG8sIE4gSzwvYXV0aG9y

PjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbnN1bGluIHJlc2lzdGFu

Y2UgaW4gcG9seWN5c3RpYyBvdmFyeSBzeW5kcm9tZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQg

bWV0YS1hbmFseXNpcyBvZiBldWdseWNhZW1pYy1oeXBlcmluc3VsaW5hZW1pYyBjbGFtcCBzdHVk

aWVzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkh1bWFuIFJlcHJvZHVjdGlvbjwvc2Vjb25kYXJ5

LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkh1bWFuIFJlcHJvZHVjdGlv

bjwvZnVsbC10aXRsZT48YWJici0xPkh1bS4gUmVwcm9kLjwvYWJici0xPjxhYmJyLTI+SHVtIFJl

cHJvZDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFnZXM+MjYxOeKAkzI2MzEgKGh0dHBzOi8vZG9p

Lm9yZy8xMC4xMDkzL2h1bXJlcC9kZXcyNDMpPC9wYWdlcz48dm9sdW1lPjMxPC92b2x1bWU+PGRh

dGVzPjx5ZWFyPjIwMTc8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRl

PjwvRW5kTm90ZT5=

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5Tbm93bGluZzwvQXV0aG9yPjxZZWFyPjIwMDY8L1llYXI+

PFJlY051bT41PC9SZWNOdW0+PERpc3BsYXlUZXh0PihCb25ldHRpIGFuZCBIb3BraW5zLCAyMDA5

OyBCcmFha2h1aXMgYW5kIEhvcGtpbnMsIDIwMTU7IENhcnIgZXQgYWwuLCAyMDExOyBDYXNzYXIg

ZXQgYWwuLCAyMDE3OyBNY0dyYXRoIGV0IGFsLiwgMjAxNTsgU25vd2xpbmcgYW5kIEhvcGtpbnMs

IDIwMDY7IFNvbWVydmlsbGUgZXQgYWwuLCAyMDE2OyBWYW5kZW5ib2dhZXJkZSBhbmQgSG9wa2lu

cywgMjAxMTsgV2VzdG9uIGV0IGFsLiwgMjAxNCk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1u

dW1iZXI+NTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5

cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+NTwva2V5PjwvZm9yZWlnbi1rZXlz

PjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0

b3JzPjxhdXRob3JzPjxhdXRob3I+U25vd2xpbmcsIE5laWwgSjwvYXV0aG9yPjxhdXRob3I+SG9w

a2lucywgV2lsbCBHPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp

dGxlPkVmZmVjdHMgb2YgZGlmZmVyZW50IG1vZGVzIG9mIGV4ZXJjaXNlIHRyYWluaW5nIG9uIGds

dWNvc2UgY29udHJvbCBhbmQgcmlzayBmYWN0b3JzIGZvciBjb21wbGljYXRpb25zIGluIHR5cGUg

MiBkaWFiZXRpYyBwYXRpZW50cyBhIG1ldGEtYW5hbHlzaXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0

bGU+RGlhYmV0ZXMgQ2FyZTwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxm

dWxsLXRpdGxlPkRpYWJldGVzIENhcmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5EaWFiZXRlcyBDYXJl

PC9hYmJyLTE+PGFiYnItMj5EaWFiZXRlcyBDYXJlPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdl

cz4yNTE4LTI1Mjc8L3BhZ2VzPjx2b2x1bWU+Mjk8L3ZvbHVtZT48bnVtYmVyPjExPC9udW1iZXI+

PGRhdGVzPjx5ZWFyPjIwMDY8L3llYXI+PC9kYXRlcz48aXNibj4wMTQ5LTU5OTI8L2lzYm4+PHVy

bHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkJvbmV0dGk8L0F1dGhvcj48

WWVhcj4yMDA5PC9ZZWFyPjxSZWNOdW0+NjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+Njwv

cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoy

eHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+Njwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw

ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo

b3JzPjxhdXRob3I+Qm9uZXR0aSwgRGFycmVsbCBMPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5zLCBX

aWxsIEc8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+U2Vh

LWxldmVsIGV4ZXJjaXNlIHBlcmZvcm1hbmNlIGZvbGxvd2luZyBhZGFwdGF0aW9uIHRvIGh5cG94

aWE8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0

bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxs

LXRpdGxlPjxhYmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJi

ci0yPjxhYmJyLTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjEwNy0xMjc8L3BhZ2Vz

Pjx2b2x1bWU+Mzk8L3ZvbHVtZT48bnVtYmVyPjI8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAwOTwv

eWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+

PC9DaXRlPjxDaXRlPjxBdXRob3I+Q2FycjwvQXV0aG9yPjxZZWFyPjIwMTE8L1llYXI+PFJlY051

bT43PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj43PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtl

eXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnow

OXJ4Ij43PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs

ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DYXJyLCBBbWVs

aWEgSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywgV2lsbCBHPC9hdXRob3I+PGF1dGhvcj5Hb3Jl

LCBDaHJpc3RvcGhlciBKPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+

PHRpdGxlPkVmZmVjdHMgb2YgYWN1dGUgYWxrYWxvc2lzIGFuZCBhY2lkb3NpcyBvbiBwZXJmb3Jt

YW5jZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3NlY29uZGFyeS10

aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L2Z1

bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3BvcnQgTWVkPC9h

YmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+ODAxLTgxNDwvcGFn

ZXM+PHZvbHVtZT40MTwvdm9sdW1lPjxudW1iZXI+MTA8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAx

MTwveWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNv

cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+V2VzdG9uPC9BdXRob3I+PFllYXI+MjAxNDwvWWVhcj48

UmVjTnVtPjg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjg8L3JlYy1udW1iZXI+PGZvcmVp

Z24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5

eDB2ejA5cngiPjg8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBB

cnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldlc3Rv

biwgTWF0dGhldzwvYXV0aG9yPjxhdXRob3I+VGF5bG9yLCBLYXRocnluIEw8L2F1dGhvcj48YXV0

aG9yPkJhdHRlcmhhbSwgQWxhbiBNPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5zLCBXaWxsIEc8L2F1

dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RWZmZWN0cyBvZiBs

b3ctdm9sdW1lIGhpZ2gtaW50ZW5zaXR5IGludGVydmFsIHRyYWluaW5nIChISVQpIG9uIGZpdG5l

c3MgaW4gYWR1bHRzOiBhIG1ldGEtYW5hbHlzaXMgb2YgY29udHJvbGxlZCBhbmQgbm9uLWNvbnRy

b2xsZWQgdHJpYWxzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvc2Vj

b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlNwb3J0cyBNZWRp

Y2luZTwvZnVsbC10aXRsZT48YWJici0xPlNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5TcG9y

dCBNZWQ8L2FiYnItMj48YWJici0zPlNNPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMDA1

LTEwMTc8L3BhZ2VzPjx2b2x1bWU+NDQ8L3ZvbHVtZT48bnVtYmVyPjc8L251bWJlcj48ZGF0ZXM+

PHllYXI+MjAxNDwveWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3Vy

bHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+VmFuZGVuYm9nYWVyZGU8L0F1dGhvcj48

WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+OTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+OTwv

cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoy

eHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+OTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlw

ZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRo

b3JzPjxhdXRob3I+VmFuZGVuYm9nYWVyZGUsIFRvbSBKPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5z

LCBXaWxsIEc8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+

RWZmZWN0cyBvZiBhY3V0ZSBjYXJib2h5ZHJhdGUgc3VwcGxlbWVudGF0aW9uIG9uIGVuZHVyYW5j

ZSBwZXJmb3JtYW5jZTwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3Nl

Y29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVk

aWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3Bv

cnQgTWVkPC9hYmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+Nzcz

LTc5MjwvcGFnZXM+PHZvbHVtZT40MTwvdm9sdW1lPjxudW1iZXI+OTwvbnVtYmVyPjxkYXRlcz48

eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PGlzYm4+MDExMi0xNjQyPC9pc2JuPjx1cmxzPjwvdXJs

cz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5CcmFha2h1aXM8L0F1dGhvcj48WWVhcj4y

MDE1PC9ZZWFyPjxSZWNOdW0+MTA8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjEwPC9yZWMt

bnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdl

cGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4xMDwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBu

YW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3Jz

PjxhdXRob3I+QnJhYWtodWlzLCBBbmRyZWEgSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywgV2ls

bCBHPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkltcGFj

dCBvZiBkaWV0YXJ5IGFudGlveGlkYW50cyBvbiBzcG9ydCBwZXJmb3JtYW5jZTogYSByZXZpZXc8

L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+

PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxsLXRp

dGxlPjxhYmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJici0y

PjxhYmJyLTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjkzOS05NTU8L3BhZ2VzPjx2

b2x1bWU+NDU8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxpc2JuPjAx

MTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+

U29tZXJ2aWxsZTwvQXV0aG9yPjxZZWFyPjIwMTY8L1llYXI+PFJlY051bT4zNDwvUmVjTnVtPjxy

ZWNvcmQ+PHJlYy1udW1iZXI+MzQ8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i

RU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjM0PC9rZXk+

PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10

eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Tb21lcnZpbGxlLCBWYXVnaGFuIFM8

L2F1dGhvcj48YXV0aG9yPkJyYWFraHVpcywgQW5kcmVhIEo8L2F1dGhvcj48YXV0aG9yPkhvcGtp

bnMsIFdpbGwgRzwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs

ZT5FZmZlY3Qgb2YgZmxhdm9ub2lkcyBvbiB1cHBlciByZXNwaXJhdG9yeSB0cmFjdCBpbmZlY3Rp

b25zIGFuZCBpbW11bmUgZnVuY3Rpb246IEEgc3lzdGVtYXRpYyByZXZpZXcgYW5kIG1ldGEtYW5h

bHlzaXM8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+QWR2YW5jZXMgaW4gTnV0cml0aW9uPC9zZWNv

bmRhcnktdGl0bGU+PC90aXRsZXM+PHBhZ2VzPjQ4OC00OTc8L3BhZ2VzPjx2b2x1bWU+Nzwvdm9s

dW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MjE1Ni01Mzc2PC9pc2Ju

Pjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5NY0dyYXRoPC9BdXRo

b3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjMzPC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJl

cj4zMzwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4

NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MzM8L2tleT48L2ZvcmVpZ24ta2V5cz48

cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y

cz48YXV0aG9ycz48YXV0aG9yPk1jR3JhdGgsIEwgSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywg

VyBHPC9hdXRob3I+PGF1dGhvcj5IaW5ja3NvbiwgRSBBPC9hdXRob3I+PC9hdXRob3JzPjwvY29u

dHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkFzc29jaWF0aW9ucyBvZiBvYmplY3RpdmVseSBtZWFz

dXJlZCBidWlsdC1lbnZpcm9ubWVudCBhdHRyaWJ1dGVzIHdpdGggeW91dGggbW9kZXJhdGXigJN2

aWdvcm91cyBwaHlzaWNhbCBhY3Rpdml0eTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1h

bmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5TcG9ydHMgTWVkaWNpbmU8L3NlY29uZGFy

eS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5TcG9ydHMgTWVkaWNpbmU8

L2Z1bGwtdGl0bGU+PGFiYnItMT5TcG9ydHMgTWVkLjwvYWJici0xPjxhYmJyLTI+U3BvcnQgTWVk

PC9hYmJyLTI+PGFiYnItMz5TTTwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+ODQxLTg2NTwv

cGFnZXM+PHZvbHVtZT40NTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+

PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNhc3NhcjwvQXV0aG9y

PjxZZWFyPjIwMTc8L1llYXI+PFJlY051bT4zMjwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+

MzI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVm

MHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjMyPC9rZXk+PC9mb3JlaWduLWtleXM+PHJl

Zi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+

PGF1dGhvcnM+PGF1dGhvcj5DYXNzYXIsIFM8L2F1dGhvcj48YXV0aG9yPk1pc3NvLCBNIEw8L2F1

dGhvcj48YXV0aG9yPkhvcGtpbnMsIFcgRzwvYXV0aG9yPjxhdXRob3I+U2hhdywgQyBTPC9hdXRo

b3I+PGF1dGhvcj5UZWVkZSwgSCBKPC9hdXRob3I+PGF1dGhvcj5TdGVwdG8sIE4gSzwvYXV0aG9y

PjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbnN1bGluIHJlc2lzdGFu

Y2UgaW4gcG9seWN5c3RpYyBvdmFyeSBzeW5kcm9tZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQg

bWV0YS1hbmFseXNpcyBvZiBldWdseWNhZW1pYy1oeXBlcmluc3VsaW5hZW1pYyBjbGFtcCBzdHVk

aWVzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkh1bWFuIFJlcHJvZHVjdGlvbjwvc2Vjb25kYXJ5

LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkh1bWFuIFJlcHJvZHVjdGlv

bjwvZnVsbC10aXRsZT48YWJici0xPkh1bS4gUmVwcm9kLjwvYWJici0xPjxhYmJyLTI+SHVtIFJl

cHJvZDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFnZXM+MjYxOeKAkzI2MzEgKGh0dHBzOi8vZG9p

Lm9yZy8xMC4xMDkzL2h1bXJlcC9kZXcyNDMpPC9wYWdlcz48dm9sdW1lPjMxPC92b2x1bWU+PGRh

dGVzPjx5ZWFyPjIwMTc8L3llYXI+PC9kYXRlcz48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRl

PjwvRW5kTm90ZT5=

ADDIN EN.CITE.DATA (Bonetti and Hopkins, 2009; Braakhuis and Hopkins, 2015; Carr et al., 2011; Cassar et al., 2017; McGrath et al., 2015; Snowling and Hopkins, 2006; Somerville et al., 2016; Vandenbogaerde and Hopkins, 2011; Weston et al., 2014). The Cochrane handbook ADDIN EN.CITE <EndNote><Cite><Author>Higgins</Author><Year>2011</Year><RecNum>25</RecNum><DisplayText>(Higgins and Green, 2011)</DisplayText><record><rec-number>25</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">25</key></foreign-keys><ref-type name="Edited Book">28</ref-type><contributors><authors><author>Higgins, J P T</author><author>Green, S</author></authors></contributors><titles><title>Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]</title></titles><pages> cochrane-. Accessed 15 Jan 2016.</pages><number>January</number><dates><year>2011</year></dates><publisher>Cochrane Collaboration</publisher><urls></urls></record></Cite></EndNote>(Higgins and Green, 2011) is also a useful source of wisdom and software for simple meta-analyses, but only a powerful mixed modeling procedure (e.g., Proc Mixed in the Statistical Analysis System) can provide the appropriate random effects to deal with repeated measurement represented by multiple study-estimates from the same or different subjects in each study. Expressing Effects in the Same MetricA meta-analyzed effect is a weighted mean of the effect across all selected studies. All effects must therefore be expressed as the same kind of measure in the same units for the mean to be meaningful. Most of the meta-analyses I chose for this article used unsuitable approaches here. For differences or changes in means, a common approach is to standardize each effect by dividing by the appropriate between-subject standard deviation (SD), for example, the baseline SD of all subjects in a controlled trial. Although the magnitude of the resulting effects can be interpreted directly, differences in the SD between studies–reflecting different populations, different methods of sampling, or just sampling variation–will introduce heterogeneity that is unrelated to any real differences in the effect between studies. Josefsson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Josefsson</Author><Year>2014</Year><RecNum>17</RecNum><DisplayText>(2014)</DisplayText><record><rec-number>17</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">17</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Josefsson, Torbj?rn</author><author>Lindwall, Magnus</author><author>Archer, Trevor</author></authors></contributors><titles><title>Physical exercise intervention in depressive disorders: Meta‐analysis and systematic review</title><secondary-title>Scandinavian journal of medicine and science in sports</secondary-title></titles><periodical><full-title>Scandinavian Journal of Medicine and Science in Sports</full-title><abbr-1>Scand. J. Med. Sci. Sports</abbr-1><abbr-2>Scand J Med Sci Sports</abbr-2></periodical><pages>259-272</pages><volume>24</volume><number>2</number><dates><year>2014</year></dates><isbn>1600-0838</isbn><urls></urls></record></Cite></EndNote>(2014) used standardization to cope with the various depression scales in different studies of the effect of exercise interventions on depression. In the six studies using the Beck depression inventory, the range in SD of baseline depression amounted to a factor of 7.5, which translates directly into artefactual heterogeneity. (The lowest SD is actually a standard error in the original publication; Josefsson et al. apparently used it incorrectly to standardize.) The best approach for dealing with disparate psychometric scales, including visual-analogue and multi-level Likert scales, is to linearly rescale all of them to a score with a minimum possible of 0 and a maximum possible of 100. The resulting score then represents an easily analyzed and interpreted percent of full range of the psychological construct. Standardization probably contributed to heterogeneity in the effects of iron supplementation on serum iron concentration, blood hemoglobin concentration and VO2max in the meta-analysis of Burden et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Burden</Author><Year>2015</Year><RecNum>20</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>20</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">20</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Burden, Richard J</author><author>Morton, Katie</author><author>Richards, Toby</author><author>Whyte, Gregory P</author><author>Pedlar, Charles R</author></authors></contributors><titles><title>Is iron treatment beneficial in, iron-deficient but non-anaemic (IDNA) endurance athletes? A meta-analysis</title><secondary-title>British journal of sports medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1389-1397</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite></EndNote>(2015). The authors provided no SD in the various studies to allow assessment of this issue or to convert the meta-analyzed standardized effect back into more meaningful units. By checking data in some of the original publications, I determined that much of the heterogeneity must have arisen from incorrect use of standard errors rather than SD for standardizing some effects. Standardization would also have contributed to heterogeneity in the meta-analysis that Salvador et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Salvador</Author><Year>2016</Year><RecNum>24</RecNum><DisplayText>(2016)</DisplayText><record><rec-number>24</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">24</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Salvador, AF</author><author>De Aguiar, RA</author><author>Lisboa, FD</author><author>Pereira, KL</author><author>Cruz, RS</author><author>Caputo, F</author></authors></contributors><titles><title>Ischemic preconditioning and exercise performance: a systematic review and meta-analysis</title><secondary-title>International journal of sports physiology and performance</secondary-title></titles><periodical><full-title>International Journal of Sports Physiology and Performance</full-title><abbr-1>Int. J. Sports Physiol. Perform.</abbr-1><abbr-2>Int J Sports Physiol Perform</abbr-2><abbr-3>IJSPP</abbr-3></periodical><pages>4-14</pages><volume>11</volume><dates><year>2016</year></dates><isbn>1555-0265</isbn><urls></urls></record></Cite></EndNote>(2016) conducted on the effect of ischemic preconditioning on exercise performance, if they had used the baseline standard deviation to standardize. Unfortunately, through misuse of the analysis program they effectively used the standard error of the change scores, which made the resulting standardized magnitudes meaningless. Log transformation of effects expressed as factors followed by back-transformation to factors or percents is usually the best way to deal with physical performance and many other physiological measures. The decision between analysis of raw vs log-transformed effects hinges on which approach produces less heterogeneity. Peterson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Peterson</Author><Year>2011</Year><RecNum>22</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>22</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">22</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Peterson, Mark D</author><author>Sen, Ananda</author><author>Gordon, Paul M</author></authors></contributors><titles><title>Influence of resistance exercise on lean body mass in aging adults: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>249</pages><volume>43</volume><number>2</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(2011) used raw units (kg) for their meta-analysis of the effect of resistance exercise on lean body mass in aging adults, but if they had used log transformation, the apparently smaller effect in older adults would likely have disappeared or even reversed following back-transformation to percent units. Strangely, they provided irrelevant detail of a method of standardization using the SD of change scores, an error promulgated by the software package Comprehensive Meta-Analysis that again would have made the magnitudes meaningless. Dependent variables in the meta-analyses of Burden et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Burden</Author><Year>2015</Year><RecNum>20</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>20</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">20</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Burden, Richard J</author><author>Morton, Katie</author><author>Richards, Toby</author><author>Whyte, Gregory P</author><author>Pedlar, Charles R</author></authors></contributors><titles><title>Is iron treatment beneficial in, iron-deficient but non-anaemic (IDNA) endurance athletes? A meta-analysis</title><secondary-title>British journal of sports medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1389-1397</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite></EndNote>(2015) and Salvador et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Salvador</Author><Year>2016</Year><RecNum>24</RecNum><DisplayText>(2016)</DisplayText><record><rec-number>24</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">24</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Salvador, AF</author><author>De Aguiar, RA</author><author>Lisboa, FD</author><author>Pereira, KL</author><author>Cruz, RS</author><author>Caputo, F</author></authors></contributors><titles><title>Ischemic preconditioning and exercise performance: a systematic review and meta-analysis</title><secondary-title>International journal of sports physiology and performance</secondary-title></titles><periodical><full-title>International Journal of Sports Physiology and Performance</full-title><abbr-1>Int. J. Sports Physiol. Perform.</abbr-1><abbr-2>Int J Sports Physiol Perform</abbr-2><abbr-3>IJSPP</abbr-3></periodical><pages>4-14</pages><volume>11</volume><dates><year>2016</year></dates><isbn>1555-0265</isbn><urls></urls></record></Cite></EndNote>(2016) were likely candidates for log transformation, as was strength in the meta-analysis of Tomlinson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Tomlinson</Author><Year>2015</Year><RecNum>21</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>21</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">21</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Tomlinson, Peter B</author><author>Joseph, Corey</author><author>Angioi, Manuela</author></authors></contributors><titles><title>Effects of vitamin D supplementation on upper and lower body muscle strength levels in healthy individuals. A systematic review with meta-analysis</title><secondary-title>Journal of Science and Medicine in Sport</secondary-title></titles><periodical><full-title>Journal of Science and Medicine in Sport</full-title><abbr-1>J. Sci. Med. Sport</abbr-1><abbr-2>J Sci Med Sport</abbr-2></periodical><pages>575-580</pages><volume>18</volume><dates><year>2015</year></dates><isbn>1440-2440</isbn><urls></urls></record></Cite></EndNote>(2015) on the effects of supplementation with vitamin D. Chu et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Chu</Author><Year>2016</Year><RecNum>23</RecNum><DisplayText>(2016)</DisplayText><record><rec-number>23</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">23</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Chu, Anna</author><author>Petocz, Peter</author><author>Samman, Samir</author></authors></contributors><titles><title>Immediate effects of aerobic exercise on plasma/serum zinc levels: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>726-733</pages><volume>48</volume><dates><year>2016</year></dates><isbn>0195-9131</isbn><urls></urls></record></Cite></EndNote>(2016) may have made the right choice of raw units of concentration in their meta-analysis of the effect of exercise on plasma zinc, but they did not show enough data from each study for me to assess whether the effects were more uniform as factors.Effects on time-dependent events such as injury incidence are reported as ratios of odds, risks, incidence rates (hazards), or other rates (counts per some measure of exposure). When the risks are low–that is, <10% of the sample experience the event during the period of observation–all these ratios effectively have the same value, so they can be meta-analyzed and interpreted as risk ratios. When risks are higher, the hazard and other rate ratios coincide and are appropriate for interpreting and meta-analyzing factors affecting risk, whereas the odds ratio and the usual relative risk ratio increasingly overestimate and underestimate effects, respectively. Use of odds ratios by Wright et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Wright</Author><Year>2015</Year><RecNum>19</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>19</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">19</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Wright, Alexis A</author><author>Taylor, Jeffrey B</author><author>Ford, Kevin R</author><author>Siska, Lindsey</author><author>Smoliga, James M</author></authors></contributors><titles><title>Risk factors associated with lower extremity stress fractures in runners: a systematic review with meta-analysis</title><secondary-title>British Journal of Sports Medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1517-1523</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite></EndNote>(2015) to meta-analyze risk factors in prospective studies of stress fractures in runners was therefore misguided, and they did not provide injury counts and sample sizes in each study to allow assessment of the extent to which their use of odds ratios introduced upward bias and artefactual heterogeneity in the risk factors they analyzed. (Anyone planning an injury meta-analysis should note that odds ratios in case-control studies are effectively hazard ratios, and that prevalence proportions should be meta-analyzed as odds ratios and converted back to proportion ratios for interpretation of clinical importance.)Validity studies in which a practical measure is compared with a criterion provide several candidate measures for meta-analysis: the correlation coefficient, mean bias, bias at a predicted value, random error, and limits of agreement. The correlation coefficient is sensitive to the between-subject SD, so I would usually avoid it. If biases and errors are more uniform across studies when expressed as percents, they should be converted to factors and log-transformed before analysis. As an SD, the random error suffers from small-sample downward bias, a problem that is easily solved by expressing it as a variance (after any log-transformation), then taking the square root of the meta-analyzed mean. The variance also has a well-defined standard error, which is needed for weighting the effects (see below). The bias in the SD is practically negligible for the usual sample sizes in validity studies, so no real harm was done when Coquart et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(2016a) meta-analyzed SDs of the difference between criterion VO2max in a maximal test and VO2max predicted by submaximal tests. They then converted the meta-analyzed mean bias and mean random error into mean limits of agreement, and in a serious omission they provided no uncertainties (confidence limits) for any of these measures. Furthermore, they showed meta-analyzed random-error components as about ±4 ml.min-1.kg-1, an impossible outcome when the values in the individual studies ranged from ±10 to ±15 ml.min-1.kg-1. Understanding limits of agreement is evidently difficult enough without also having to consider their uncertainty, so this statistic should not be presented as the outcome of a meta-analysis of validity studies, or indeed of reliability studies.Dealing with Standard ErrorsA meta-analyzed effect is a weighted mean of effects, where the weighting factor is the inverse of the square of each effect's expected sampling variation, its standard error. Using a study-quality score as the weighting factor, as Hume et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Hume</Author><Year>2015</Year><RecNum>12</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>12</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">12</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hume, Patria A</author><author>Lorimer, Anna V</author><author>Griffiths, Peter C</author><author>Carlson, Isaac</author><author>Lamont, Mike</author></authors></contributors><titles><title>Recreational snow-sports injury risk factors and countermeasures: a meta-analysis review and Haddon matrix evaluation</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>1175-1190</pages><volume>45</volume><number>8</number><dates><year>2015</year></dates><isbn>0112-1642</isbn><urls></urls></record></Cite></EndNote>(2015) did in a meta-analysis of snow-sport injuries, is incorrect. Depending on the design of the studies and the analysis package, the meta-analyst may input data or inferential statistics (p values or confidence limits) from each study without having to derive or impute the standard error for each effect. Exactly what was done needs to be stated, to satisfy readers that this step was performed correctly and to guide future meta-analyses. Coquart et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(2016a) provided no inferential statistics or information about the standard errors for the two validity statistics they meta-analyzed, bias and random error. Tomlinson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Tomlinson</Author><Year>2015</Year><RecNum>21</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>21</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">21</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Tomlinson, Peter B</author><author>Joseph, Corey</author><author>Angioi, Manuela</author></authors></contributors><titles><title>Effects of vitamin D supplementation on upper and lower body muscle strength levels in healthy individuals. A systematic review with meta-analysis</title><secondary-title>Journal of Science and Medicine in Sport</secondary-title></titles><periodical><full-title>Journal of Science and Medicine in Sport</full-title><abbr-1>J. Sci. Med. Sport</abbr-1><abbr-2>J Sci Med Sport</abbr-2></periodical><pages>575-580</pages><volume>18</volume><dates><year>2015</year></dates><isbn>1440-2440</isbn><urls></urls></record></Cite></EndNote>(2015) input post-intervention means and SD into the meta-analysis software, when they should have input mean pre-post change scores and associated inferential statistics. The way Burden et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Burden</Author><Year>2015</Year><RecNum>20</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>20</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">20</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Burden, Richard J</author><author>Morton, Katie</author><author>Richards, Toby</author><author>Whyte, Gregory P</author><author>Pedlar, Charles R</author></authors></contributors><titles><title>Is iron treatment beneficial in, iron-deficient but non-anaemic (IDNA) endurance athletes? A meta-analysis</title><secondary-title>British journal of sports medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1389-1397</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite></EndNote>(2015) combined pre and post scores is unclear, and Peterson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Peterson</Author><Year>2011</Year><RecNum>22</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>22</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">22</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Peterson, Mark D</author><author>Sen, Ananda</author><author>Gordon, Paul M</author></authors></contributors><titles><title>Influence of resistance exercise on lean body mass in aging adults: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>249</pages><volume>43</volume><number>2</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(2011) did not provide enough data from each study for me to check their analyses. Most of the other meta-analysts did PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DaHU8L0F1dGhvcj48WWVhcj4yMDE2PC9ZZWFyPjxSZWNO

dW0+MjM8L1JlY051bT48RGlzcGxheVRleHQ+KENodSBldCBhbC4sIDIwMTY7IEpvc2Vmc3NvbiBl

dCBhbC4sIDIwMTQ7IFNhbHZhZG9yIGV0IGFsLiwgMjAxNjsgU29vbXJvIGV0IGFsLiwgMjAxNTsg

V3JpZ2h0IGV0IGFsLiwgMjAxNSk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjM8

L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6

Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10

eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1

dGhvcnM+PGF1dGhvcj5DaHUsIEFubmE8L2F1dGhvcj48YXV0aG9yPlBldG9jeiwgUGV0ZXI8L2F1

dGhvcj48YXV0aG9yPlNhbW1hbiwgU2FtaXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRv

cnM+PHRpdGxlcz48dGl0bGU+SW1tZWRpYXRlIGVmZmVjdHMgb2YgYWVyb2JpYyBleGVyY2lzZSBv

biBwbGFzbWEvc2VydW0gemluYyBsZXZlbHM6IGEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29u

ZGFyeS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9ydHMgYW5kIGV4ZXJjaXNlPC9z

ZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TWVkaWNpbmUg

YW5kIFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVsbC10aXRsZT48YWJici0xPk1l

ZC4gU2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5NZWQgU2NpIFNwb3J0IEV4ZXJj

aXNlPC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz43MjYt

NzMzPC9wYWdlcz48dm9sdW1lPjQ4PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMTY8L3llYXI+PC9k

YXRlcz48aXNibj4wMTk1LTkxMzE8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48

Q2l0ZT48QXV0aG9yPkpvc2Vmc3NvbjwvQXV0aG9yPjxZZWFyPjIwMTQ8L1llYXI+PFJlY051bT4x

NzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5

cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5

cngiPjE3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs

ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Kb3NlZnNzb24s

IFRvcmJqw7ZybjwvYXV0aG9yPjxhdXRob3I+TGluZHdhbGwsIE1hZ251czwvYXV0aG9yPjxhdXRo

b3I+QXJjaGVyLCBUcmV2b3I8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxl

cz48dGl0bGU+UGh5c2ljYWwgZXhlcmNpc2UgaW50ZXJ2ZW50aW9uIGluIGRlcHJlc3NpdmUgZGlz

b3JkZXJzOiBNZXRh4oCQYW5hbHlzaXMgYW5kIHN5c3RlbWF0aWMgcmV2aWV3PC90aXRsZT48c2Vj

b25kYXJ5LXRpdGxlPlNjYW5kaW5hdmlhbiBqb3VybmFsIG9mIG1lZGljaW5lIGFuZCBzY2llbmNl

IGluIHNwb3J0czwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp

dGxlPlNjYW5kaW5hdmlhbiBKb3VybmFsIG9mIE1lZGljaW5lIGFuZCBTY2llbmNlIGluIFNwb3J0

czwvZnVsbC10aXRsZT48YWJici0xPlNjYW5kLiBKLiBNZWQuIFNjaS4gU3BvcnRzPC9hYmJyLTE+

PGFiYnItMj5TY2FuZCBKIE1lZCBTY2kgU3BvcnRzPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdl

cz4yNTktMjcyPC9wYWdlcz48dm9sdW1lPjI0PC92b2x1bWU+PG51bWJlcj4yPC9udW1iZXI+PGRh

dGVzPjx5ZWFyPjIwMTQ8L3llYXI+PC9kYXRlcz48aXNibj4xNjAwLTA4Mzg8L2lzYm4+PHVybHM+

PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFll

YXI+MjAxNjwvWWVhcj48UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwv

cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoy

eHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5

cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0

aG9ycz48YXV0aG9yPlNhbHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwv

YXV0aG9yPjxhdXRob3I+TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1

dGhvcj48YXV0aG9yPkNydXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48

L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0

aW9uaW5nIGFuZCBleGVyY2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQg

bWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJu

YWwgb2Ygc3BvcnRzIHBoeXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+

PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9m

IFNwb3J0cyBQaHlzaW9sb2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPklu

dC4gSi4gU3BvcnRzIFBoeXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9y

dHMgUGh5c2lvbCBQZXJmb3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9k

aWNhbD48cGFnZXM+NC0xNDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4y

MDE2PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3Jl

Y29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Tb29tcm88L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFy

PjxSZWNOdW0+MTg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjE4PC9yZWMtbnVtYmVyPjxm

b3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4

OWF6OXgwdnowOXJ4Ij4xODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy

bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+

U29vbXJvLCBOYWplZWJ1bGxhaDwvYXV0aG9yPjxhdXRob3I+U2FuZGVycywgUm9zczwvYXV0aG9y

PjxhdXRob3I+SGFja2V0dCwgRGFuaWVsPC9hdXRob3I+PGF1dGhvcj5IdWJrYSwgVGF0ZTwvYXV0

aG9yPjxhdXRob3I+RWJyYWhpbWksIFNhYWhpbDwvYXV0aG9yPjxhdXRob3I+RnJlZXN0b24sIEpv

bmF0aGFuPC9hdXRob3I+PGF1dGhvcj5Db2JsZXksIFN0ZXBoZW48L2F1dGhvcj48L2F1dGhvcnM+

PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VGhlIGVmaWNhY3kgb2YgaW5qdXJ5IHByZXZl

bnRpb24gcHJvZ3JhbXMgaW4gYWRvbGVzY2VudCB0ZWFtIHNwb3J0czogYSBtZXRhLWFuYWx5c2lz

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFtZXJpY2FuIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGlj

aW5lPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QW1l

cmljYW4gSm91cm5hbCBvZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5BbS4g

Si4gU3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPkFtIEogU3BvcnRzIE1lZDwvYWJici0yPjwv

cGVyaW9kaWNhbD48cGFnZXM+aHR0cHM6Ly9kb2kub3JnLzEwLjExNzcvMDM2MzU0NjUxNTYxODM3

MjwvcGFnZXM+PGRhdGVzPjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4wMzYzLTU0NjU8

L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPldyaWdodDwv

QXV0aG9yPjxZZWFyPjIwMTU8L1llYXI+PFJlY051bT4xOTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1u

dW1iZXI+MTk8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJy

OXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjE5PC9rZXk+PC9mb3JlaWduLWtl

eXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmli

dXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XcmlnaHQsIEFsZXhpcyBBPC9hdXRob3I+PGF1dGhvcj5U

YXlsb3IsIEplZmZyZXkgQjwvYXV0aG9yPjxhdXRob3I+Rm9yZCwgS2V2aW4gUjwvYXV0aG9yPjxh

dXRob3I+U2lza2EsIExpbmRzZXk8L2F1dGhvcj48YXV0aG9yPlNtb2xpZ2EsIEphbWVzIE08L2F1

dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+UmlzayBmYWN0b3Jz

IGFzc29jaWF0ZWQgd2l0aCBsb3dlciBleHRyZW1pdHkgc3RyZXNzIGZyYWN0dXJlcyBpbiBydW5u

ZXJzOiBhIHN5c3RlbWF0aWMgcmV2aWV3IHdpdGggbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29u

ZGFyeS10aXRsZT5Ccml0aXNoIEpvdXJuYWwgb2YgU3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnkt

dGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9m

IFNwb3J0cyBNZWRpY2luZTwvZnVsbC10aXRsZT48YWJici0xPkJyLiBKLiBTcG9ydHMgTWVkLjwv

YWJici0xPjxhYmJyLTI+QnIgSiBTcG9ydHMgTWVkPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdl

cz4xNTE3LTE1MjM8L3BhZ2VzPjx2b2x1bWU+NDk8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwv

eWVhcj48L2RhdGVzPjxpc2JuPjE0NzMtMDQ4MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+

PC9DaXRlPjwvRW5kTm90ZT4A

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DaHU8L0F1dGhvcj48WWVhcj4yMDE2PC9ZZWFyPjxSZWNO

dW0+MjM8L1JlY051bT48RGlzcGxheVRleHQ+KENodSBldCBhbC4sIDIwMTY7IEpvc2Vmc3NvbiBl

dCBhbC4sIDIwMTQ7IFNhbHZhZG9yIGV0IGFsLiwgMjAxNjsgU29vbXJvIGV0IGFsLiwgMjAxNTsg

V3JpZ2h0IGV0IGFsLiwgMjAxNSk8L0Rpc3BsYXlUZXh0PjxyZWNvcmQ+PHJlYy1udW1iZXI+MjM8

L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6

Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIzPC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10

eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1

dGhvcnM+PGF1dGhvcj5DaHUsIEFubmE8L2F1dGhvcj48YXV0aG9yPlBldG9jeiwgUGV0ZXI8L2F1

dGhvcj48YXV0aG9yPlNhbW1hbiwgU2FtaXI8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRv

cnM+PHRpdGxlcz48dGl0bGU+SW1tZWRpYXRlIGVmZmVjdHMgb2YgYWVyb2JpYyBleGVyY2lzZSBv

biBwbGFzbWEvc2VydW0gemluYyBsZXZlbHM6IGEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29u

ZGFyeS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9ydHMgYW5kIGV4ZXJjaXNlPC9z

ZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+TWVkaWNpbmUg

YW5kIFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVsbC10aXRsZT48YWJici0xPk1l

ZC4gU2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5NZWQgU2NpIFNwb3J0IEV4ZXJj

aXNlPC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz43MjYt

NzMzPC9wYWdlcz48dm9sdW1lPjQ4PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIwMTY8L3llYXI+PC9k

YXRlcz48aXNibj4wMTk1LTkxMzE8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48

Q2l0ZT48QXV0aG9yPkpvc2Vmc3NvbjwvQXV0aG9yPjxZZWFyPjIwMTQ8L1llYXI+PFJlY051bT4x

NzwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTc8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5

cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5

cngiPjE3PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNs

ZSI+MTc8L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5Kb3NlZnNzb24s

IFRvcmJqw7ZybjwvYXV0aG9yPjxhdXRob3I+TGluZHdhbGwsIE1hZ251czwvYXV0aG9yPjxhdXRo

b3I+QXJjaGVyLCBUcmV2b3I8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxl

cz48dGl0bGU+UGh5c2ljYWwgZXhlcmNpc2UgaW50ZXJ2ZW50aW9uIGluIGRlcHJlc3NpdmUgZGlz

b3JkZXJzOiBNZXRh4oCQYW5hbHlzaXMgYW5kIHN5c3RlbWF0aWMgcmV2aWV3PC90aXRsZT48c2Vj

b25kYXJ5LXRpdGxlPlNjYW5kaW5hdmlhbiBqb3VybmFsIG9mIG1lZGljaW5lIGFuZCBzY2llbmNl

IGluIHNwb3J0czwvc2Vjb25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRp

dGxlPlNjYW5kaW5hdmlhbiBKb3VybmFsIG9mIE1lZGljaW5lIGFuZCBTY2llbmNlIGluIFNwb3J0

czwvZnVsbC10aXRsZT48YWJici0xPlNjYW5kLiBKLiBNZWQuIFNjaS4gU3BvcnRzPC9hYmJyLTE+

PGFiYnItMj5TY2FuZCBKIE1lZCBTY2kgU3BvcnRzPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdl

cz4yNTktMjcyPC9wYWdlcz48dm9sdW1lPjI0PC92b2x1bWU+PG51bWJlcj4yPC9udW1iZXI+PGRh

dGVzPjx5ZWFyPjIwMTQ8L3llYXI+PC9kYXRlcz48aXNibj4xNjAwLTA4Mzg8L2lzYm4+PHVybHM+

PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFll

YXI+MjAxNjwvWWVhcj48UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwv

cmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoy

eHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5

cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0

aG9ycz48YXV0aG9yPlNhbHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwv

YXV0aG9yPjxhdXRob3I+TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1

dGhvcj48YXV0aG9yPkNydXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48

L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0

aW9uaW5nIGFuZCBleGVyY2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQg

bWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJu

YWwgb2Ygc3BvcnRzIHBoeXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+

PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9m

IFNwb3J0cyBQaHlzaW9sb2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPklu

dC4gSi4gU3BvcnRzIFBoeXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9y

dHMgUGh5c2lvbCBQZXJmb3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9k

aWNhbD48cGFnZXM+NC0xNDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4y

MDE2PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3Jl

Y29yZD48L0NpdGU+PENpdGU+PEF1dGhvcj5Tb29tcm88L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFy

PjxSZWNOdW0+MTg8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjE4PC9yZWMtbnVtYmVyPjxm

b3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4

OWF6OXgwdnowOXJ4Ij4xODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy

bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+

U29vbXJvLCBOYWplZWJ1bGxhaDwvYXV0aG9yPjxhdXRob3I+U2FuZGVycywgUm9zczwvYXV0aG9y

PjxhdXRob3I+SGFja2V0dCwgRGFuaWVsPC9hdXRob3I+PGF1dGhvcj5IdWJrYSwgVGF0ZTwvYXV0

aG9yPjxhdXRob3I+RWJyYWhpbWksIFNhYWhpbDwvYXV0aG9yPjxhdXRob3I+RnJlZXN0b24sIEpv

bmF0aGFuPC9hdXRob3I+PGF1dGhvcj5Db2JsZXksIFN0ZXBoZW48L2F1dGhvcj48L2F1dGhvcnM+

PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+VGhlIGVmaWNhY3kgb2YgaW5qdXJ5IHByZXZl

bnRpb24gcHJvZ3JhbXMgaW4gYWRvbGVzY2VudCB0ZWFtIHNwb3J0czogYSBtZXRhLWFuYWx5c2lz

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkFtZXJpY2FuIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGlj

aW5lPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QW1l

cmljYW4gSm91cm5hbCBvZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5BbS4g

Si4gU3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPkFtIEogU3BvcnRzIE1lZDwvYWJici0yPjwv

cGVyaW9kaWNhbD48cGFnZXM+aHR0cHM6Ly9kb2kub3JnLzEwLjExNzcvMDM2MzU0NjUxNTYxODM3

MjwvcGFnZXM+PGRhdGVzPjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4wMzYzLTU0NjU8

L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPldyaWdodDwv

QXV0aG9yPjxZZWFyPjIwMTU8L1llYXI+PFJlY051bT4xOTwvUmVjTnVtPjxyZWNvcmQ+PHJlYy1u

dW1iZXI+MTk8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJy

OXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjE5PC9rZXk+PC9mb3JlaWduLWtl

eXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250cmli

dXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5XcmlnaHQsIEFsZXhpcyBBPC9hdXRob3I+PGF1dGhvcj5U

YXlsb3IsIEplZmZyZXkgQjwvYXV0aG9yPjxhdXRob3I+Rm9yZCwgS2V2aW4gUjwvYXV0aG9yPjxh

dXRob3I+U2lza2EsIExpbmRzZXk8L2F1dGhvcj48YXV0aG9yPlNtb2xpZ2EsIEphbWVzIE08L2F1

dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+UmlzayBmYWN0b3Jz

IGFzc29jaWF0ZWQgd2l0aCBsb3dlciBleHRyZW1pdHkgc3RyZXNzIGZyYWN0dXJlcyBpbiBydW5u

ZXJzOiBhIHN5c3RlbWF0aWMgcmV2aWV3IHdpdGggbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29u

ZGFyeS10aXRsZT5Ccml0aXNoIEpvdXJuYWwgb2YgU3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnkt

dGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9m

IFNwb3J0cyBNZWRpY2luZTwvZnVsbC10aXRsZT48YWJici0xPkJyLiBKLiBTcG9ydHMgTWVkLjwv

YWJici0xPjxhYmJyLTI+QnIgSiBTcG9ydHMgTWVkPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdl

cz4xNTE3LTE1MjM8L3BhZ2VzPjx2b2x1bWU+NDk8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwv

eWVhcj48L2RhdGVzPjxpc2JuPjE0NzMtMDQ4MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+

PC9DaXRlPjwvRW5kTm90ZT4A

ADDIN EN.CITE.DATA (Chu et al., 2016; Josefsson et al., 2014; Salvador et al., 2016; Soomro et al., 2015; Wright et al., 2015), but only Chu et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Chu</Author><Year>2016</Year><RecNum>23</RecNum><DisplayText>(2016)</DisplayText><record><rec-number>23</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">23</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Chu, Anna</author><author>Petocz, Peter</author><author>Samman, Samir</author></authors></contributors><titles><title>Immediate effects of aerobic exercise on plasma/serum zinc levels: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>726-733</pages><volume>48</volume><dates><year>2016</year></dates><isbn>0195-9131</isbn><urls></urls></record></Cite></EndNote>(2016) and Soomro et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Soomro</Author><Year>2015</Year><RecNum>18</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">18</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Soomro, Najeebullah</author><author>Sanders, Ross</author><author>Hackett, Daniel</author><author>Hubka, Tate</author><author>Ebrahimi, Saahil</author><author>Freeston, Jonathan</author><author>Cobley, Stephen</author></authors></contributors><titles><title>The eficacy of injury prevention programs in adolescent team sports: a meta-analysis</title><secondary-title>American journal of sports medicine</secondary-title></titles><periodical><full-title>American Journal of Sports Medicine</full-title><abbr-1>Am. J. Sports Med.</abbr-1><abbr-2>Am J Sports Med</abbr-2></periodical><pages>;(2015) also provided adequate documentation. Accounting for HeterogeneityHeterogeneity in a meta-analysis refers to real differences between effect magnitudes, which arise not from sampling variation but from moderation of the effect by differences between studies in subject characteristics, environmental factors, study design, measurement techniques, and/or method of analysis. The typical practice of testing for heterogeneity with the I2 statistic is futile, because non-significance does not usually exclude the possibility of substantial heterogeneity, and neither the I2 nor the related Q statistic properly represent the magnitude of heterogeneity ADDIN EN.CITE <EndNote><Cite><Author>Higgins</Author><Year>2008</Year><RecNum>14</RecNum><DisplayText>(Higgins, 2008)</DisplayText><record><rec-number>14</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">14</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Higgins, J P T</author></authors></contributors><titles><title>Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified</title><secondary-title>International Journal of Epidemiology</secondary-title></titles><periodical><full-title>International Journal of Epidemiology</full-title><abbr-1>Int. J. Epidemiol.</abbr-1><abbr-2>Int J Epidemiol</abbr-2></periodical><pages>1158-1160</pages><volume>37</volume><dates><year>2008</year></dates><urls></urls></record></Cite></EndNote>(Higgins, 2008). The best statistic to deal with heterogeneity is the SD derived from a study-estimate random effect (often represented by ), which should be included in all meta-analyses. This SD should be spelt out to readers as the typical difference in the true effects in different study settings. As such, it may be as important as the mean effect, in the same way that individual differences in the effect of an intervention may be as important as the mean. As with the SD representing individual differences, it should be doubled before it is interpreted against the magnitude thresholds normally used to interpret differences in means, or equivalently, the thresholds should be halved ADDIN EN.CITE <EndNote><Cite><Author>Hopkins</Author><Year>2015</Year><RecNum>2</RecNum><DisplayText>(Hopkins, 2015)</DisplayText><record><rec-number>2</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">2</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hopkins, W G</author></authors></contributors><titles><title>Individual responses made easy</title><secondary-title>Journal of Applied Physiology</secondary-title></titles><periodical><full-title>Journal of Applied Physiology</full-title><abbr-1>J. Appl. Physiol.</abbr-1><abbr-2>J Appl Physiol</abbr-2></periodical><pages>1444-1446</pages><volume>118</volume><dates><year>2015</year></dates><urls></urls><electronic-resource-num>10.?1152/?japplphysiol.?00098.?2015 </electronic-resource-num></record></Cite></EndNote>(Hopkins, 2015). The SD has its own uncertainty, which needs to be estimated, presented, and taken into account in the interpretation of its magnitude, and the analysis has to allow for negative estimates of its value and of its confidence limits. Having shown that there could be substantial heterogeneity (i.e., the upper confidence limit is substantial), the meta-analyst should then try to explain it by performing separate meta-analyses of subgroups of studies or by performing a meta-regression with study characteristics, including mean subject characteristics, as predictors. The latter approach is preferable, especially when there are enough studies to allow for proper estimation and adjustment for mutual confounding with multiple predictors. These two approaches have been referred to disparagingly as deluded and daft, respectively, by authors who advocate what they call a deft approach: separate meta-analysis of the modifying effect of a given subject characteristic from each study ADDIN EN.CITE <EndNote><Cite><Author>Fisher</Author><Year>2017</Year><RecNum>36</RecNum><DisplayText>(Fisher et al., 2017)</DisplayText><record><rec-number>36</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">36</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Fisher, David J</author><author>Carpenter, James R</author><author>Morris, Tim P</author><author>Freeman, Suzanne C</author><author>Tierney, Jayne F</author></authors></contributors><titles><title>Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach?</title><secondary-title>BMJ</secondary-title></titles><periodical><full-title>BMJ</full-title><abbr-1>BMJ</abbr-1><abbr-2>BMJ</abbr-2></periodical><pages>j573</pages><volume>356</volume><dates><year>2017</year></dates><isbn>0959-8138</isbn><urls></urls></record></Cite></EndNote>(Fisher et al., 2017). Unfortunately, the deft approach is generally unrealizable: either the studies are done on relatively homogeneous groups of subjects (to obtain better precision for the kind of subject studied), or the authors do not report adequate information for the effect of the subject characteristic (e.g., only a p-value inequality), or they simply did not investigate the modifying effect of the subject characteristic. Meta-analysis with individual-participant data is probably the best way to account for modifying effects, but until such data become generally available, I recommend the daft approach, taking care to reduce bias by including potentially confounding study and mean subject characteristics in a single meta-regression. The daft approach still has the potential problem of so-called ecological bias, "whereby [modifying effects] at the aggregate level might not reflect the true [modifying effects] at the individual participant level" ADDIN EN.CITE <EndNote><Cite><Author>Fisher</Author><Year>2017</Year><RecNum>36</RecNum><DisplayText>(Fisher et al., 2017)</DisplayText><record><rec-number>36</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">36</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Fisher, David J</author><author>Carpenter, James R</author><author>Morris, Tim P</author><author>Freeman, Suzanne C</author><author>Tierney, Jayne F</author></authors></contributors><titles><title>Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach?</title><secondary-title>BMJ</secondary-title></titles><periodical><full-title>BMJ</full-title><abbr-1>BMJ</abbr-1><abbr-2>BMJ</abbr-2></periodical><pages>j573</pages><volume>356</volume><dates><year>2017</year></dates><isbn>0959-8138</isbn><urls></urls></record></Cite></EndNote>(Fisher et al., 2017). True, but analysis of individual participant data can itself produce biased estimates, such as attenuation by error of measurement, which is likely negligible with estimates based on study means. Daft is inappropriately dismissive.Of the meta-analyses I reviewed for this article, only that of Soomro et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Soomro</Author><Year>2015</Year><RecNum>18</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">18</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Soomro, Najeebullah</author><author>Sanders, Ross</author><author>Hackett, Daniel</author><author>Hubka, Tate</author><author>Ebrahimi, Saahil</author><author>Freeston, Jonathan</author><author>Cobley, Stephen</author></authors></contributors><titles><title>The eficacy of injury prevention programs in adolescent team sports: a meta-analysis</title><secondary-title>American journal of sports medicine</secondary-title></titles><periodical><full-title>American Journal of Sports Medicine</full-title><abbr-1>Am. J. Sports Med.</abbr-1><abbr-2>Am J Sports Med</abbr-2></periodical><pages>;(2015), on injury-prevention programs in team sports, included adequate assessment of heterogeneity and subgroup analyses. They did not present the random-effect SD in comprehensible units, nor did they present its uncertainty, but they did provide a prediction interval ADDIN EN.CITE <EndNote><Cite><Author>Higgins</Author><Year>2008</Year><RecNum>14</RecNum><DisplayText>(Higgins, 2008; Higgins et al., 2009)</DisplayText><record><rec-number>14</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">14</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Higgins, J P T</author></authors></contributors><titles><title>Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified</title><secondary-title>International Journal of Epidemiology</secondary-title></titles><periodical><full-title>International Journal of Epidemiology</full-title><abbr-1>Int. J. Epidemiol.</abbr-1><abbr-2>Int J Epidemiol</abbr-2></periodical><pages>1158-1160</pages><volume>37</volume><dates><year>2008</year></dates><urls></urls></record></Cite><Cite><Author>Higgins</Author><Year>2009</Year><RecNum>35</RecNum><record><rec-number>35</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">35</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Higgins, J P T</author><author>Thompson., S G</author><author>Spiegelhalter, D J</author></authors></contributors><titles><title>A re-evaluation of random-effects meta-analysis</title><secondary-title>Jurnal of the Royal Statistical Society A</secondary-title></titles><pages>137-159</pages><volume>172</volume><dates><year>2009</year></dates><urls></urls></record></Cite></EndNote>(Higgins, 2008; Higgins et al., 2009) representing the range of the true effect in 95% of study settings, akin to the reference range of a clinical test measure. Higgins et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Higgins</Author><Year>2009</Year><RecNum>35</RecNum><DisplayText>(2009)</DisplayText><record><rec-number>35</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">35</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Higgins, J P T</author><author>Thompson., S G</author><author>Spiegelhalter, D J</author></authors></contributors><titles><title>A re-evaluation of random-effects meta-analysis</title><secondary-title>Jurnal of the Royal Statistical Society A</secondary-title></titles><pages>137-159</pages><volume>172</volume><dates><year>2009</year></dates><urls></urls></record></Cite></EndNote>(2009) suggested that the prediction interval can be calculated by assuming the true values of the effect in different study settings have a t distribution centered on the meta-analyzed mean, with a variance given by the sum of the random-effect variance (2) and error variance of the mean, and with degrees of freedom equal to the number of studies minus 2. These assumptions appear to me to be untenable, when there is sufficient uncertainty in 2 for its confidence interval to include negative values, which is the usual scenario with the kind of small-scale meta-analyses we see in our disciplines. Bootstrapping could be used to derive the lower and upper confidence limits for the lower and upper limits of the prediction interval (by setting negative values of 2 in the bootstrapped samples to zero), but easier statistics to understand would be the proportions of study settings showing substantially positive, substantially negative, and trivial true effects. The meta-analytic models almost invariably involve the unrealistic assumption of a single estimate of 2 for different predicted means, so on balance I think it is best to go no further than interpreting and its confidence limits simply as the SD representing typical differences in the true effects between study settings.Most other meta-analysts performed random-effect analyses and subgroup analyses, but any conclusions they based on the I2 and Q statistics should be ignored. Josefsson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Josefsson</Author><Year>2014</Year><RecNum>17</RecNum><DisplayText>(2014)</DisplayText><record><rec-number>17</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">17</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Josefsson, Torbj?rn</author><author>Lindwall, Magnus</author><author>Archer, Trevor</author></authors></contributors><titles><title>Physical exercise intervention in depressive disorders: Meta‐analysis and systematic review</title><secondary-title>Scandinavian journal of medicine and science in sports</secondary-title></titles><periodical><full-title>Scandinavian Journal of Medicine and Science in Sports</full-title><abbr-1>Scand. J. Med. Sci. Sports</abbr-1><abbr-2>Scand J Med Sci Sports</abbr-2></periodical><pages>259-272</pages><volume>24</volume><number>2</number><dates><year>2014</year></dates><isbn>1600-0838</isbn><urls></urls></record></Cite></EndNote>(2014) and Soomro et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Soomro</Author><Year>2015</Year><RecNum>18</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>18</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">18</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Soomro, Najeebullah</author><author>Sanders, Ross</author><author>Hackett, Daniel</author><author>Hubka, Tate</author><author>Ebrahimi, Saahil</author><author>Freeston, Jonathan</author><author>Cobley, Stephen</author></authors></contributors><titles><title>The eficacy of injury prevention programs in adolescent team sports: a meta-analysis</title><secondary-title>American journal of sports medicine</secondary-title></titles><periodical><full-title>American Journal of Sports Medicine</full-title><abbr-1>Am. J. Sports Med.</abbr-1><abbr-2>Am J Sports Med</abbr-2></periodical><pages>;(2015) deserve a commendation for including study quality in a subgroup or moderator analysis. Only Peterson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Peterson</Author><Year>2011</Year><RecNum>22</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>22</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">22</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Peterson, Mark D</author><author>Sen, Ananda</author><author>Gordon, Paul M</author></authors></contributors><titles><title>Influence of resistance exercise on lean body mass in aging adults: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>249</pages><volume>43</volume><number>2</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(2011) performed meta-regression with multiple study characteristics. Burden et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Burden</Author><Year>2015</Year><RecNum>20</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>20</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">20</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Burden, Richard J</author><author>Morton, Katie</author><author>Richards, Toby</author><author>Whyte, Gregory P</author><author>Pedlar, Charles R</author></authors></contributors><titles><title>Is iron treatment beneficial in, iron-deficient but non-anaemic (IDNA) endurance athletes? A meta-analysis</title><secondary-title>British journal of sports medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1389-1397</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite></EndNote>(2015) used only a fixed-effects model but included covariates when they thought the I2 statistic indicated heterogeneity. Hume et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Hume</Author><Year>2015</Year><RecNum>12</RecNum><DisplayText>(2015)</DisplayText><record><rec-number>12</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">12</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hume, Patria A</author><author>Lorimer, Anna V</author><author>Griffiths, Peter C</author><author>Carlson, Isaac</author><author>Lamont, Mike</author></authors></contributors><titles><title>Recreational snow-sports injury risk factors and countermeasures: a meta-analysis review and Haddon matrix evaluation</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>1175-1190</pages><volume>45</volume><number>8</number><dates><year>2015</year></dates><isbn>0112-1642</isbn><urls></urls></record></Cite></EndNote>(2015) performed fixed-effects meta-analyses without any consideration of heterogeneity, but they did perform subgroup analyses.Coping with Repeated MeasurementA given study often provides several estimates of an effect that can be included in a meta-analysis, such as effects on males and females, or effects for different doses or time points. Such effects represent repeated measurement on the same study, so the usual meta-analytic mixed model with a single between-study random effect is not appropriate. Meta-analysts in the studies I reviewed attempted to cope with this problem either by treating the estimates as if they came from separate studies PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IENodSBl

dCBhbC4sIDIwMTY7IFBldGVyc29uIGV0IGFsLiwgMjAxMTsgU2FsdmFkb3IgZXQgYWwuLCAyMDE2

OyBUb21saW5zb24gZXQgYWwuLCAyMDE1KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJl

cj4yMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4

NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjA8L2tleT48L2ZvcmVpZ24ta2V5cz48

cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y

cz48YXV0aG9ycz48YXV0aG9yPkJ1cmRlbiwgUmljaGFyZCBKPC9hdXRob3I+PGF1dGhvcj5Nb3J0

b24sIEthdGllPC9hdXRob3I+PGF1dGhvcj5SaWNoYXJkcywgVG9ieTwvYXV0aG9yPjxhdXRob3I+

V2h5dGUsIEdyZWdvcnkgUDwvYXV0aG9yPjxhdXRob3I+UGVkbGFyLCBDaGFybGVzIFI8L2F1dGhv

cj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXMgaXJvbiB0cmVhdG1l

bnQgYmVuZWZpY2lhbCBpbiwgaXJvbi1kZWZpY2llbnQgYnV0IG5vbi1hbmFlbWljIChJRE5BKSBl

bmR1cmFuY2UgYXRobGV0ZXM/IEEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRs

ZT5Ccml0aXNoIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90

aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9mIFNwb3J0cyBN

ZWRpY2luZTwvZnVsbC10aXRsZT48YWJici0xPkJyLiBKLiBTcG9ydHMgTWVkLjwvYWJici0xPjxh

YmJyLTI+QnIgSiBTcG9ydHMgTWVkPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMzg5LTEz

OTc8L3BhZ2VzPjx2b2x1bWU+NDk8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2Rh

dGVzPjxpc2JuPjE0NzMtMDQ4MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+VG9tbGluc29uPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjIx

PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yMTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlz

PjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDly

eCI+MjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl

Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlRvbWxpbnNvbiwg

UGV0ZXIgQjwvYXV0aG9yPjxhdXRob3I+Sm9zZXBoLCBDb3JleTwvYXV0aG9yPjxhdXRob3I+QW5n

aW9pLCBNYW51ZWxhPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp

dGxlPkVmZmVjdHMgb2Ygdml0YW1pbiBEIHN1cHBsZW1lbnRhdGlvbiBvbiB1cHBlciBhbmQgbG93

ZXIgYm9keSBtdXNjbGUgc3RyZW5ndGggbGV2ZWxzIGluIGhlYWx0aHkgaW5kaXZpZHVhbHMuIEEg

c3lzdGVtYXRpYyByZXZpZXcgd2l0aCBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRp

dGxlPkpvdXJuYWwgb2YgU2NpZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L3NlY29uZGFyeS10

aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNjaWVuY2Ug

YW5kIE1lZGljaW5lIGluIFNwb3J0PC9mdWxsLXRpdGxlPjxhYmJyLTE+Si4gU2NpLiBNZWQuIFNw

b3J0PC9hYmJyLTE+PGFiYnItMj5KIFNjaSBNZWQgU3BvcnQ8L2FiYnItMj48L3BlcmlvZGljYWw+

PHBhZ2VzPjU3NS01ODA8L3BhZ2VzPjx2b2x1bWU+MTg8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAx

NTwveWVhcj48L2RhdGVzPjxpc2JuPjE0NDAtMjQ0MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNv

cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+UGV0ZXJzb248L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFy

PjxSZWNOdW0+MjI8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjIyPC9yZWMtbnVtYmVyPjxm

b3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4

OWF6OXgwdnowOXJ4Ij4yMjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy

bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+

UGV0ZXJzb24sIE1hcmsgRDwvYXV0aG9yPjxhdXRob3I+U2VuLCBBbmFuZGE8L2F1dGhvcj48YXV0

aG9yPkdvcmRvbiwgUGF1bCBNPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRs

ZXM+PHRpdGxlPkluZmx1ZW5jZSBvZiByZXNpc3RhbmNlIGV4ZXJjaXNlIG9uIGxlYW4gYm9keSBt

YXNzIGluIGFnaW5nIGFkdWx0czogYSBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRp

dGxlPk1lZGljaW5lIGFuZCBzY2llbmNlIGluIHNwb3J0cyBhbmQgZXhlcmNpc2U8L3NlY29uZGFy

eS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5NZWRpY2luZSBhbmQgU2Np

ZW5jZSBpbiBTcG9ydHMgYW5kIEV4ZXJjaXNlPC9mdWxsLXRpdGxlPjxhYmJyLTE+TWVkLiBTY2ku

IFNwb3J0cyBFeGVyYy48L2FiYnItMT48YWJici0yPk1lZCBTY2kgU3BvcnQgRXhlcmNpc2U8L2Fi

YnItMj48YWJici0zPk1TU0U8L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjI0OTwvcGFnZXM+

PHZvbHVtZT40Mzwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDExPC95

ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNo

dTwvQXV0aG9yPjxZZWFyPjIwMTY8L1llYXI+PFJlY051bT4yMzwvUmVjTnVtPjxyZWNvcmQ+PHJl

Yy1udW1iZXI+MjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlk

PSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIzPC9rZXk+PC9mb3JlaWdu

LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250

cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DaHUsIEFubmE8L2F1dGhvcj48YXV0aG9yPlBldG9j

eiwgUGV0ZXI8L2F1dGhvcj48YXV0aG9yPlNhbW1hbiwgU2FtaXI8L2F1dGhvcj48L2F1dGhvcnM+

PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SW1tZWRpYXRlIGVmZmVjdHMgb2YgYWVyb2Jp

YyBleGVyY2lzZSBvbiBwbGFzbWEvc2VydW0gemluYyBsZXZlbHM6IGEgbWV0YS1hbmFseXNpczwv

dGl0bGU+PHNlY29uZGFyeS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9ydHMgYW5k

IGV4ZXJjaXNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0

bGU+TWVkaWNpbmUgYW5kIFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVsbC10aXRs

ZT48YWJici0xPk1lZC4gU2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5NZWQgU2Np

IFNwb3J0IEV4ZXJjaXNlPC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJpb2RpY2Fs

PjxwYWdlcz43MjYtNzMzPC9wYWdlcz48dm9sdW1lPjQ4PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIw

MTY8L3llYXI+PC9kYXRlcz48aXNibj4wMTk1LTkxMzE8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVj

b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFllYXI+MjAxNjwvWWVh

cj48UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwvcmVjLW51bWJlcj48

Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3Nm

eDlhejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91

cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y

PlNhbHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwvYXV0aG9yPjxhdXRo

b3I+TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1dGhvcj48YXV0aG9y

PkNydXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48L2F1dGhvcnM+PC9j

b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0aW9uaW5nIGFuZCBl

eGVyY2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1hbmFseXNp

czwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJuYWwgb2Ygc3BvcnRz

IHBoeXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBl

cmlvZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9mIFNwb3J0cyBQaHlz

aW9sb2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPkludC4gSi4gU3BvcnRz

IFBoeXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9ydHMgUGh5c2lvbCBQ

ZXJmb3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+

NC0xNDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFyPjwv

ZGF0ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+

PC9FbmROb3RlPn==

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IENodSBl

dCBhbC4sIDIwMTY7IFBldGVyc29uIGV0IGFsLiwgMjAxMTsgU2FsdmFkb3IgZXQgYWwuLCAyMDE2

OyBUb21saW5zb24gZXQgYWwuLCAyMDE1KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJl

cj4yMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4

NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjA8L2tleT48L2ZvcmVpZ24ta2V5cz48

cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y

cz48YXV0aG9ycz48YXV0aG9yPkJ1cmRlbiwgUmljaGFyZCBKPC9hdXRob3I+PGF1dGhvcj5Nb3J0

b24sIEthdGllPC9hdXRob3I+PGF1dGhvcj5SaWNoYXJkcywgVG9ieTwvYXV0aG9yPjxhdXRob3I+

V2h5dGUsIEdyZWdvcnkgUDwvYXV0aG9yPjxhdXRob3I+UGVkbGFyLCBDaGFybGVzIFI8L2F1dGhv

cj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXMgaXJvbiB0cmVhdG1l

bnQgYmVuZWZpY2lhbCBpbiwgaXJvbi1kZWZpY2llbnQgYnV0IG5vbi1hbmFlbWljIChJRE5BKSBl

bmR1cmFuY2UgYXRobGV0ZXM/IEEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRs

ZT5Ccml0aXNoIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90

aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9mIFNwb3J0cyBN

ZWRpY2luZTwvZnVsbC10aXRsZT48YWJici0xPkJyLiBKLiBTcG9ydHMgTWVkLjwvYWJici0xPjxh

YmJyLTI+QnIgSiBTcG9ydHMgTWVkPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMzg5LTEz

OTc8L3BhZ2VzPjx2b2x1bWU+NDk8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2Rh

dGVzPjxpc2JuPjE0NzMtMDQ4MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+VG9tbGluc29uPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjIx

PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yMTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlz

PjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDly

eCI+MjE8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xl

Ij4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlRvbWxpbnNvbiwg

UGV0ZXIgQjwvYXV0aG9yPjxhdXRob3I+Sm9zZXBoLCBDb3JleTwvYXV0aG9yPjxhdXRob3I+QW5n

aW9pLCBNYW51ZWxhPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRp

dGxlPkVmZmVjdHMgb2Ygdml0YW1pbiBEIHN1cHBsZW1lbnRhdGlvbiBvbiB1cHBlciBhbmQgbG93

ZXIgYm9keSBtdXNjbGUgc3RyZW5ndGggbGV2ZWxzIGluIGhlYWx0aHkgaW5kaXZpZHVhbHMuIEEg

c3lzdGVtYXRpYyByZXZpZXcgd2l0aCBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRp

dGxlPkpvdXJuYWwgb2YgU2NpZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L3NlY29uZGFyeS10

aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNjaWVuY2Ug

YW5kIE1lZGljaW5lIGluIFNwb3J0PC9mdWxsLXRpdGxlPjxhYmJyLTE+Si4gU2NpLiBNZWQuIFNw

b3J0PC9hYmJyLTE+PGFiYnItMj5KIFNjaSBNZWQgU3BvcnQ8L2FiYnItMj48L3BlcmlvZGljYWw+

PHBhZ2VzPjU3NS01ODA8L3BhZ2VzPjx2b2x1bWU+MTg8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAx

NTwveWVhcj48L2RhdGVzPjxpc2JuPjE0NDAtMjQ0MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNv

cmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+UGV0ZXJzb248L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFy

PjxSZWNOdW0+MjI8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjIyPC9yZWMtbnVtYmVyPjxm

b3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4

OWF6OXgwdnowOXJ4Ij4yMjwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3Vy

bmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+

UGV0ZXJzb24sIE1hcmsgRDwvYXV0aG9yPjxhdXRob3I+U2VuLCBBbmFuZGE8L2F1dGhvcj48YXV0

aG9yPkdvcmRvbiwgUGF1bCBNPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRs

ZXM+PHRpdGxlPkluZmx1ZW5jZSBvZiByZXNpc3RhbmNlIGV4ZXJjaXNlIG9uIGxlYW4gYm9keSBt

YXNzIGluIGFnaW5nIGFkdWx0czogYSBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRp

dGxlPk1lZGljaW5lIGFuZCBzY2llbmNlIGluIHNwb3J0cyBhbmQgZXhlcmNpc2U8L3NlY29uZGFy

eS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5NZWRpY2luZSBhbmQgU2Np

ZW5jZSBpbiBTcG9ydHMgYW5kIEV4ZXJjaXNlPC9mdWxsLXRpdGxlPjxhYmJyLTE+TWVkLiBTY2ku

IFNwb3J0cyBFeGVyYy48L2FiYnItMT48YWJici0yPk1lZCBTY2kgU3BvcnQgRXhlcmNpc2U8L2Fi

YnItMj48YWJici0zPk1TU0U8L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjI0OTwvcGFnZXM+

PHZvbHVtZT40Mzwvdm9sdW1lPjxudW1iZXI+MjwvbnVtYmVyPjxkYXRlcz48eWVhcj4yMDExPC95

ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPkNo

dTwvQXV0aG9yPjxZZWFyPjIwMTY8L1llYXI+PFJlY051bT4yMzwvUmVjTnVtPjxyZWNvcmQ+PHJl

Yy1udW1iZXI+MjM8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0iRU4iIGRiLWlk

PSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIzPC9rZXk+PC9mb3JlaWdu

LWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10eXBlPjxjb250

cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5DaHUsIEFubmE8L2F1dGhvcj48YXV0aG9yPlBldG9j

eiwgUGV0ZXI8L2F1dGhvcj48YXV0aG9yPlNhbW1hbiwgU2FtaXI8L2F1dGhvcj48L2F1dGhvcnM+

PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SW1tZWRpYXRlIGVmZmVjdHMgb2YgYWVyb2Jp

YyBleGVyY2lzZSBvbiBwbGFzbWEvc2VydW0gemluYyBsZXZlbHM6IGEgbWV0YS1hbmFseXNpczwv

dGl0bGU+PHNlY29uZGFyeS10aXRsZT5NZWRpY2luZSBhbmQgc2NpZW5jZSBpbiBzcG9ydHMgYW5k

IGV4ZXJjaXNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0

bGU+TWVkaWNpbmUgYW5kIFNjaWVuY2UgaW4gU3BvcnRzIGFuZCBFeGVyY2lzZTwvZnVsbC10aXRs

ZT48YWJici0xPk1lZC4gU2NpLiBTcG9ydHMgRXhlcmMuPC9hYmJyLTE+PGFiYnItMj5NZWQgU2Np

IFNwb3J0IEV4ZXJjaXNlPC9hYmJyLTI+PGFiYnItMz5NU1NFPC9hYmJyLTM+PC9wZXJpb2RpY2Fs

PjxwYWdlcz43MjYtNzMzPC9wYWdlcz48dm9sdW1lPjQ4PC92b2x1bWU+PGRhdGVzPjx5ZWFyPjIw

MTY8L3llYXI+PC9kYXRlcz48aXNibj4wMTk1LTkxMzE8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVj

b3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFllYXI+MjAxNjwvWWVh

cj48UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwvcmVjLW51bWJlcj48

Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3Nm

eDlhejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91

cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9y

PlNhbHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwvYXV0aG9yPjxhdXRo

b3I+TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1dGhvcj48YXV0aG9y

PkNydXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48L2F1dGhvcnM+PC9j

b250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0aW9uaW5nIGFuZCBl

eGVyY2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1hbmFseXNp

czwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJuYWwgb2Ygc3BvcnRz

IHBoeXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBl

cmlvZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9mIFNwb3J0cyBQaHlz

aW9sb2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPkludC4gSi4gU3BvcnRz

IFBoeXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9ydHMgUGh5c2lvbCBQ

ZXJmb3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+

NC0xNDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFyPjwv

ZGF0ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+

PC9FbmROb3RlPn==

ADDIN EN.CITE.DATA (Burden et al., 2015; Chu et al., 2016; Peterson et al., 2011; Salvador et al., 2016; Tomlinson et al., 2015), or by performing subgroup analyses that did not include repeated measurement ADDIN EN.CITE <EndNote><Cite><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(Coquart et al., 2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(Coquart et al., 2016a). The problem with the former approach is that the resulting confidence interval for the overall mean effect is too narrow, while the resulting confidence intervals for any within-study moderators included in a meta-regression are wider than they need be. The problem with the latter approach is that the separately meta-analyzed effects in the subgroups cannot be compared inferentially, because they are not independent. The study of Coquart et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(2016a) illustrates how wrong conclusions can be reached with an inappropriate analysis. They found similar meta-analyzed mean estimates of VO2max when they performed separate analyses for estimates predicted at perceived exertions of 19 and 20, but as you would expect, VO2max was substantially higher at the higher intensity in those studies where VO2max was predicted at both intensities, and I have little doubt that the right kind of meta-analysis would show that difference clearly.The correct approach to including and comparing multiple within-study estimates is a repeated-measures meta-regression, achieved by including one or more covariates to account for and estimate the within-study effects, and by including one or more random effects additional to the usual between-study random effect to account for clustering of estimates within studies. I have only ever included a single additional random effect in meta-analyses PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DYXJyPC9BdXRob3I+PFllYXI+MjAxMTwvWWVhcj48UmVj

TnVtPjc8L1JlY051bT48RGlzcGxheVRleHQ+KENhcnIgZXQgYWwuLCAyMDExOyBWYW5kZW5ib2dh

ZXJkZSBhbmQgSG9wa2lucywgMjAxMTsgV2VzdG9uIGV0IGFsLiwgMjAxNCk8L0Rpc3BsYXlUZXh0

PjxyZWNvcmQ+PHJlYy1udW1iZXI+NzwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBw

PSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+Nzwva2V5

PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYt

dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q2FyciwgQW1lbGlhIEo8L2F1dGhv

cj48YXV0aG9yPkhvcGtpbnMsIFdpbGwgRzwvYXV0aG9yPjxhdXRob3I+R29yZSwgQ2hyaXN0b3Bo

ZXIgSjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5FZmZl

Y3RzIG9mIGFjdXRlIGFsa2Fsb3NpcyBhbmQgYWNpZG9zaXMgb24gcGVyZm9ybWFuY2U8L3RpdGxl

PjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs

ZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxh

YmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJici0yPjxhYmJy

LTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjgwMS04MTQ8L3BhZ2VzPjx2b2x1bWU+

NDE8L3ZvbHVtZT48bnVtYmVyPjEwPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PC9k

YXRlcz48aXNibj4wMTEyLTE2NDI8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48

Q2l0ZT48QXV0aG9yPlZhbmRlbmJvZ2FlcmRlPC9BdXRob3I+PFllYXI+MjAxMTwvWWVhcj48UmVj

TnVtPjk8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjk8L3JlYy1udW1iZXI+PGZvcmVpZ24t

a2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2

ejA5cngiPjk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRp

Y2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlZhbmRlbmJv

Z2FlcmRlLCBUb20gSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywgV2lsbCBHPC9hdXRob3I+PC9h

dXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkVmZmVjdHMgb2YgYWN1dGUgY2Fy

Ym9oeWRyYXRlIHN1cHBsZW1lbnRhdGlvbiBvbiBlbmR1cmFuY2UgcGVyZm9ybWFuY2U8L3RpdGxl

PjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs

ZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxh

YmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJici0yPjxhYmJy

LTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjc3My03OTI8L3BhZ2VzPjx2b2x1bWU+

NDE8L3ZvbHVtZT48bnVtYmVyPjk8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2Rh

dGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+V2VzdG9uPC9BdXRob3I+PFllYXI+MjAxNDwvWWVhcj48UmVjTnVtPjg8L1Jl

Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjg8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5

IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjg8

L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwv

cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldlc3RvbiwgTWF0dGhldzwv

YXV0aG9yPjxhdXRob3I+VGF5bG9yLCBLYXRocnluIEw8L2F1dGhvcj48YXV0aG9yPkJhdHRlcmhh

bSwgQWxhbiBNPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5zLCBXaWxsIEc8L2F1dGhvcj48L2F1dGhv

cnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RWZmZWN0cyBvZiBsb3ctdm9sdW1lIGhp

Z2gtaW50ZW5zaXR5IGludGVydmFsIHRyYWluaW5nIChISVQpIG9uIGZpdG5lc3MgaW4gYWR1bHRz

OiBhIG1ldGEtYW5hbHlzaXMgb2YgY29udHJvbGxlZCBhbmQgbm9uLWNvbnRyb2xsZWQgdHJpYWxz

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvc2Vjb25kYXJ5LXRpdGxl

PjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvZnVsbC10

aXRsZT48YWJici0xPlNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5TcG9ydCBNZWQ8L2FiYnIt

Mj48YWJici0zPlNNPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMDA1LTEwMTc8L3BhZ2Vz

Pjx2b2x1bWU+NDQ8L3ZvbHVtZT48bnVtYmVyPjc8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxNDwv

eWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+

PC9DaXRlPjwvRW5kTm90ZT4A

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DYXJyPC9BdXRob3I+PFllYXI+MjAxMTwvWWVhcj48UmVj

TnVtPjc8L1JlY051bT48RGlzcGxheVRleHQ+KENhcnIgZXQgYWwuLCAyMDExOyBWYW5kZW5ib2dh

ZXJkZSBhbmQgSG9wa2lucywgMjAxMTsgV2VzdG9uIGV0IGFsLiwgMjAxNCk8L0Rpc3BsYXlUZXh0

PjxyZWNvcmQ+PHJlYy1udW1iZXI+NzwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBw

PSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+Nzwva2V5

PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYt

dHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+Q2FyciwgQW1lbGlhIEo8L2F1dGhv

cj48YXV0aG9yPkhvcGtpbnMsIFdpbGwgRzwvYXV0aG9yPjxhdXRob3I+R29yZSwgQ2hyaXN0b3Bo

ZXIgSjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5FZmZl

Y3RzIG9mIGFjdXRlIGFsa2Fsb3NpcyBhbmQgYWNpZG9zaXMgb24gcGVyZm9ybWFuY2U8L3RpdGxl

PjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs

ZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxh

YmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJici0yPjxhYmJy

LTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjgwMS04MTQ8L3BhZ2VzPjx2b2x1bWU+

NDE8L3ZvbHVtZT48bnVtYmVyPjEwPC9udW1iZXI+PGRhdGVzPjx5ZWFyPjIwMTE8L3llYXI+PC9k

YXRlcz48aXNibj4wMTEyLTE2NDI8L2lzYm4+PHVybHM+PC91cmxzPjwvcmVjb3JkPjwvQ2l0ZT48

Q2l0ZT48QXV0aG9yPlZhbmRlbmJvZ2FlcmRlPC9BdXRob3I+PFllYXI+MjAxMTwvWWVhcj48UmVj

TnVtPjk8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjk8L3JlYy1udW1iZXI+PGZvcmVpZ24t

a2V5cz48a2V5IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2

ejA5cngiPjk8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRp

Y2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlZhbmRlbmJv

Z2FlcmRlLCBUb20gSjwvYXV0aG9yPjxhdXRob3I+SG9wa2lucywgV2lsbCBHPC9hdXRob3I+PC9h

dXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkVmZmVjdHMgb2YgYWN1dGUgY2Fy

Ym9oeWRyYXRlIHN1cHBsZW1lbnRhdGlvbiBvbiBlbmR1cmFuY2UgcGVyZm9ybWFuY2U8L3RpdGxl

PjxzZWNvbmRhcnktdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90aXRs

ZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+U3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxh

YmJyLTE+U3BvcnRzIE1lZC48L2FiYnItMT48YWJici0yPlNwb3J0IE1lZDwvYWJici0yPjxhYmJy

LTM+U008L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjc3My03OTI8L3BhZ2VzPjx2b2x1bWU+

NDE8L3ZvbHVtZT48bnVtYmVyPjk8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxMTwveWVhcj48L2Rh

dGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+V2VzdG9uPC9BdXRob3I+PFllYXI+MjAxNDwvWWVhcj48UmVjTnVtPjg8L1Jl

Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjg8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5

IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjg8

L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwv

cmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldlc3RvbiwgTWF0dGhldzwv

YXV0aG9yPjxhdXRob3I+VGF5bG9yLCBLYXRocnluIEw8L2F1dGhvcj48YXV0aG9yPkJhdHRlcmhh

bSwgQWxhbiBNPC9hdXRob3I+PGF1dGhvcj5Ib3BraW5zLCBXaWxsIEc8L2F1dGhvcj48L2F1dGhv

cnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+RWZmZWN0cyBvZiBsb3ctdm9sdW1lIGhp

Z2gtaW50ZW5zaXR5IGludGVydmFsIHRyYWluaW5nIChISVQpIG9uIGZpdG5lc3MgaW4gYWR1bHRz

OiBhIG1ldGEtYW5hbHlzaXMgb2YgY29udHJvbGxlZCBhbmQgbm9uLWNvbnRyb2xsZWQgdHJpYWxz

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvc2Vjb25kYXJ5LXRpdGxl

PjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvZnVsbC10

aXRsZT48YWJici0xPlNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5TcG9ydCBNZWQ8L2FiYnIt

Mj48YWJici0zPlNNPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMDA1LTEwMTc8L3BhZ2Vz

Pjx2b2x1bWU+NDQ8L3ZvbHVtZT48bnVtYmVyPjc8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxNDwv

eWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+

PC9DaXRlPjwvRW5kTm90ZT4A

ADDIN EN.CITE.DATA (Carr et al., 2011; Vandenbogaerde and Hopkins, 2011; Weston et al., 2014), but in future I may use two random effects to account for within-study between-subject clustering (e.g., sex) and within-study within-subject clustering (e.g., multiple doses or time points).Publication Bias and Outlier StudiesA pervasive tendency for only statistically significant effects to end up in print results in the overestimation of published effects, a phenomenon known as publication bias. Such bias was not an issue for the validity meta-analysis of Coquart et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(2016a); five of the other meta-analysts did not mention the possibility of publication bias in their effects PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IEh1bWUg

ZXQgYWwuLCAyMDE1OyBKb3NlZnNzb24gZXQgYWwuLCAyMDE0OyBUb21saW5zb24gZXQgYWwuLCAy

MDE1OyBXcmlnaHQgZXQgYWwuLCAyMDE1KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJl

cj4yMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4

NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjA8L2tleT48L2ZvcmVpZ24ta2V5cz48

cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y

cz48YXV0aG9ycz48YXV0aG9yPkJ1cmRlbiwgUmljaGFyZCBKPC9hdXRob3I+PGF1dGhvcj5Nb3J0

b24sIEthdGllPC9hdXRob3I+PGF1dGhvcj5SaWNoYXJkcywgVG9ieTwvYXV0aG9yPjxhdXRob3I+

V2h5dGUsIEdyZWdvcnkgUDwvYXV0aG9yPjxhdXRob3I+UGVkbGFyLCBDaGFybGVzIFI8L2F1dGhv

cj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXMgaXJvbiB0cmVhdG1l

bnQgYmVuZWZpY2lhbCBpbiwgaXJvbi1kZWZpY2llbnQgYnV0IG5vbi1hbmFlbWljIChJRE5BKSBl

bmR1cmFuY2UgYXRobGV0ZXM/IEEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRs

ZT5Ccml0aXNoIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90

aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9mIFNwb3J0cyBN

ZWRpY2luZTwvZnVsbC10aXRsZT48YWJici0xPkJyLiBKLiBTcG9ydHMgTWVkLjwvYWJici0xPjxh

YmJyLTI+QnIgSiBTcG9ydHMgTWVkPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMzg5LTEz

OTc8L3BhZ2VzPjx2b2x1bWU+NDk8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2Rh

dGVzPjxpc2JuPjE0NzMtMDQ4MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+SHVtZTwvQXV0aG9yPjxZZWFyPjIwMTU8L1llYXI+PFJlY051bT4xMjwvUmVj

TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5

IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjEy

PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8

L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IdW1lLCBQYXRyaWEgQTwv

YXV0aG9yPjxhdXRob3I+TG9yaW1lciwgQW5uYSBWPC9hdXRob3I+PGF1dGhvcj5HcmlmZml0aHMs

IFBldGVyIEM8L2F1dGhvcj48YXV0aG9yPkNhcmxzb24sIElzYWFjPC9hdXRob3I+PGF1dGhvcj5M

YW1vbnQsIE1pa2U8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0

bGU+UmVjcmVhdGlvbmFsIHNub3ctc3BvcnRzIGluanVyeSByaXNrIGZhY3RvcnMgYW5kIGNvdW50

ZXJtZWFzdXJlczogYSBtZXRhLWFuYWx5c2lzIHJldmlldyBhbmQgSGFkZG9uIG1hdHJpeCBldmFs

dWF0aW9uPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvc2Vjb25kYXJ5

LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlNwb3J0cyBNZWRpY2luZTwv

ZnVsbC10aXRsZT48YWJici0xPlNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5TcG9ydCBNZWQ8

L2FiYnItMj48YWJici0zPlNNPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMTc1LTExOTA8

L3BhZ2VzPjx2b2x1bWU+NDU8L3ZvbHVtZT48bnVtYmVyPjg8L251bWJlcj48ZGF0ZXM+PHllYXI+

MjAxNTwveWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9y

ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Sm9zZWZzc29uPC9BdXRob3I+PFllYXI+MjAxNDwv

WWVhcj48UmVjTnVtPjE3PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4xNzwvcmVjLW51bWJl

cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1

d3NmeDlhejl4MHZ6MDlyeCI+MTc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i

Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0

aG9yPkpvc2Vmc3NvbiwgVG9yYmrDtnJuPC9hdXRob3I+PGF1dGhvcj5MaW5kd2FsbCwgTWFnbnVz

PC9hdXRob3I+PGF1dGhvcj5BcmNoZXIsIFRyZXZvcjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRy

aWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5QaHlzaWNhbCBleGVyY2lzZSBpbnRlcnZlbnRpb24gaW4g

ZGVwcmVzc2l2ZSBkaXNvcmRlcnM6IE1ldGHigJBhbmFseXNpcyBhbmQgc3lzdGVtYXRpYyByZXZp

ZXc8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NhbmRpbmF2aWFuIGpvdXJuYWwgb2YgbWVkaWNp

bmUgYW5kIHNjaWVuY2UgaW4gc3BvcnRzPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlv

ZGljYWw+PGZ1bGwtdGl0bGU+U2NhbmRpbmF2aWFuIEpvdXJuYWwgb2YgTWVkaWNpbmUgYW5kIFNj

aWVuY2UgaW4gU3BvcnRzPC9mdWxsLXRpdGxlPjxhYmJyLTE+U2NhbmQuIEouIE1lZC4gU2NpLiBT

cG9ydHM8L2FiYnItMT48YWJici0yPlNjYW5kIEogTWVkIFNjaSBTcG9ydHM8L2FiYnItMj48L3Bl

cmlvZGljYWw+PHBhZ2VzPjI1OS0yNzI8L3BhZ2VzPjx2b2x1bWU+MjQ8L3ZvbHVtZT48bnVtYmVy

PjI8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxNDwveWVhcj48L2RhdGVzPjxpc2JuPjE2MDAtMDgz

ODwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+VG9tbGlu

c29uPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjIxPC9SZWNOdW0+PHJlY29yZD48

cmVjLW51bWJlcj4yMTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGIt

aWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjE8L2tleT48L2ZvcmVp

Z24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNv

bnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlRvbWxpbnNvbiwgUGV0ZXIgQjwvYXV0aG9yPjxh

dXRob3I+Sm9zZXBoLCBDb3JleTwvYXV0aG9yPjxhdXRob3I+QW5naW9pLCBNYW51ZWxhPC9hdXRo

b3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkVmZmVjdHMgb2Ygdml0

YW1pbiBEIHN1cHBsZW1lbnRhdGlvbiBvbiB1cHBlciBhbmQgbG93ZXIgYm9keSBtdXNjbGUgc3Ry

ZW5ndGggbGV2ZWxzIGluIGhlYWx0aHkgaW5kaXZpZHVhbHMuIEEgc3lzdGVtYXRpYyByZXZpZXcg

d2l0aCBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2YgU2Np

ZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVy

aW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNjaWVuY2UgYW5kIE1lZGljaW5lIGluIFNw

b3J0PC9mdWxsLXRpdGxlPjxhYmJyLTE+Si4gU2NpLiBNZWQuIFNwb3J0PC9hYmJyLTE+PGFiYnIt

Mj5KIFNjaSBNZWQgU3BvcnQ8L2FiYnItMj48L3BlcmlvZGljYWw+PHBhZ2VzPjU3NS01ODA8L3Bh

Z2VzPjx2b2x1bWU+MTg8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxp

c2JuPjE0NDAtMjQ0MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxB

dXRob3I+V3JpZ2h0PC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjE5PC9SZWNOdW0+

PHJlY29yZD48cmVjLW51bWJlcj4xOTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBw

PSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MTk8L2tl

eT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVm

LXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldyaWdodCwgQWxleGlzIEE8L2F1

dGhvcj48YXV0aG9yPlRheWxvciwgSmVmZnJleSBCPC9hdXRob3I+PGF1dGhvcj5Gb3JkLCBLZXZp

biBSPC9hdXRob3I+PGF1dGhvcj5TaXNrYSwgTGluZHNleTwvYXV0aG9yPjxhdXRob3I+U21vbGln

YSwgSmFtZXMgTTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs

ZT5SaXNrIGZhY3RvcnMgYXNzb2NpYXRlZCB3aXRoIGxvd2VyIGV4dHJlbWl0eSBzdHJlc3MgZnJh

Y3R1cmVzIGluIHJ1bm5lcnM6IGEgc3lzdGVtYXRpYyByZXZpZXcgd2l0aCBtZXRhLWFuYWx5c2lz

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkJyaXRpc2ggSm91cm5hbCBvZiBTcG9ydHMgTWVkaWNp

bmU8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Ccml0

aXNoIEpvdXJuYWwgb2YgU3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxhYmJyLTE+QnIuIEou

IFNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5CciBKIFNwb3J0cyBNZWQ8L2FiYnItMj48L3Bl

cmlvZGljYWw+PHBhZ2VzPjE1MTctMTUyMzwvcGFnZXM+PHZvbHVtZT40OTwvdm9sdW1lPjxkYXRl

cz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTQ3My0wNDgwPC9pc2JuPjx1cmxzPjwv

dXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IEh1bWUg

ZXQgYWwuLCAyMDE1OyBKb3NlZnNzb24gZXQgYWwuLCAyMDE0OyBUb21saW5zb24gZXQgYWwuLCAy

MDE1OyBXcmlnaHQgZXQgYWwuLCAyMDE1KTwvRGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJl

cj4yMDwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4

NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjA8L2tleT48L2ZvcmVpZ24ta2V5cz48

cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9y

cz48YXV0aG9ycz48YXV0aG9yPkJ1cmRlbiwgUmljaGFyZCBKPC9hdXRob3I+PGF1dGhvcj5Nb3J0

b24sIEthdGllPC9hdXRob3I+PGF1dGhvcj5SaWNoYXJkcywgVG9ieTwvYXV0aG9yPjxhdXRob3I+

V2h5dGUsIEdyZWdvcnkgUDwvYXV0aG9yPjxhdXRob3I+UGVkbGFyLCBDaGFybGVzIFI8L2F1dGhv

cj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXMgaXJvbiB0cmVhdG1l

bnQgYmVuZWZpY2lhbCBpbiwgaXJvbi1kZWZpY2llbnQgYnV0IG5vbi1hbmFlbWljIChJRE5BKSBl

bmR1cmFuY2UgYXRobGV0ZXM/IEEgbWV0YS1hbmFseXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRs

ZT5Ccml0aXNoIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGljaW5lPC9zZWNvbmRhcnktdGl0bGU+PC90

aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QnJpdGlzaCBKb3VybmFsIG9mIFNwb3J0cyBN

ZWRpY2luZTwvZnVsbC10aXRsZT48YWJici0xPkJyLiBKLiBTcG9ydHMgTWVkLjwvYWJici0xPjxh

YmJyLTI+QnIgSiBTcG9ydHMgTWVkPC9hYmJyLTI+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMzg5LTEz

OTc8L3BhZ2VzPjx2b2x1bWU+NDk8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2Rh

dGVzPjxpc2JuPjE0NzMtMDQ4MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxD

aXRlPjxBdXRob3I+SHVtZTwvQXV0aG9yPjxZZWFyPjIwMTU8L1llYXI+PFJlY051bT4xMjwvUmVj

TnVtPjxyZWNvcmQ+PHJlYy1udW1iZXI+MTI8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5

IGFwcD0iRU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjEy

PC9rZXk+PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8

L3JlZi10eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5IdW1lLCBQYXRyaWEgQTwv

YXV0aG9yPjxhdXRob3I+TG9yaW1lciwgQW5uYSBWPC9hdXRob3I+PGF1dGhvcj5HcmlmZml0aHMs

IFBldGVyIEM8L2F1dGhvcj48YXV0aG9yPkNhcmxzb24sIElzYWFjPC9hdXRob3I+PGF1dGhvcj5M

YW1vbnQsIE1pa2U8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0

bGU+UmVjcmVhdGlvbmFsIHNub3ctc3BvcnRzIGluanVyeSByaXNrIGZhY3RvcnMgYW5kIGNvdW50

ZXJtZWFzdXJlczogYSBtZXRhLWFuYWx5c2lzIHJldmlldyBhbmQgSGFkZG9uIG1hdHJpeCBldmFs

dWF0aW9uPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPlNwb3J0cyBNZWRpY2luZTwvc2Vjb25kYXJ5

LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPlNwb3J0cyBNZWRpY2luZTwv

ZnVsbC10aXRsZT48YWJici0xPlNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5TcG9ydCBNZWQ8

L2FiYnItMj48YWJici0zPlNNPC9hYmJyLTM+PC9wZXJpb2RpY2FsPjxwYWdlcz4xMTc1LTExOTA8

L3BhZ2VzPjx2b2x1bWU+NDU8L3ZvbHVtZT48bnVtYmVyPjg8L251bWJlcj48ZGF0ZXM+PHllYXI+

MjAxNTwveWVhcj48L2RhdGVzPjxpc2JuPjAxMTItMTY0MjwvaXNibj48dXJscz48L3VybHM+PC9y

ZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+Sm9zZWZzc29uPC9BdXRob3I+PFllYXI+MjAxNDwv

WWVhcj48UmVjTnVtPjE3PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4xNzwvcmVjLW51bWJl

cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1

d3NmeDlhejl4MHZ6MDlyeCI+MTc8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i

Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0

aG9yPkpvc2Vmc3NvbiwgVG9yYmrDtnJuPC9hdXRob3I+PGF1dGhvcj5MaW5kd2FsbCwgTWFnbnVz

PC9hdXRob3I+PGF1dGhvcj5BcmNoZXIsIFRyZXZvcjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRy

aWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5QaHlzaWNhbCBleGVyY2lzZSBpbnRlcnZlbnRpb24gaW4g

ZGVwcmVzc2l2ZSBkaXNvcmRlcnM6IE1ldGHigJBhbmFseXNpcyBhbmQgc3lzdGVtYXRpYyByZXZp

ZXc8L3RpdGxlPjxzZWNvbmRhcnktdGl0bGU+U2NhbmRpbmF2aWFuIGpvdXJuYWwgb2YgbWVkaWNp

bmUgYW5kIHNjaWVuY2UgaW4gc3BvcnRzPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlv

ZGljYWw+PGZ1bGwtdGl0bGU+U2NhbmRpbmF2aWFuIEpvdXJuYWwgb2YgTWVkaWNpbmUgYW5kIFNj

aWVuY2UgaW4gU3BvcnRzPC9mdWxsLXRpdGxlPjxhYmJyLTE+U2NhbmQuIEouIE1lZC4gU2NpLiBT

cG9ydHM8L2FiYnItMT48YWJici0yPlNjYW5kIEogTWVkIFNjaSBTcG9ydHM8L2FiYnItMj48L3Bl

cmlvZGljYWw+PHBhZ2VzPjI1OS0yNzI8L3BhZ2VzPjx2b2x1bWU+MjQ8L3ZvbHVtZT48bnVtYmVy

PjI8L251bWJlcj48ZGF0ZXM+PHllYXI+MjAxNDwveWVhcj48L2RhdGVzPjxpc2JuPjE2MDAtMDgz

ODwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+VG9tbGlu

c29uPC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjIxPC9SZWNOdW0+PHJlY29yZD48

cmVjLW51bWJlcj4yMTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGIt

aWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MjE8L2tleT48L2ZvcmVp

Z24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNv

bnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlRvbWxpbnNvbiwgUGV0ZXIgQjwvYXV0aG9yPjxh

dXRob3I+Sm9zZXBoLCBDb3JleTwvYXV0aG9yPjxhdXRob3I+QW5naW9pLCBNYW51ZWxhPC9hdXRo

b3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkVmZmVjdHMgb2Ygdml0

YW1pbiBEIHN1cHBsZW1lbnRhdGlvbiBvbiB1cHBlciBhbmQgbG93ZXIgYm9keSBtdXNjbGUgc3Ry

ZW5ndGggbGV2ZWxzIGluIGhlYWx0aHkgaW5kaXZpZHVhbHMuIEEgc3lzdGVtYXRpYyByZXZpZXcg

d2l0aCBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkpvdXJuYWwgb2YgU2Np

ZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVy

aW9kaWNhbD48ZnVsbC10aXRsZT5Kb3VybmFsIG9mIFNjaWVuY2UgYW5kIE1lZGljaW5lIGluIFNw

b3J0PC9mdWxsLXRpdGxlPjxhYmJyLTE+Si4gU2NpLiBNZWQuIFNwb3J0PC9hYmJyLTE+PGFiYnIt

Mj5KIFNjaSBNZWQgU3BvcnQ8L2FiYnItMj48L3BlcmlvZGljYWw+PHBhZ2VzPjU3NS01ODA8L3Bh

Z2VzPjx2b2x1bWU+MTg8L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNTwveWVhcj48L2RhdGVzPjxp

c2JuPjE0NDAtMjQ0MDwvaXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxB

dXRob3I+V3JpZ2h0PC9BdXRob3I+PFllYXI+MjAxNTwvWWVhcj48UmVjTnVtPjE5PC9SZWNOdW0+

PHJlY29yZD48cmVjLW51bWJlcj4xOTwvcmVjLW51bWJlcj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBw

PSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6MDlyeCI+MTk8L2tl

eT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRpY2xlIj4xNzwvcmVm

LXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPldyaWdodCwgQWxleGlzIEE8L2F1

dGhvcj48YXV0aG9yPlRheWxvciwgSmVmZnJleSBCPC9hdXRob3I+PGF1dGhvcj5Gb3JkLCBLZXZp

biBSPC9hdXRob3I+PGF1dGhvcj5TaXNrYSwgTGluZHNleTwvYXV0aG9yPjxhdXRob3I+U21vbGln

YSwgSmFtZXMgTTwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRs

ZT5SaXNrIGZhY3RvcnMgYXNzb2NpYXRlZCB3aXRoIGxvd2VyIGV4dHJlbWl0eSBzdHJlc3MgZnJh

Y3R1cmVzIGluIHJ1bm5lcnM6IGEgc3lzdGVtYXRpYyByZXZpZXcgd2l0aCBtZXRhLWFuYWx5c2lz

PC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPkJyaXRpc2ggSm91cm5hbCBvZiBTcG9ydHMgTWVkaWNp

bmU8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5Ccml0

aXNoIEpvdXJuYWwgb2YgU3BvcnRzIE1lZGljaW5lPC9mdWxsLXRpdGxlPjxhYmJyLTE+QnIuIEou

IFNwb3J0cyBNZWQuPC9hYmJyLTE+PGFiYnItMj5CciBKIFNwb3J0cyBNZWQ8L2FiYnItMj48L3Bl

cmlvZGljYWw+PHBhZ2VzPjE1MTctMTUyMzwvcGFnZXM+PHZvbHVtZT40OTwvdm9sdW1lPjxkYXRl

cz48eWVhcj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTQ3My0wNDgwPC9pc2JuPjx1cmxzPjwv

dXJscz48L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=

ADDIN EN.CITE.DATA (Burden et al., 2015; Hume et al., 2015; Josefsson et al., 2014; Tomlinson et al., 2015; Wright et al., 2015) while four PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DaHU8L0F1dGhvcj48WWVhcj4yMDE2PC9ZZWFyPjxSZWNO

dW0+MjM8L1JlY051bT48RGlzcGxheVRleHQ+KENodSBldCBhbC4sIDIwMTY7IFBldGVyc29uIGV0

IGFsLiwgMjAxMTsgU2FsdmFkb3IgZXQgYWwuLCAyMDE2OyBTb29tcm8gZXQgYWwuLCAyMDE1KTwv

RGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4yMzwvcmVjLW51bWJlcj48Zm9yZWlnbi1r

ZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6

MDlyeCI+MjM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRp

Y2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkNodSwgQW5u

YTwvYXV0aG9yPjxhdXRob3I+UGV0b2N6LCBQZXRlcjwvYXV0aG9yPjxhdXRob3I+U2FtbWFuLCBT

YW1pcjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbW1l

ZGlhdGUgZWZmZWN0cyBvZiBhZXJvYmljIGV4ZXJjaXNlIG9uIHBsYXNtYS9zZXJ1bSB6aW5jIGxl

dmVsczogYSBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk1lZGljaW5lIGFu

ZCBzY2llbmNlIGluIHNwb3J0cyBhbmQgZXhlcmNpc2U8L3NlY29uZGFyeS10aXRsZT48L3RpdGxl

cz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5NZWRpY2luZSBhbmQgU2NpZW5jZSBpbiBTcG9ydHMg

YW5kIEV4ZXJjaXNlPC9mdWxsLXRpdGxlPjxhYmJyLTE+TWVkLiBTY2kuIFNwb3J0cyBFeGVyYy48

L2FiYnItMT48YWJici0yPk1lZCBTY2kgU3BvcnQgRXhlcmNpc2U8L2FiYnItMj48YWJici0zPk1T

U0U8L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjcyNi03MzM8L3BhZ2VzPjx2b2x1bWU+NDg8

L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNjwveWVhcj48L2RhdGVzPjxpc2JuPjAxOTUtOTEzMTwv

aXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+UGV0ZXJzb248

L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+MjI8L1JlY051bT48cmVjb3JkPjxyZWMt

bnVtYmVyPjIyPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i

cjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4yMjwva2V5PjwvZm9yZWlnbi1r

ZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJp

YnV0b3JzPjxhdXRob3JzPjxhdXRob3I+UGV0ZXJzb24sIE1hcmsgRDwvYXV0aG9yPjxhdXRob3I+

U2VuLCBBbmFuZGE8L2F1dGhvcj48YXV0aG9yPkdvcmRvbiwgUGF1bCBNPC9hdXRob3I+PC9hdXRo

b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkluZmx1ZW5jZSBvZiByZXNpc3RhbmNl

IGV4ZXJjaXNlIG9uIGxlYW4gYm9keSBtYXNzIGluIGFnaW5nIGFkdWx0czogYSBtZXRhLWFuYWx5

c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk1lZGljaW5lIGFuZCBzY2llbmNlIGluIHNwb3J0

cyBhbmQgZXhlcmNpc2U8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVs

bC10aXRsZT5NZWRpY2luZSBhbmQgU2NpZW5jZSBpbiBTcG9ydHMgYW5kIEV4ZXJjaXNlPC9mdWxs

LXRpdGxlPjxhYmJyLTE+TWVkLiBTY2kuIFNwb3J0cyBFeGVyYy48L2FiYnItMT48YWJici0yPk1l

ZCBTY2kgU3BvcnQgRXhlcmNpc2U8L2FiYnItMj48YWJici0zPk1TU0U8L2FiYnItMz48L3Blcmlv

ZGljYWw+PHBhZ2VzPjI0OTwvcGFnZXM+PHZvbHVtZT40Mzwvdm9sdW1lPjxudW1iZXI+MjwvbnVt

YmVyPjxkYXRlcz48eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3Jk

PjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFllYXI+MjAxNjwvWWVhcj48

UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwvcmVjLW51bWJlcj48Zm9y

ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlh

ejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h

bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlNh

bHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwvYXV0aG9yPjxhdXRob3I+

TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1dGhvcj48YXV0aG9yPkNy

dXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250

cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0aW9uaW5nIGFuZCBleGVy

Y2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1hbmFseXNpczwv

dGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJuYWwgb2Ygc3BvcnRzIHBo

eXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlv

ZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9mIFNwb3J0cyBQaHlzaW9s

b2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPkludC4gSi4gU3BvcnRzIFBo

eXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9ydHMgUGh5c2lvbCBQZXJm

b3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+NC0x

NDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFyPjwvZGF0

ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENp

dGU+PEF1dGhvcj5Tb29tcm88L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+MTg8L1Jl

Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjE4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl

eSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4x

ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3

PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+U29vbXJvLCBOYWplZWJ1

bGxhaDwvYXV0aG9yPjxhdXRob3I+U2FuZGVycywgUm9zczwvYXV0aG9yPjxhdXRob3I+SGFja2V0

dCwgRGFuaWVsPC9hdXRob3I+PGF1dGhvcj5IdWJrYSwgVGF0ZTwvYXV0aG9yPjxhdXRob3I+RWJy

YWhpbWksIFNhYWhpbDwvYXV0aG9yPjxhdXRob3I+RnJlZXN0b24sIEpvbmF0aGFuPC9hdXRob3I+

PGF1dGhvcj5Db2JsZXksIFN0ZXBoZW48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+

PHRpdGxlcz48dGl0bGU+VGhlIGVmaWNhY3kgb2YgaW5qdXJ5IHByZXZlbnRpb24gcHJvZ3JhbXMg

aW4gYWRvbGVzY2VudCB0ZWFtIHNwb3J0czogYSBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25k

YXJ5LXRpdGxlPkFtZXJpY2FuIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGljaW5lPC9zZWNvbmRhcnkt

dGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QW1lcmljYW4gSm91cm5hbCBv

ZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5BbS4gSi4gU3BvcnRzIE1lZC48

L2FiYnItMT48YWJici0yPkFtIEogU3BvcnRzIE1lZDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFn

ZXM+aHR0cHM6Ly9kb2kub3JnLzEwLjExNzcvMDM2MzU0NjUxNTYxODM3MjwvcGFnZXM+PGRhdGVz

Pjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4wMzYzLTU0NjU8L2lzYm4+PHVybHM+PC91

cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5DaHU8L0F1dGhvcj48WWVhcj4yMDE2PC9ZZWFyPjxSZWNO

dW0+MjM8L1JlY051bT48RGlzcGxheVRleHQ+KENodSBldCBhbC4sIDIwMTY7IFBldGVyc29uIGV0

IGFsLiwgMjAxMTsgU2FsdmFkb3IgZXQgYWwuLCAyMDE2OyBTb29tcm8gZXQgYWwuLCAyMDE1KTwv

RGlzcGxheVRleHQ+PHJlY29yZD48cmVjLW51bWJlcj4yMzwvcmVjLW51bWJlcj48Zm9yZWlnbi1r

ZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlhejl4MHZ6

MDlyeCI+MjM8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5hbCBBcnRp

Y2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPkNodSwgQW5u

YTwvYXV0aG9yPjxhdXRob3I+UGV0b2N6LCBQZXRlcjwvYXV0aG9yPjxhdXRob3I+U2FtbWFuLCBT

YW1pcjwvYXV0aG9yPjwvYXV0aG9ycz48L2NvbnRyaWJ1dG9ycz48dGl0bGVzPjx0aXRsZT5JbW1l

ZGlhdGUgZWZmZWN0cyBvZiBhZXJvYmljIGV4ZXJjaXNlIG9uIHBsYXNtYS9zZXJ1bSB6aW5jIGxl

dmVsczogYSBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk1lZGljaW5lIGFu

ZCBzY2llbmNlIGluIHNwb3J0cyBhbmQgZXhlcmNpc2U8L3NlY29uZGFyeS10aXRsZT48L3RpdGxl

cz48cGVyaW9kaWNhbD48ZnVsbC10aXRsZT5NZWRpY2luZSBhbmQgU2NpZW5jZSBpbiBTcG9ydHMg

YW5kIEV4ZXJjaXNlPC9mdWxsLXRpdGxlPjxhYmJyLTE+TWVkLiBTY2kuIFNwb3J0cyBFeGVyYy48

L2FiYnItMT48YWJici0yPk1lZCBTY2kgU3BvcnQgRXhlcmNpc2U8L2FiYnItMj48YWJici0zPk1T

U0U8L2FiYnItMz48L3BlcmlvZGljYWw+PHBhZ2VzPjcyNi03MzM8L3BhZ2VzPjx2b2x1bWU+NDg8

L3ZvbHVtZT48ZGF0ZXM+PHllYXI+MjAxNjwveWVhcj48L2RhdGVzPjxpc2JuPjAxOTUtOTEzMTwv

aXNibj48dXJscz48L3VybHM+PC9yZWNvcmQ+PC9DaXRlPjxDaXRlPjxBdXRob3I+UGV0ZXJzb248

L0F1dGhvcj48WWVhcj4yMDExPC9ZZWFyPjxSZWNOdW0+MjI8L1JlY051bT48cmVjb3JkPjxyZWMt

bnVtYmVyPjIyPC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0i

cjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4yMjwva2V5PjwvZm9yZWlnbi1r

ZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJp

YnV0b3JzPjxhdXRob3JzPjxhdXRob3I+UGV0ZXJzb24sIE1hcmsgRDwvYXV0aG9yPjxhdXRob3I+

U2VuLCBBbmFuZGE8L2F1dGhvcj48YXV0aG9yPkdvcmRvbiwgUGF1bCBNPC9hdXRob3I+PC9hdXRo

b3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxlPkluZmx1ZW5jZSBvZiByZXNpc3RhbmNl

IGV4ZXJjaXNlIG9uIGxlYW4gYm9keSBtYXNzIGluIGFnaW5nIGFkdWx0czogYSBtZXRhLWFuYWx5

c2lzPC90aXRsZT48c2Vjb25kYXJ5LXRpdGxlPk1lZGljaW5lIGFuZCBzY2llbmNlIGluIHNwb3J0

cyBhbmQgZXhlcmNpc2U8L3NlY29uZGFyeS10aXRsZT48L3RpdGxlcz48cGVyaW9kaWNhbD48ZnVs

bC10aXRsZT5NZWRpY2luZSBhbmQgU2NpZW5jZSBpbiBTcG9ydHMgYW5kIEV4ZXJjaXNlPC9mdWxs

LXRpdGxlPjxhYmJyLTE+TWVkLiBTY2kuIFNwb3J0cyBFeGVyYy48L2FiYnItMT48YWJici0yPk1l

ZCBTY2kgU3BvcnQgRXhlcmNpc2U8L2FiYnItMj48YWJici0zPk1TU0U8L2FiYnItMz48L3Blcmlv

ZGljYWw+PHBhZ2VzPjI0OTwvcGFnZXM+PHZvbHVtZT40Mzwvdm9sdW1lPjxudW1iZXI+MjwvbnVt

YmVyPjxkYXRlcz48eWVhcj4yMDExPC95ZWFyPjwvZGF0ZXM+PHVybHM+PC91cmxzPjwvcmVjb3Jk

PjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFllYXI+MjAxNjwvWWVhcj48

UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwvcmVjLW51bWJlcj48Zm9y

ZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1d3NmeDlh

ejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0iSm91cm5h

bCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0aG9yPlNh

bHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwvYXV0aG9yPjxhdXRob3I+

TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1dGhvcj48YXV0aG9yPkNy

dXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250

cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0aW9uaW5nIGFuZCBleGVy

Y2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1hbmFseXNpczwv

dGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJuYWwgb2Ygc3BvcnRzIHBo

eXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+PHBlcmlv

ZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9mIFNwb3J0cyBQaHlzaW9s

b2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPkludC4gSi4gU3BvcnRzIFBo

eXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9ydHMgUGh5c2lvbCBQZXJm

b3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFnZXM+NC0x

NDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFyPjwvZGF0

ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0NpdGU+PENp

dGU+PEF1dGhvcj5Tb29tcm88L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNOdW0+MTg8L1Jl

Y051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjE4PC9yZWMtbnVtYmVyPjxmb3JlaWduLWtleXM+PGtl

eSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgwdnowOXJ4Ij4x

ODwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFydGljbGUiPjE3

PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+U29vbXJvLCBOYWplZWJ1

bGxhaDwvYXV0aG9yPjxhdXRob3I+U2FuZGVycywgUm9zczwvYXV0aG9yPjxhdXRob3I+SGFja2V0

dCwgRGFuaWVsPC9hdXRob3I+PGF1dGhvcj5IdWJrYSwgVGF0ZTwvYXV0aG9yPjxhdXRob3I+RWJy

YWhpbWksIFNhYWhpbDwvYXV0aG9yPjxhdXRob3I+RnJlZXN0b24sIEpvbmF0aGFuPC9hdXRob3I+

PGF1dGhvcj5Db2JsZXksIFN0ZXBoZW48L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+

PHRpdGxlcz48dGl0bGU+VGhlIGVmaWNhY3kgb2YgaW5qdXJ5IHByZXZlbnRpb24gcHJvZ3JhbXMg

aW4gYWRvbGVzY2VudCB0ZWFtIHNwb3J0czogYSBtZXRhLWFuYWx5c2lzPC90aXRsZT48c2Vjb25k

YXJ5LXRpdGxlPkFtZXJpY2FuIGpvdXJuYWwgb2Ygc3BvcnRzIG1lZGljaW5lPC9zZWNvbmRhcnkt

dGl0bGU+PC90aXRsZXM+PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+QW1lcmljYW4gSm91cm5hbCBv

ZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5BbS4gSi4gU3BvcnRzIE1lZC48

L2FiYnItMT48YWJici0yPkFtIEogU3BvcnRzIE1lZDwvYWJici0yPjwvcGVyaW9kaWNhbD48cGFn

ZXM+aHR0cHM6Ly9kb2kub3JnLzEwLjExNzcvMDM2MzU0NjUxNTYxODM3MjwvcGFnZXM+PGRhdGVz

Pjx5ZWFyPjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4wMzYzLTU0NjU8L2lzYm4+PHVybHM+PC91

cmxzPjwvcmVjb3JkPjwvQ2l0ZT48L0VuZE5vdGU+

ADDIN EN.CITE.DATA (Chu et al., 2016; Peterson et al., 2011; Salvador et al., 2016; Soomro et al., 2015) investigated asymmetry in the funnel-shaped plot of observed effects vs their standard errors, which is a sign of publication bias. There are two problems with this approach and corrections based on it: heterogeneity disrupts the funnel shape, thereby increasing the likelihood of false-negative and false-positive decisions about publication bias, and it does not take into account any heterogeneity explained in a meta-regression. A plot of the values of the study random-effect solution (effectively the study residuals) vs the study standard error solves these problems: publication bias manifests as a tendency for the residuals to be distributed non-uniformly for studies with higher values of the standard error, and repeating the analysis after deleting all such studies reduces or removes the bias ADDIN EN.CITE <EndNote><Cite><Author>Vandenbogaerde</Author><Year>2011</Year><RecNum>9</RecNum><Prefix>see especially </Prefix><DisplayText>(see especially Carr et al., 2011; Vandenbogaerde and Hopkins, 2011)</DisplayText><record><rec-number>9</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">9</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Vandenbogaerde, Tom J</author><author>Hopkins, Will G</author></authors></contributors><titles><title>Effects of acute carbohydrate supplementation on endurance performance</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>773-792</pages><volume>41</volume><number>9</number><dates><year>2011</year></dates><isbn>0112-1642</isbn><urls></urls></record></Cite><Cite><Author>Carr</Author><Year>2011</Year><RecNum>7</RecNum><record><rec-number>7</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">7</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Carr, Amelia J</author><author>Hopkins, Will G</author><author>Gore, Christopher J</author></authors></contributors><titles><title>Effects of acute alkalosis and acidosis on performance</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>801-814</pages><volume>41</volume><number>10</number><dates><year>2011</year></dates><isbn>0112-1642</isbn><urls></urls></record></Cite></EndNote>(see especially Carr et al., 2011; Vandenbogaerde and Hopkins, 2011). Standardization of the random-effect values converts them to z scores, which also allow for objective identification and elimination of outlier studies. Chu et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Chu</Author><Year>2016</Year><RecNum>23</RecNum><DisplayText>(2016)</DisplayText><record><rec-number>23</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">23</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Chu, Anna</author><author>Petocz, Peter</author><author>Samman, Samir</author></authors></contributors><titles><title>Immediate effects of aerobic exercise on plasma/serum zinc levels: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>726-733</pages><volume>48</volume><dates><year>2016</year></dates><isbn>0195-9131</isbn><urls></urls></record></Cite></EndNote>(2016) and Peterson et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Peterson</Author><Year>2011</Year><RecNum>22</RecNum><DisplayText>(2011)</DisplayText><record><rec-number>22</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">22</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Peterson, Mark D</author><author>Sen, Ananda</author><author>Gordon, Paul M</author></authors></contributors><titles><title>Influence of resistance exercise on lean body mass in aging adults: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>249</pages><volume>43</volume><number>2</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(2011) investigated the change in the magnitude of the meta-analyzed effect following deletion of one or more studies, presumably checking for outliers; this kind of sensitivity analysis is pointless, because the change in magnitude will be smaller with a larger total number of meta-analyzed studies, and there is no associated rationale for eliminating studies. The researchers may have done this kind of analysis simply because it was available in the analysis package Comprehensive Meta-Analysis.Interpreting MagnitudesA shortcoming with several of the meta-analyses is inadequate attention to the clinical or practical importance of the meta-analyzed effects, let alone that of their moderators and heterogeneity. Coquart et al. ADDIN EN.CITE <EndNote><Cite ExcludeAuth="1"><Author>Coquart</Author><Year>2016</Year><RecNum>3</RecNum><DisplayText>(2016a)</DisplayText><record><rec-number>3</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">3</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Coquart, J B</author><author>Tabben, M</author><author>Farooq, A</author><author>Tourny, C</author><author>Eston, R</author></authors></contributors><titles><title>Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>885-897</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(2016a) made no assessment of the implications of the magnitude of the meta-analyzed validity statistics for assessment of individual patients. Some authors apparently assumed that statistical significance automatically confers importance on the effect, without considering the magnitude of the observed effect or its confidence limits ADDIN EN.CITE <EndNote><Cite><Author>Chu</Author><Year>2016</Year><RecNum>23</RecNum><DisplayText>(Chu et al., 2016; Wright et al., 2015)</DisplayText><record><rec-number>23</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">23</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Chu, Anna</author><author>Petocz, Peter</author><author>Samman, Samir</author></authors></contributors><titles><title>Immediate effects of aerobic exercise on plasma/serum zinc levels: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>726-733</pages><volume>48</volume><dates><year>2016</year></dates><isbn>0195-9131</isbn><urls></urls></record></Cite><Cite><Author>Wright</Author><Year>2015</Year><RecNum>19</RecNum><record><rec-number>19</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">19</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Wright, Alexis A</author><author>Taylor, Jeffrey B</author><author>Ford, Kevin R</author><author>Siska, Lindsey</author><author>Smoliga, James M</author></authors></contributors><titles><title>Risk factors associated with lower extremity stress fractures in runners: a systematic review with meta-analysis</title><secondary-title>British Journal of Sports Medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1517-1523</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite></EndNote>(Chu et al., 2016; Wright et al., 2015). Others used various scales to interpret standardized differences in means, without converting them back into real units to consider whether the standardized magnitude could represent an important clinical or practical effect in all or any populations PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IFNhbHZh

ZG9yIGV0IGFsLiwgMjAxNjsgVG9tbGluc29uIGV0IGFsLiwgMjAxNSk8L0Rpc3BsYXlUZXh0Pjxy

ZWNvcmQ+PHJlYy1udW1iZXI+MjA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i

RU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIwPC9rZXk+

PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10

eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CdXJkZW4sIFJpY2hhcmQgSjwvYXV0

aG9yPjxhdXRob3I+TW9ydG9uLCBLYXRpZTwvYXV0aG9yPjxhdXRob3I+UmljaGFyZHMsIFRvYnk8

L2F1dGhvcj48YXV0aG9yPldoeXRlLCBHcmVnb3J5IFA8L2F1dGhvcj48YXV0aG9yPlBlZGxhciwg

Q2hhcmxlcyBSPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl

PklzIGlyb24gdHJlYXRtZW50IGJlbmVmaWNpYWwgaW4sIGlyb24tZGVmaWNpZW50IGJ1dCBub24t

YW5hZW1pYyAoSUROQSkgZW5kdXJhbmNlIGF0aGxldGVzPyBBIG1ldGEtYW5hbHlzaXM8L3RpdGxl

PjxzZWNvbmRhcnktdGl0bGU+QnJpdGlzaCBqb3VybmFsIG9mIHNwb3J0cyBtZWRpY2luZTwvc2Vj

b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkJyaXRpc2ggSm91

cm5hbCBvZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5Cci4gSi4gU3BvcnRz

IE1lZC48L2FiYnItMT48YWJici0yPkJyIEogU3BvcnRzIE1lZDwvYWJici0yPjwvcGVyaW9kaWNh

bD48cGFnZXM+MTM4OS0xMzk3PC9wYWdlcz48dm9sdW1lPjQ5PC92b2x1bWU+PGRhdGVzPjx5ZWFy

PjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4xNDczLTA0ODA8L2lzYm4+PHVybHM+PC91cmxzPjwv

cmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFllYXI+MjAxNjwv

WWVhcj48UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwvcmVjLW51bWJl

cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1

d3NmeDlhejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i

Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0

aG9yPlNhbHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwvYXV0aG9yPjxh

dXRob3I+TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1dGhvcj48YXV0

aG9yPkNydXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48L2F1dGhvcnM+

PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0aW9uaW5nIGFu

ZCBleGVyY2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1hbmFs

eXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJuYWwgb2Ygc3Bv

cnRzIHBoeXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+

PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9mIFNwb3J0cyBQ

aHlzaW9sb2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPkludC4gSi4gU3Bv

cnRzIFBoeXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9ydHMgUGh5c2lv

bCBQZXJmb3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFn

ZXM+NC0xNDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFy

PjwvZGF0ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0Np

dGU+PENpdGU+PEF1dGhvcj5Ub21saW5zb248L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNO

dW0+MjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjIxPC9yZWMtbnVtYmVyPjxmb3JlaWdu

LWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgw

dnowOXJ4Ij4yMTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFy

dGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+VG9tbGlu

c29uLCBQZXRlciBCPC9hdXRob3I+PGF1dGhvcj5Kb3NlcGgsIENvcmV5PC9hdXRob3I+PGF1dGhv

cj5Bbmdpb2ksIE1hbnVlbGE8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxl

cz48dGl0bGU+RWZmZWN0cyBvZiB2aXRhbWluIEQgc3VwcGxlbWVudGF0aW9uIG9uIHVwcGVyIGFu

ZCBsb3dlciBib2R5IG11c2NsZSBzdHJlbmd0aCBsZXZlbHMgaW4gaGVhbHRoeSBpbmRpdmlkdWFs

cy4gQSBzeXN0ZW1hdGljIHJldmlldyB3aXRoIG1ldGEtYW5hbHlzaXM8L3RpdGxlPjxzZWNvbmRh

cnktdGl0bGU+Sm91cm5hbCBvZiBTY2llbmNlIGFuZCBNZWRpY2luZSBpbiBTcG9ydDwvc2Vjb25k

YXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkpvdXJuYWwgb2YgU2Np

ZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L2Z1bGwtdGl0bGU+PGFiYnItMT5KLiBTY2kuIE1l

ZC4gU3BvcnQ8L2FiYnItMT48YWJici0yPkogU2NpIE1lZCBTcG9ydDwvYWJici0yPjwvcGVyaW9k

aWNhbD48cGFnZXM+NTc1LTU4MDwvcGFnZXM+PHZvbHVtZT4xODwvdm9sdW1lPjxkYXRlcz48eWVh

cj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTQ0MC0yNDQwPC9pc2JuPjx1cmxzPjwvdXJscz48

L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=

ADDIN EN.CITE PEVuZE5vdGU+PENpdGU+PEF1dGhvcj5CdXJkZW48L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxS

ZWNOdW0+MjA8L1JlY051bT48RGlzcGxheVRleHQ+KEJ1cmRlbiBldCBhbC4sIDIwMTU7IFNhbHZh

ZG9yIGV0IGFsLiwgMjAxNjsgVG9tbGluc29uIGV0IGFsLiwgMjAxNSk8L0Rpc3BsYXlUZXh0Pjxy

ZWNvcmQ+PHJlYy1udW1iZXI+MjA8L3JlYy1udW1iZXI+PGZvcmVpZ24ta2V5cz48a2V5IGFwcD0i

RU4iIGRiLWlkPSJyOXB2eDVmMHB6Mnh6Z2VwZTBkNXdzZng5YXo5eDB2ejA5cngiPjIwPC9rZXk+

PC9mb3JlaWduLWtleXM+PHJlZi10eXBlIG5hbWU9IkpvdXJuYWwgQXJ0aWNsZSI+MTc8L3JlZi10

eXBlPjxjb250cmlidXRvcnM+PGF1dGhvcnM+PGF1dGhvcj5CdXJkZW4sIFJpY2hhcmQgSjwvYXV0

aG9yPjxhdXRob3I+TW9ydG9uLCBLYXRpZTwvYXV0aG9yPjxhdXRob3I+UmljaGFyZHMsIFRvYnk8

L2F1dGhvcj48YXV0aG9yPldoeXRlLCBHcmVnb3J5IFA8L2F1dGhvcj48YXV0aG9yPlBlZGxhciwg

Q2hhcmxlcyBSPC9hdXRob3I+PC9hdXRob3JzPjwvY29udHJpYnV0b3JzPjx0aXRsZXM+PHRpdGxl

PklzIGlyb24gdHJlYXRtZW50IGJlbmVmaWNpYWwgaW4sIGlyb24tZGVmaWNpZW50IGJ1dCBub24t

YW5hZW1pYyAoSUROQSkgZW5kdXJhbmNlIGF0aGxldGVzPyBBIG1ldGEtYW5hbHlzaXM8L3RpdGxl

PjxzZWNvbmRhcnktdGl0bGU+QnJpdGlzaCBqb3VybmFsIG9mIHNwb3J0cyBtZWRpY2luZTwvc2Vj

b25kYXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkJyaXRpc2ggSm91

cm5hbCBvZiBTcG9ydHMgTWVkaWNpbmU8L2Z1bGwtdGl0bGU+PGFiYnItMT5Cci4gSi4gU3BvcnRz

IE1lZC48L2FiYnItMT48YWJici0yPkJyIEogU3BvcnRzIE1lZDwvYWJici0yPjwvcGVyaW9kaWNh

bD48cGFnZXM+MTM4OS0xMzk3PC9wYWdlcz48dm9sdW1lPjQ5PC92b2x1bWU+PGRhdGVzPjx5ZWFy

PjIwMTU8L3llYXI+PC9kYXRlcz48aXNibj4xNDczLTA0ODA8L2lzYm4+PHVybHM+PC91cmxzPjwv

cmVjb3JkPjwvQ2l0ZT48Q2l0ZT48QXV0aG9yPlNhbHZhZG9yPC9BdXRob3I+PFllYXI+MjAxNjwv

WWVhcj48UmVjTnVtPjI0PC9SZWNOdW0+PHJlY29yZD48cmVjLW51bWJlcj4yNDwvcmVjLW51bWJl

cj48Zm9yZWlnbi1rZXlzPjxrZXkgYXBwPSJFTiIgZGItaWQ9InI5cHZ4NWYwcHoyeHpnZXBlMGQ1

d3NmeDlhejl4MHZ6MDlyeCI+MjQ8L2tleT48L2ZvcmVpZ24ta2V5cz48cmVmLXR5cGUgbmFtZT0i

Sm91cm5hbCBBcnRpY2xlIj4xNzwvcmVmLXR5cGU+PGNvbnRyaWJ1dG9ycz48YXV0aG9ycz48YXV0

aG9yPlNhbHZhZG9yLCBBRjwvYXV0aG9yPjxhdXRob3I+RGUgQWd1aWFyLCBSQTwvYXV0aG9yPjxh

dXRob3I+TGlzYm9hLCBGRDwvYXV0aG9yPjxhdXRob3I+UGVyZWlyYSwgS0w8L2F1dGhvcj48YXV0

aG9yPkNydXosIFJTPC9hdXRob3I+PGF1dGhvcj5DYXB1dG8sIEY8L2F1dGhvcj48L2F1dGhvcnM+

PC9jb250cmlidXRvcnM+PHRpdGxlcz48dGl0bGU+SXNjaGVtaWMgcHJlY29uZGl0aW9uaW5nIGFu

ZCBleGVyY2lzZSBwZXJmb3JtYW5jZTogYSBzeXN0ZW1hdGljIHJldmlldyBhbmQgbWV0YS1hbmFs

eXNpczwvdGl0bGU+PHNlY29uZGFyeS10aXRsZT5JbnRlcm5hdGlvbmFsIGpvdXJuYWwgb2Ygc3Bv

cnRzIHBoeXNpb2xvZ3kgYW5kIHBlcmZvcm1hbmNlPC9zZWNvbmRhcnktdGl0bGU+PC90aXRsZXM+

PHBlcmlvZGljYWw+PGZ1bGwtdGl0bGU+SW50ZXJuYXRpb25hbCBKb3VybmFsIG9mIFNwb3J0cyBQ

aHlzaW9sb2d5IGFuZCBQZXJmb3JtYW5jZTwvZnVsbC10aXRsZT48YWJici0xPkludC4gSi4gU3Bv

cnRzIFBoeXNpb2wuIFBlcmZvcm0uPC9hYmJyLTE+PGFiYnItMj5JbnQgSiBTcG9ydHMgUGh5c2lv

bCBQZXJmb3JtPC9hYmJyLTI+PGFiYnItMz5JSlNQUDwvYWJici0zPjwvcGVyaW9kaWNhbD48cGFn

ZXM+NC0xNDwvcGFnZXM+PHZvbHVtZT4xMTwvdm9sdW1lPjxkYXRlcz48eWVhcj4yMDE2PC95ZWFy

PjwvZGF0ZXM+PGlzYm4+MTU1NS0wMjY1PC9pc2JuPjx1cmxzPjwvdXJscz48L3JlY29yZD48L0Np

dGU+PENpdGU+PEF1dGhvcj5Ub21saW5zb248L0F1dGhvcj48WWVhcj4yMDE1PC9ZZWFyPjxSZWNO

dW0+MjE8L1JlY051bT48cmVjb3JkPjxyZWMtbnVtYmVyPjIxPC9yZWMtbnVtYmVyPjxmb3JlaWdu

LWtleXM+PGtleSBhcHA9IkVOIiBkYi1pZD0icjlwdng1ZjBwejJ4emdlcGUwZDV3c2Z4OWF6OXgw

dnowOXJ4Ij4yMTwva2V5PjwvZm9yZWlnbi1rZXlzPjxyZWYtdHlwZSBuYW1lPSJKb3VybmFsIEFy

dGljbGUiPjE3PC9yZWYtdHlwZT48Y29udHJpYnV0b3JzPjxhdXRob3JzPjxhdXRob3I+VG9tbGlu

c29uLCBQZXRlciBCPC9hdXRob3I+PGF1dGhvcj5Kb3NlcGgsIENvcmV5PC9hdXRob3I+PGF1dGhv

cj5Bbmdpb2ksIE1hbnVlbGE8L2F1dGhvcj48L2F1dGhvcnM+PC9jb250cmlidXRvcnM+PHRpdGxl

cz48dGl0bGU+RWZmZWN0cyBvZiB2aXRhbWluIEQgc3VwcGxlbWVudGF0aW9uIG9uIHVwcGVyIGFu

ZCBsb3dlciBib2R5IG11c2NsZSBzdHJlbmd0aCBsZXZlbHMgaW4gaGVhbHRoeSBpbmRpdmlkdWFs

cy4gQSBzeXN0ZW1hdGljIHJldmlldyB3aXRoIG1ldGEtYW5hbHlzaXM8L3RpdGxlPjxzZWNvbmRh

cnktdGl0bGU+Sm91cm5hbCBvZiBTY2llbmNlIGFuZCBNZWRpY2luZSBpbiBTcG9ydDwvc2Vjb25k

YXJ5LXRpdGxlPjwvdGl0bGVzPjxwZXJpb2RpY2FsPjxmdWxsLXRpdGxlPkpvdXJuYWwgb2YgU2Np

ZW5jZSBhbmQgTWVkaWNpbmUgaW4gU3BvcnQ8L2Z1bGwtdGl0bGU+PGFiYnItMT5KLiBTY2kuIE1l

ZC4gU3BvcnQ8L2FiYnItMT48YWJici0yPkogU2NpIE1lZCBTcG9ydDwvYWJici0yPjwvcGVyaW9k

aWNhbD48cGFnZXM+NTc1LTU4MDwvcGFnZXM+PHZvbHVtZT4xODwvdm9sdW1lPjxkYXRlcz48eWVh

cj4yMDE1PC95ZWFyPjwvZGF0ZXM+PGlzYm4+MTQ0MC0yNDQwPC9pc2JuPjx1cmxzPjwvdXJscz48

L3JlY29yZD48L0NpdGU+PC9FbmROb3RlPgB=

ADDIN EN.CITE.DATA (Burden et al., 2015; Salvador et al., 2016; Tomlinson et al., 2015). I support standardization for assessing differences or changes in means when there is no real-world scale, but the standardization should be done after the meta-analysis, using appropriately averaged between-subject SD from studies representing a population of interest. The average should be derived via variances weighted by degrees of freedom.As already noted, the SD representing heterogeneity should be doubled (or squared for factor SD) before assessing its magnitude with the same scale as that for assessing the mean effect. Effects of moderators expressed as correlation coefficients ADDIN EN.CITE <EndNote><Cite><Author>Salvador</Author><Year>2016</Year><RecNum>24</RecNum><DisplayText>(Salvador et al., 2016)</DisplayText><record><rec-number>24</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">24</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Salvador, AF</author><author>De Aguiar, RA</author><author>Lisboa, FD</author><author>Pereira, KL</author><author>Cruz, RS</author><author>Caputo, F</author></authors></contributors><titles><title>Ischemic preconditioning and exercise performance: a systematic review and meta-analysis</title><secondary-title>International journal of sports physiology and performance</secondary-title></titles><periodical><full-title>International Journal of Sports Physiology and Performance</full-title><abbr-1>Int. J. Sports Physiol. Perform.</abbr-1><abbr-2>Int J Sports Physiol Perform</abbr-2><abbr-3>IJSPP</abbr-3></periodical><pages>4-14</pages><volume>11</volume><dates><year>2016</year></dates><isbn>1555-0265</isbn><urls></urls></record></Cite></EndNote>(Salvador et al., 2016), "beta" coefficients ADDIN EN.CITE <EndNote><Cite><Author>Burden</Author><Year>2015</Year><RecNum>20</RecNum><DisplayText>(Burden et al., 2015; Peterson et al., 2011)</DisplayText><record><rec-number>20</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">20</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Burden, Richard J</author><author>Morton, Katie</author><author>Richards, Toby</author><author>Whyte, Gregory P</author><author>Pedlar, Charles R</author></authors></contributors><titles><title>Is iron treatment beneficial in, iron-deficient but non-anaemic (IDNA) endurance athletes? A meta-analysis</title><secondary-title>British journal of sports medicine</secondary-title></titles><periodical><full-title>British Journal of Sports Medicine</full-title><abbr-1>Br. J. Sports Med.</abbr-1><abbr-2>Br J Sports Med</abbr-2></periodical><pages>1389-1397</pages><volume>49</volume><dates><year>2015</year></dates><isbn>1473-0480</isbn><urls></urls></record></Cite><Cite><Author>Peterson</Author><Year>2011</Year><RecNum>22</RecNum><record><rec-number>22</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">22</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Peterson, Mark D</author><author>Sen, Ananda</author><author>Gordon, Paul M</author></authors></contributors><titles><title>Influence of resistance exercise on lean body mass in aging adults: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>249</pages><volume>43</volume><number>2</number><dates><year>2011</year></dates><urls></urls></record></Cite></EndNote>(Burden et al., 2015; Peterson et al., 2011) and p values ADDIN EN.CITE <EndNote><Cite><Author>Chu</Author><Year>2016</Year><RecNum>23</RecNum><DisplayText>(Chu et al., 2016)</DisplayText><record><rec-number>23</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">23</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Chu, Anna</author><author>Petocz, Peter</author><author>Samman, Samir</author></authors></contributors><titles><title>Immediate effects of aerobic exercise on plasma/serum zinc levels: a meta-analysis</title><secondary-title>Medicine and science in sports and exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>726-733</pages><volume>48</volume><dates><year>2016</year></dates><isbn>0195-9131</isbn><urls></urls></record></Cite></EndNote>(Chu et al., 2016) do not communicate magnitude to the reader. Moderators representing numeric subject characteristics (e.g., mean age) that have been included as simple linear predictors should be evaluated for a difference in the characteristic equal to two between-subject SDs appropriately averaged from selected studies (again, via weighted variances). A suspected non-linear moderator can be coded as quantiles or other sub-group levels and evaluated accordingly. Almost all the meta-analysts showed some reliance on p values to make conclusions, a practice that in my opinion is particularly inappropriate for meta-analyses. Inferences about the magnitude of all statistics should be based one way or another on the uncertainty represented by the magnitude of lower and upper confidence limits ADDIN EN.CITE <EndNote><Cite><Author>Hopkins</Author><Year>2009</Year><RecNum>13</RecNum><DisplayText>(Hopkins et al., 2009; Hopkins and Batterham, 2016)</DisplayText><record><rec-number>13</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">13</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hopkins, W G</author><author>Marshall, S W</author><author>Batterham, A M</author><author>Hanin, J</author></authors></contributors><titles><title>Progressive statistics for studies in sports medicine and exercise science</title><secondary-title>Medicine and Science in Sports and Exercise</secondary-title></titles><periodical><full-title>Medicine and Science in Sports and Exercise</full-title><abbr-1>Med. Sci. Sports Exerc.</abbr-1><abbr-2>Med Sci Sport Exercise</abbr-2><abbr-3>MSSE</abbr-3></periodical><pages>3-12</pages><volume>41</volume><dates><year>2009</year></dates><urls></urls></record></Cite><Cite><Author>Hopkins</Author><Year>2016</Year><RecNum>26</RecNum><record><rec-number>26</rec-number><foreign-keys><key app="EN" db-id="r9pvx5f0pz2xzgepe0d5wsfx9az9x0vz09rx">26</key></foreign-keys><ref-type name="Journal Article">17</ref-type><contributors><authors><author>Hopkins, W.G.</author><author>Batterham, A.M.</author></authors></contributors><titles><title>Error rates, decisive outcomes and publication bias with several inferential methods</title><secondary-title>Sports Medicine</secondary-title></titles><periodical><full-title>Sports Medicine</full-title><abbr-1>Sports Med.</abbr-1><abbr-2>Sport Med</abbr-2><abbr-3>SM</abbr-3></periodical><pages>1563-1573</pages><volume>46</volume><dates><year>2016</year></dates><urls></urls></record></Cite></EndNote>(Hopkins et al., 2009; Hopkins and Batterham, 2016). ConclusionI rate failure to use a random-effect meta-analysis and failure to properly account for heterogeneity in a random-effect meta-analysis as the most serious flaws, because heterogeneity combined with the mean effect determines a probabilistic range in the clinical or practical importance of the effect in a specific setting. As a researcher or practitioner, you should be cautious about implementing the findings of a meta-analysis lacking a full account of heterogeneity; use it primarily as a convenient reference list to find studies from settings similar to your own, and use these studies to draw your own conclusions about the magnitude of the effect in your setting. You should also be skeptical about any meta-analyzed differences or changes in means based on standardization: there is a good chance the authors will have made major errors, and even when done correctly, standardization results in artefactual heterogeneity. Authors need to provide more documentation about these and the other error-prone aspects of meta-analysis I have identified here, if readers are to have more trust in the findings.When I sent the first submitted version of this article to the authors of the meta-analyses for comment, one of them asked me to revise the article into a full meta-analysis of all recent meta-analyses. Such an article would represent a more even-handed critique, given that these meta-analysts would likely find themselves in the company of the authors of most other recent meta-analyses. A longer article will be justified if the quality of meta-analyses in our subject areas does not improve in the next year or two.Acknowledgements: I thank Alan Batterham for reviewing the manuscript and providing valuable suggestions for improvement.References ADDIN EN.REFLIST Bonetti DL, Hopkins WG (2009). Sea-level exercise performance following adaptation to hypoxia. Sports Medicine 39, 107-127Braakhuis AJ, Hopkins WG (2015). Impact of dietary antioxidants on sport performance: a review. Sports Medicine 45, 939-955Burden RJ, Morton K, Richards T, Whyte GP, Pedlar CR (2015). Is iron treatment beneficial in, iron-deficient but non-anaemic (IDNA) endurance athletes? A meta-analysis. British Journal of Sports Medicine 49, 1389-1397Carr AJ, Hopkins WG, Gore CJ (2011). Effects of acute alkalosis and acidosis on performance. Sports Medicine 41, 801-814Cassar S, Misso ML, Hopkins WG, Shaw CS, Teede HJ, Stepto NK (2017). Insulin resistance in polycystic ovary syndrome: a systematic review and meta-analysis of euglycaemic-hyperinsulinaemic clamp studies. Human Reproduction 31, 2619–2631 ()Chu A, Petocz P, Samman S (2016). Immediate effects of aerobic exercise on plasma/serum zinc levels: a meta-analysis. Medicine and Science in Sports and Exercise 48, 726-733Coquart JB, Tabben M, Farooq A, Tourny C, Eston R (2016a). Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study. Sports Medicine 46, 885-897Coquart JB, Tabben M, Tourny C, Chamari K, Eston RG (2016b). Author’s Reply to Will G. Hopkins: “Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study”. Sports Medicine 46, 1197-1198Fisher DJ, Carpenter JR, Morris TP, Freeman SC, Tierney JF (2017). Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach? BMJ 356, j573Higgins JPT (2008). Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified. International Journal of Epidemiology 37, 1158-1160Higgins JPT, Thompson. SG, Spiegelhalter DJ (2009). A re-evaluation of random-effects meta-analysis. Jurnal of the Royal Statistical Society A 172, 137-159Higgins JPT, Green S (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Cochrane Collaboration. cochrane-. Accessed 15 Jan 2016.Hopkins WG (2004). An introduction to meta-analysis. Sportscience 8, 20-24Hopkins WG, Marshall SW, Batterham AM, Hanin J (2009). Progressive statistics for studies in sports medicine and exercise science. Medicine and Science in Sports and Exercise 41, 3-12Hopkins WG (2015). Individual responses made easy. Journal of Applied Physiology 118, 1444-1446Hopkins WG (2016). Comment on: “Submaximal, perceptually regulated exercise testing predicts maximal oxygen uptake: a meta-analysis study”. Sports Medicine 8, 1195-1196Hopkins WG, Batterham AM (2016). Error rates, decisive outcomes and publication bias with several inferential methods. Sports Medicine 46, 1563-1573Hume PA, Lorimer AV, Griffiths PC, Carlson I, Lamont M (2015). Recreational snow-sports injury risk factors and countermeasures: a meta-analysis review and Haddon matrix evaluation. Sports Medicine 45, 1175-1190Josefsson T, Lindwall M, Archer T (2014). Physical exercise intervention in depressive disorders: Meta‐analysis and systematic review. Scandinavian Journal of Medicine and Science in Sports 24, 259-272McGrath LJ, Hopkins WG, Hinckson EA (2015). Associations of objectively measured built-environment attributes with youth moderate–vigorous physical activity: a systematic review and meta-analysis. Sports Medicine 45, 841-865Peterson MD, Sen A, Gordon PM (2011). Influence of resistance exercise on lean body mass in aging adults: a meta-analysis. Medicine and Science in Sports and Exercise 43, 249Salvador A, De Aguiar R, Lisboa F, Pereira K, Cruz R, Caputo F (2016). Ischemic preconditioning and exercise performance: a systematic review and meta-analysis. International Journal of Sports Physiology and Performance 11, 4-14Snowling NJ, Hopkins WG (2006). Effects of different modes of exercise training on glucose control and risk factors for complications in type 2 diabetic patients a meta-analysis. Diabetes Care 29, 2518-2527Somerville VS, Braakhuis AJ, Hopkins WG (2016). Effect of flavonoids on upper respiratory tract infections and immune function: A systematic review and meta-analysis. Advances in Nutrition 7, 488-497Soomro N, Sanders R, Hackett D, Hubka T, Ebrahimi S, Freeston J, Cobley S (2015). The eficacy of injury prevention programs in adolescent team sports: a meta-analysis. American Journal of Sports Medicine, PB, Joseph C, Angioi M (2015). Effects of vitamin D supplementation on upper and lower body muscle strength levels in healthy individuals. A systematic review with meta-analysis. Journal of Science and Medicine in Sport 18, 575-580Vandenbogaerde TJ, Hopkins WG (2011). Effects of acute carbohydrate supplementation on endurance performance. Sports Medicine 41, 773-792Weston M, Taylor KL, Batterham AM, Hopkins WG (2014). Effects of low-volume high-intensity interval training (HIT) on fitness in adults: a meta-analysis of controlled and non-controlled trials. Sports Medicine 44, 1005-1017Wright AA, Taylor JB, Ford KR, Siska L, Smoliga JM (2015). Risk factors associated with lower extremity stress fractures in runners: a systematic review with meta-analysis. British Journal of Sports Medicine 49, 1517-1523Published Jan 2018. ?2018 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download