New Metrics and Tools for Evaluating the Impact of NIH ...

New metrics and tools for evaluating the impact of NIH-funded research

Council of Councils January 27, 2017 Jim Anderson, DPCPSI George Santangelo, DPCPSI/OPA

Office of Portfolio Analysis

Mission of the NIH Office of Portfolio Analysis (OPA)

? Coordination of portfolio analysis activities at NIH

? Plan and host seminars, workshops, and symposia ? Create opportunities for crosstalk within the NIH community

Portfolio Analysis Interest Group and blog (The Analyst)

? Training

? Formal classes, ad hoc sessions, and office hours ? OPA web site: user manuals, FAQs, etc.

? Consultation & other services

? Conduct analyses for NIH senior leadership ? Data acquisition, data cleaning, database management ? Assist NIH staff in the 27 Institutes and Centers (ICs) with analyses

Has resulted in collaborative development of tools, case studies, etc.

? Developing a science of portfolio analysis

? Establish and disseminate best practices in analysis and validation ? Develop new tools / approaches and augment pre-existing ones ? Build a community of experts: government, academia, private sector

Office of Portfolio Analysis

OPA iTools suite to find gaps and new opportunities, flag overlap, and evaluate project outcomes

? Data-driven approaches to characterize research investments and the resulting impact

? Validated and effective methodology

? OPA offers training in portfolio analysis methodologies, including specific training in the use of OPA iTools, to all NIH staff:

Functionality

Content analysis Efficient disambiguation Effective bibliometrics Map translational science Track patents, licensing, start-up activity NextGen portfolio analysis platform

Tool

IN-SPIRE and Word2vec iClean iCite iTrans

iTech (in dev.) iSearch

Office of Portfolio Analysis

A new way to measure the influence of NIH awards on progress in biomedical science

Public website to retrieve RCR data: iCite.od.

Office of Portfolio Analysis

Diversified metrics are needed in research assessment: tracking bench-to-bedside translation

Molecular/Cellular MeSH terms

Phase II-IV, JAMA Phase I, Neurology

Human Medical Subject Heading (MeSH) terms

BMJ

N Engl J Med

Brain, Annals Neurol

Griffin Weber Identifying translation science within the triangle of biomedicine J Trans Med 11:126 (2013)

Nat Genet J Translational Med

Cereb Cortex Nat Med, Sci Transl Med

Nature Science

Sci Signaling Cell

J Mol Biol

Dev Neurobiol

J Neurosci Neuron, Nat Neurosci

J Comparative Neurology

Neural Dev

Animal MeSH terms

Office of Portfolio Analysis

Diversified metrics are needed in research assessment: tracking bench-to-bedside translation

Clinical trials

Mutations in genes of interest discovered in human patients

Candidate pharmacological modulators developed

Biological mechanism of disease elucidated in model systems

Pharmaceuticals tested in animals

Office of Portfolio Analysis

Diversified metrics are needed in research assessment: tracking bench-to-bedside translation

Voraxaze is used to treat methotrexate buildup in patients with kidney failure

Office of Portfolio Analysis

Tracking translational development with iTrans

Example: Cancer immunotherapeutic agents

iTrans content visualization

Citation pattern

2010

1987 to 2014

to 2014

human

2010 to

2014

2003 to

2007

1987 to

1996

Low Article Density High

2003 to

2007

1987 to

1996

Clinical Research Translational Research Basic Research

Office of Portfolio Analysis

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download