A scientist’s performance is often judged on two factors: productivity, as measured through the number of publications; and impact, as indicated by the number of times those papers are cited. It's generally thought that impact peaks early in a scientist's career, while having a mature, established research program boosts productivity. But little is actually known about how productivity and impact evolve over time.
In a recent investigation published in Science, people actually looked at how these measures change throughout the course of a scientist’s career, finding that our assumptions aren't always well founded.
In order to understand how performance measures change over time, the publication profiles of scientists across multiple disciplines were reconstructed. For each scientist, the number of publications and impact were recorded. The impact was captured by the total number of citations 10 years after publication. The most influential work was identified as the single paper with the most citations at that time.
The team also grouped the scientists based on their peak impact, regardless of when it occurred: high maximum impact (top 5 percent), low maximum impact (bottom 20 percent), and medium maximum impact (middle 75 percent).
Productivity and impact
The first question tackled was how productivity changes over time. Productivity, as determined by the number of publications authored, generally increases during the course of a scientist’s career. For high-impact scientists, productivity grew almost threefold. However, for low-impact scientists, productivity growth was more modest. The team also randomized each scientist's career by exchanging the impact of all their papers; this let them test whether trends or timing influenced when the highest-impact work is done.
Next, the study looked at the times during which a scientist's most influential publications occurred. For each scientist, researchers determined the number of years that had passed between the scientist’s first publication and their highest-impact paper. This information was used to evaluate the probability that the highest-impact paper would occur in any given year following the first publication. Analysis indicates that most scientists publish their highest-impact paper within the first 20 years of their career; after 20 years that probability drops.
To explore the origin of this pattern, researchers randomized the order of publications and performed the same analysis. They found that the impact probability of the fabricated career is indistinguishable from the original study. This suggests that the probability is dependent on the year-to-year variations in productivity through an individual’s career.
Finally, the researchers measured the probability of the most-cited work coming early or late in the sequence of published papers. This analysis revealed that impact is randomly distributed within a scientist’s body of work.
Neither publication time nor publication order influences this finding. The highest-impact work can occur anywhere in the sequence of publications with equal probability regardless of discipline, career length, career time period, number of authors, and assigned contributions of authors. Further analysis revealed that growth in average impact during a career in science can be attributed to growing productivity and that ability or excellence do not influence how impact evolves over time.
Predicting success
Using this information, the team created a model that reveals underlying patterns governing the emergence of scientific success. Since impact varies greatly between scientists, a unique individual parameter Q was determined for each scientist to capture this aspect. The Q parameter is stable during an individual’s career and accurately predicts the evolution of a scientist’s impact, which comes through factors that include independent recognition and cumulative citations.
This investigation suggests that it may be possible to conduct a standardized evaluation of an individual scientist’s performance in a quantitative manner. This news could be particularly welcome for academic institutes that rely on measures like number of publications and number of citations to gauge an academic’s success. However, we're unsure what impact this predictive model could have if it is used to judge scientists in their nascent years.
Science, 2016. DOI: 10.1126/science.aaf5239 (About DOIs).
reader comments
18