PhD merit needs to be defined by more than just publications

There needs to be a change in how PhDs are assessed, with more emphasis placed on other aspects of training, to promote a positive culture for early career researchers.

Two researchers look at a computer screen

Credit: Wellcome

Anne-Marie Coriat: "Designing PhD training to deliver high-impact journal publications as a standalone tool for assessment is not appropriate."

The importance of research to society and its link to prosperity and competitiveness is well documented. Global organisations and governments are increasingly prioritising research skills and the need for PhD training in national strategies. Beneath this glowing rhetoric and increased funding for PhD training are ever more personal accounts from PhD students and early career researchers about the struggles, challenges and detrimental effects of the current system.

As my colleague at Wellcome, Ben Bleasdale, recently highlighted in his blog, whilst there are many fulfilling aspects of a career in research, and the majority of PhD students remain fulfilled, satisfaction levels have been in sustained decline for decades.

More graduates are being trained than ever before, but concerns exist about the hypercompetitive nature of research and the extent to which declines in wellbeing are linked to an over-reliance on blunt, easy-to-collect metrics. If assessment focuses on measures of what is done and pays little attention to how research is conducted, pressures are likely to build and the declines we see in wellbeing will continue.

The use of blunt metrics has a potentially distorting impact on careers, collaboration, innovation, research quality and wellbeing more generally. James Wilsden’s report, The Metric Tide, sets out the issues well. The questions of whether and why students need to publish peer-reviewed journal articles hint at wider issues in the research system, beyond whether publishing in peer-reviewed journals is a good thing.

The PhD is a useful qualification for a very broad range of careers. Why, then, is there such an early and persistent focus on publications? Our own review of PhD training shows that supervisors place roughly equal weight on first-author publication as they do on completing and defending a thesis or transitioning to a career that utilises skills gained during PhD training.

There are many interconnected factors that make peer review publication such hard currency:

  • frameworks for allocation of research funding use publication metrics as shorthand for productivity and impact
  • research is international – publications are seen as a ‘passport’ providing easily understood credibility for researchers
  • supervisors are incentivised through assessment to ensure publications are a product of research.

A UK academic I spoke to recently summarised often-referenced concerns and said, 'supervisors are very concerned with assessment frameworks; we get beaten about the head with them by our institutions. It affects the way we run our research. I’d be in a difficult position if I didn’t publish student work, sometimes in high-profile journals, and that means aiming for that most of the time.'

To make real progress, we must set new expectations for what excellence looks like. This is something we want to address at Wellcome and we’re committing to trying different initiatives to drive forward change.

Help us reimagine research

This article is part of our campaign to build a better research culture – one that is creative, inclusive and honest.

Tell us what matters >

An example is the redesign of our approach to PhD training. We now explicitly require programmes to address not only how they will ensure excellence in research quality but also how they will support a more positive research culture. Research excellence alone will not be enough. When it comes to publications, we are clear that learning how to write research articles and engaging with peer review is an important part of PhD training, but that designing PhD training to deliver high-impact journal publications as a standalone tool for assessment is not appropriate.

We are encouraging programmes and supervisors to support students to submit their research to platforms such as Wellcome Open Research, or to preprint servers like bioRxiv. This will allow trainees to get credit for their work in a timely manner and, in the case of Wellcome Open Research, to also get credit for any peer reviews they perform (in partnership with their supervisor).

As a signatory to the Declaration on Research Assessment (DORA), Wellcome assesses researchers based on the work they have done and does not use journal names or journal impact factors as proxies for quality.

We don’t have the answers yet, but we do know it is time for us to start taking action. We’re open to testing different approaches and hearing from researchers at all levels about their ideas of how we can improve.

So, my answer to the question ‘should PhD students publish in peer reviewed journals?’ is yes. But, if asked ‘should publishing in peer-reviewed journals be an absolute requirement for successful completion of PhD training?’ I would answer no. Learning to analyse data, articulate and explain results clearly, and be open to peer review is a key foundation of research.

A traditional peer-reviewed publication is only one way of achieving this, but we need to change assessment and expectations to ensure a more balanced view of PhD training evolves. 

This article was first published in Nature Human Behaviour World View,10 October 2019. 

Related links

Related news