What counts in science?
A recent pan-EMBL event provided an opportunity to reflect on responsible research assessment in scientific institutions
By Lisanna Paladin, Dana-Maria Grozavu, Victoria Yan, and Alicia Perez Saturnino from EMBL Heidelberg, Katherine Anne Hughes from EMBL-EBI, and Daniel Santos-Oliván from EMBL Barcelona
Reforming how research and researchers are assessed is an essential step toward ensuring a more accurate, fair, and meaningful evaluation of scientific work. Across research institutions, funders, and scientific communities, there is increasing acknowledgement that the traditional emphasis solely on metrics and publications fails to capture the full scope and impact of scientific work. To explore these issues in depth, we recently hosted a pan-EMBL hybrid event on Responsible Research Assessment (RRA) to share insights, experiences, and practical approaches for change.
Wolfgang Huber, Group Leader and co-chair of the RRA Working Group at EMBL, opened the day by highlighting the issues surrounding overreliance on metrics and the need to consider all research outputs. After that, four interactive sessions covered topics ranging from funding practices and narrative CVs to the recognition of diverse research outputs and system-wide reform strategies. Here are four key takeaways from the event.
1. Funding and hiring decisions still struggle with opacity and biases
During a roundtable discussion on career transitions and funding applications, participants noted that the vagueness in job descriptions or the unstructured nature of hiring processes can be challenging. This lack of clarity often leads to confusion about what applicants should highlight. When it comes to research funding, decision-makers struggle to differentiate between applications of equal scientific merit, the so-called “Grey Zone”. Broader contributions, such as outreach or data sharing, are a part of the scientific merit and should therefore be considered in funding decisions. Participants also agreed on the need for increased transparency and formal feedback mechanisms in both funding and hiring processes to achieve fairer and more effective practices.
2. Narrative CVs offer promise but require adequate guidance
A narrative CV is generally recognised as a structured written description of a researcher’s contributions and achievements that reflects a broad range of relevant skills and experiences. This definition is still evolving, however. The goal of one of the sessions was to collectively assess narrative CVs through a SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis. While narrative CVs allow researchers to showcase skills and contributions in a more holistic way, they also introduce new challenges. These include the additional time and writing skills required, and a fear that some applicants may exaggerate achievements. Despite these concerns, narrative CVs present an important opportunity to move beyond metrics to better reflect impact, career stage, and context, especially if supported by clear guidance and fair assessment practices.
3. Recognition of diverse research outputs is crucial but underdeveloped
A collective brainstorming session highlighted the strong interest across EMBL in recognising diverse research outputs and contributions to the scientific community beyond traditional publications. Such outputs can include datasets, protocols, software and infrastructure, outreach and education, as well as policy contributions. Participants agreed that these outputs are vital to scientific progress and should be more visible in evaluation processes. However, many questions arise around implementation: How should these contributions be documented? What specific contributions should be taken into account, and how can they be assessed? The group emphasised the need for transparent frameworks that make expectations clear for both candidates and evaluators, ensuring all forms of impactful work are recognised.
4. Reforming research assessment requires iteration, collaboration, and a shared vision
In the concluding panel discussion, speakers addressed the broader cultural and systemic barriers to reform. Institutional inertia and ingrained habits often hinder progress, even when the need for change is widely supported. Panellists James Sharpe (EMBL Barcelona), Angeliki Tzouganatou (OpenAIRE), Vinciane Gaillard (European University Association), Bernd Pulverer (EMBO Press), and Emma Ganley (protocols.io), recommended an iterative approach: start with small, evidence-based reforms and adapt as lessons are learned. Piloting new approaches in willing institutions or departments can provide models for broader adoption. Whether it’s shifting to narrative CVs, valuing non-traditional outputs, or changing hiring and funding practices, progress requires alignment across the research ecosystem. Funders, institutions, publishers, and researchers all have a role to play. The importance of collaboration was also a recurring theme during the discussion, with initiatives like DORA, CoARA, and cOAlition S cited as examples of cross-institutional efforts that can help drive system-wide change. Establishing a shared vision and working collectively will be essential for creating lasting, meaningful improvements in research assessment.
Conclusion: a community effort toward meaningful change
The event underlined both the urgency and the complexity of reforming research assessment as well as the need for shared commitment to transparency, inclusivity, and evidence-based practices. Research institutions, including EMBL, are increasingly recognising the need to explore new approaches and invite ongoing input and discussion, fostering a more responsible approach to research assessment. Sustained dialogue, experimentation, and collaboration will be the keys to ensuring the system better serves both science and society. This effort requires a combination of both top-down and bottom-up approaches. Funding agencies and governments have the power to implement systemic change that incorporates society’s needs, while the scientific community must be actively involved to ensure that these changes are meaningful and address the real needs of the people doing science.
As an international research organisation, EMBL has a responsibility – and an opportunity – to help lead this shift. Redefining what counts in science to value what truly matters, not just what is easy to measure, is essential to be able to advance towards a more responsible research assessment.
