-A +A
Print

Reforming scientific assessment: the CNRS among the first signatories

Corporate

On 28 September, the European Commission launched the collection of signatures for an agreement whose signatories will form a coalition to reform research assessment. The CNRS has already signed the document. Explanations provided by Alain Schuhl, CNRS Deputy CEO for Science.

A ceremony to mark the launch of signature collection

Mariya Gabriel devant un écran où l'on peut voir le logo du CNRS

On 28 September, during the European Research and Innovation Days—the fourth edition of a major event bringing together high-level stakeholders in the worlds of research, business, policy, and civil society to shape the future of research and innovation in Europe—a ceremony opened the signature collection period for the agreement. Participants include the European Commissioner Mariya Gabriel, the French Minister for Higher Education and Research Sylvie Retailleau, the Vice President of the European University Association (EUA) Paul Boyle, and the Secretary General of Science Europe Lidia Borrell-Damián, on behalf of her President Marc Schiltz who led the effort to draft the document between January and July 2022, with Sylvie Rousset, Head of the CNRS’s Open Research Data Department, notably taking part.

Why was it so important for the CNRS to be among the first signatories of this document, published last 20 July?
Alain Schuhl: This agreement, which is in keeping with the DORA declaration1  that the CNRS signed in 2018, presents the central principles for reforming current research assessment practices, and proposes constituting a Coalition with signatory institutions. Based on five major principles—quality, impact, diversity, inclusiveness, and collaboration—we believe this reform is necessary to strengthen the impact of science, with the Coalition providing a solid basis for moving forward. Our participation is therefore in line with our earlier commitment in this area.

Assessment is one of the four fundamental pillars of the CNRS’s roadmap for open science published in November 2019, designed to implement more quality-based rather than quantity-based criteria. We hope in this way to attract and promote more varied profiles, with the most recent competitive entrance examination results suggesting that we are on the right path. The CNRS aims to be proactive regarding assessment, and to bring with it the entire French academic world. Presented during the Open Science European Conference (OSEC 2022) in February—which launched a dozen events organized by the CNRS in connection with the French Presidency of the Council of the European Union—the Paris Call expands this process to the European level, with the signing of this far-reaching reform agreement extending these initiatives.

What are the key points of the agreement?
A. S.: The agreement focuses on the assessment of research organizations, research projects, and individual researchers. This latter element is the most important for us. It entails recognizing diverse contributions and careers in the procedures used for the individual assessment of scientists, taking into account activities such as teaching, supervising students, scientific mediation, innovation, involvement in open science or the community, etc. Freeing ourselves of quantitative assessment based exclusively on publications is central to the approach, namely by emphasizing qualitative assessment by peers instead. This involves abandoning the systematic use of bibliometric indicators (impact factor, H-index, etc.) to assess the quality of individual research, and avoiding the use of international institutional rankings in the assessment criteria for organizations. This will be done all while adhering to ethics, parity, integrity, scientific freedom, and the autonomy of organizations.

We thoroughly agree with these principles, and even anticipated them. Before the publication of our roadmap for open science, the CNRS had already signed the DORA declaration in July 2018. We are also working with the sections and commissions of the CoNRS2  to improve assessment criteria for decisions relating to recruitment, tenure, and promotion. All references to quantitative indicators, especially those based on publications, are being removed. At the same time, the format of the assessment form for scientists was revised: they are henceforth asked to situate themselves in relation to five aspects3  of their activities, and to emphasize the importance of a few well-selected publications rather than to give an exhaustive list with no details.

How was the reform proposal drafted?
A. S.: The collection of signatures is the culmination of a process that began in 2021 with the European Commission’s publication of a report entitled Towards a reform of the research assessment system, which was written after lengthy consultations with numerous actors. It was followed by a call for expression of interest to join a coalition to initiate the reform. The call was open to all organizations engaging in the assessment of research activity: universities, schools, organizations, funding agencies, specialized organizations (such as HCERES), etc. The CNRS and over 350 organizations from more than 40 countries expressed their interest. In early 2022, the Commission established a Committee to draft a document presenting a common orientation. This Committee interacted regularly with a group of 20 experts representing a variety of organizations, including the CNRS. We were therefore involved from the very beginning in constructing this Agreement on reforming research assessment. It is this document that is now open to signature as of 28 September 2022.

What are the next steps in the process?
A. S.: The Coalition should be established by the end of the year. The signatories commit to three points: to mobilize organizational resources explicitly devoted to reforming research assessment; to develop and implement criteria and assessment methods for research units and institutions, projects, and scientists, by involving the research community; and to communicate regarding these changes, as well as to assess with the greatest transparency, both within and beyond the coalition, with a view to mutual learning. The coalition is also intended to be global, and not simply reserved for the European Union. Research assessment reform is an extended but crucial undertaking.

 

More: The website of the future Coalition for Advancing Research Assessment (CoARA) is now publicly available at: www.coara.eu.

  • 1The San Francisco Declaration on Research Assessment brought together over 20,000 signatories from 148 countries—namely assessment stakeholders, including both institutions and individual scientists—around 18 commitments, with particular emphasis on assessment based on scientific content.
  • 2A collective body made up of the Scientific Council, Scientific Councils from CNRS’ institutes, sections specializing in each of the disciplines, and interdisciplinary commissions, the National Committee for Scientific Research (CoNRS) contributes to the development of the CNRS' scientific policy and participates in the recruitment and monitoring of the careers of researchers, and in monitoring the activity of the research units.
  • 3The five activities are: contributing to scientific advances; training through research, teaching, and exploiting and diffusing scientific knowledge; innovating and transferring research results; managing and/or assuming collective responsibilities; and engaging in geographical, thematic, or functional mobilities.