Research Evaluation

Informatics Europe supports research in informatics by promoting quality standards and best practices in diverse areas of informatics.

In 2008, Informatics Europe published the report "Research Evaluation for Computer Science" (a shorter version was also published in the Communications of the ACM). The report was the first attempt to put forward a set of best practices for the evaluation of researchers in Europe. Building on the CRA’s work in the US, the report has been highly influential in the European Informatics community, and an important reference for numerous departments and colleagues across Europe.

A lot has changed in the past decade: pressure for using quantitative measures got even stronger, open-source publications and archives have been established, new solutions have appeared around the debate on conferences vs. journals, new conference rankings are available, artifact evaluation is rising, transfer to industry and society is more and more encouraged, and the use of indexes and bibliometrics has become even stronger. In 2016, Informatics Europe established a working group to examine all these changes and bring forward an updated set of recommendations for research evaluation in Informatics and closely related areas while reinforcing the ongoing issues raised in the previous report. The final report on "Informatics Research Evaluation" was released in March 2018 (available for logged in IE members, a summary is openly accessible by following this link)

More about our achievements and other contributions to research evaluation in informatics can also be found under the tab “Resources”.

• Task Forces
• Outcomes
• Resources

• 2024 - "Informatics Research Evaluation Report Revision"

  • Pekka Orponen, Aalto University (Chair)
  • Carlos Baquero-Moreno, University of Porto
  • Manuel Carro, Technical University of Madrid / IMDEA Software Institute 
  • Michael Goedicke, University of Duisburg-Essen
  • Martin Hitz, Alpen-Adria-Universität Klagenfurt
  • Pierre Paradinas, SIF - Société Informatique de France
  • Silvio Peroni, GRIN - Associazione Italiana dei Docenti Universitari di Informatica
  • Heri Ramampiaro, NTNU
  • Ernest Teniente, SCIE / Universitat Politècnica de Catalunya
  • Heike Trautmann, University of Paderborn

  2017-2018 - "Informatics Research Evaluation" Report

  • Floriana Esposito, University of Bari (Chair)
  • Carlo Ghezzi, Politecnico di Milano
  • Manuel Hermenegildo, IMDEA Software Institute, Technical University of Madrid
  • Hélène Kirchner, Inria
  • Luke Ong, University of Oxford

  2008-2009 - "Research Evaluation for Computer Science" Report

  • Bertrand Meyer, ETH Zurich (Chair)
  • Christine Choppy, University of Paris 13
  • Jørgen Staunstrup, IT University of Copenhagen
  • Jan van Leeuwen, University of Utrecht

  *Note that the affiliation indicated was the one at the time of their participation in the Informatics education activities. Some people have changed affiliation since then.

• • News Section: “Research Evaluation in Informatics” 
    Read More
    Date: April 2018 
    Details: A section about “Research Evaluation in Informatics" was published in the ERCIM News (Issue 113). 
  • Report: “Informatics Research Evaluation”
    Openly Accessible Summary ¦¦ Full Report (for logged in IE members)
    Date: March 2018 
    Details: Updating the contents and reinforcing the lasting issues raised in the previous IE report on research evaluation.
  • Research Evaluation Activity Progress Update
    Read More
    Date: 24 October 2017
    Details: An update on the progress of the Research Evaluation activity was presented at the special session on Informatics Europe activities during the ECSS 2017.
  • Discussion Panel on Research Evaluation
    Read More
    Date: 26 October 2016
    Details: A discussion panel on Research Evaluation was held as part of the program of the ECSS 2016.
  • Paper "Research Evaluation for Computer Science"
    Read More
    Date: April 2009
    Details: Viewpoint about the Research Evaluation for Computer Science published in the Communications of the ACM, Vol. 52 No. 4, Pages 31-34, 2009. 
  • Report "Research Evaluation for Computer Science"
    Read More
    Date: 20 May 2008
    Details: The first version of the report detailing the evaluation of research in computer science.
    The updated version of the report was published in 2018. Logged in IE members can download the full 2018 report through this link, while a summary is openly accessible by following this link.

• CRA

  • Best Practices Memo - Incentivizing Quality and Impact: Evaluating Scholarship in Hiring, Tenure, and Promotion (2015)
  • Best Practices Memo - Evaluating Computer Scientists and Engineers For Promotion and Tenure (1999)

  Académie des Sciences, Leopoldina and Royal Society

  • Statement on good practice in the evaluation of researchers and research programmes

  European University Association

  • Reflections on University Research Assessment: key concepts, issues and actors
  • 2017-2018 EUA Open Access Survey Results