Suomen Akatemia

What do publication and citation analyses tell us about research funded by the Academy of Finland?

25 Jan 2022

The results of bibliometric analyses show that competitive research funding from the Academy is beneficial with regard to the level and impact of Finnish science. Measuring the quality of research by means of peer review also promotes the social impact of science.

The Academy of Finland distributes more than 400 million euros of competitive research funding each year. Key funding criteria include the scientific quality of applications and the impact of the research funded in its various forms. The Academy is interested in how funded research affects the level of research in Finland and what the resulting positive social impacts of that research are.

In recent years, analyses of publications from researchers funded by the Academy and the number of citations gained by those publications have been important ways of obtaining information on the impact of Academy funding. These analyses describe the impact that scientific publications have on the scientific community and on scientific debate in different disciplines. The Academy has commissioned the calculation of a bibliometric indicator since 2016. The indicator compares the scientific impact of publishing activities by researchers who have received Academy Project and Academy Research Fellow funding from the Academy of Finland with the scientific impact of publishing activities in Finland as a whole.

The comparison uses the top 10 index, which describes the relative proportion of publications that belong to the most highly cited 10% of publications over a given period of time in a certain field. If the index is 1, publications have the same scientific impact as the world average. If the index is more than one, publications have greater scientific impact than the world average.

The indicator results show that research published by Academy Projects and Academy Research Fellows clearly has more scientific impact than the world average. But even more importantly, research published by Academy Projects and Academy Research Fellows with funding from the Academy of Finland has greater scientific impact than research published in Finland in general.

Indicator calculation year Publication years used in the calculation Publication numbers Top 10 index
2016 2011–2014 Academy: 895
Finland: 36,035
Academy: 1.51
Finland: 1.09
2017 2011–2014 Academy: 827
Finland: 36,035
Academy: 1.21
Finland: 1.09
2018 2012–2015 Academy: 1,270
Finland: 38,060
Academy: 1.31
Finland: 1.12
2019 2013–2016 Academy: 1,572
Finland: 39,399
Academy: 1.36
Finland: 1.09
2020 2014–2017 Academy: 3,518
Finland: 42,886
Academy: 1.29
Finland: 1.09
2021 2015–2018 Academy: 3,085
Finland: 43,210
Academy: 1.19
Finland: 1.07

When reading the table, it should be noted that in 2016-2017 the indicator was calculated only on the basis of publications by Academy Projects, while since 2018 the indicator has included publications by Academy Projects and Academy Research Fellows. The indicator also included publications that were reported to the Academy by Academy Projects and Research Fellows until 2019, but all publications have been included since 2020.

Competitive Academy research funding is beneficial with regard to the level and impact of Finnish science

Results similar to those presented above have also been obtained from other analyses that examine the scientific impact of research funded by the Academy. For example, the scientific impact of publications that received Centre of Excellence funding or Postdoctoral Researcher funding is higher than the world or Finnish level.

However, there are certain reservations associated with bibliometric analyses. The publication databases used clearly cover publishing activities in social sciences and humanities less comprehensively than other fields, and this should be taken into account when interpreting the results. Secondly, the fact that scientific publication and citation analyses describe the growth of information and impact of publications on the scientific community rather than dimensions of research quality should also be taken into consideration. For example, it is very difficult to express the novelty value of research or the accuracy and reliability of methods and data as a single figure, such as a top 10 index.

In addition to scientific impact, research has broader social impact, which is presented in reports such as, The state of scientific research in Finland 2016: science builds world views and human understanding, produces wealth and prosperity, provides a foundation for decision-making, and develops professional practices. The impacts are transferred to the surrounding society through research results, cooperation and interaction as well as proficient people. Bibliometric analyses do not measure any broader impact than this, so they offer only one perspective on the impact of research.

The broader impact of Academy of Finland activities is assessed, for example, when examining the final reports of research projects and monitoring the impact of programme-type funding. However, the scientific impact illustrated by bibliometric analyses and the quality of research measured by means of peer review also lay the foundation for broader social impact.

*The results presented here are derived from the Web of Science ® prepared by CLARIVATE ANALYTICS ®, Inc. (Formerly the IP & Science business of Thomson Reuters®), Philadelphia, Pennsylvania, USA: © Copyright CLARIVATE ANALYTICS ® 2016-2021. All rights reserved. The results are taken with permission from the bibliometric analysis system provided by CSC – IT Center for Science Ltd., Espoo, Finland.

Do you have questions or feedback for us?