The Academy of Finland’s efforts to find the best and most promising research projects depend most importantly on high-level international peer reviews. Review reports are the primary basis for funding decisions made by the Academy’s research councils, the Strategic Research Council, the Finnish Research Infrastructure Committee and the Academy’s subcommittees. The decision-makers also consider objectives related to science policy.
The reports are prepared by panels of leading international experts. The Academy receives each year around 4,000 applications. These applications are reviewed by almost 1,000 reviewers, some 95 per cent of whom come from outside Finland. This ensures that the projects approved for funding are of the highest possible international standard.
A lot of thought goes into making sure that the panel compositions are just right for the full range of applications received. The first part of this article series looks more closely at how panels are assembled and at how the applications are checked.
The transition from checking the applications to the panel review discussions takes some time because it is crucial that once the panels have been appointed, their expert members can form an impression of the applications well ahead of the panel meetings. The duration of one panel depends on its working methods and the number of applications, but usually a panel will spend one to two days reviewing applications.
“For staff reasons we can’t have several panels meeting at the same time. From mid-January through to the end of February, our unit will have one panel or another hard at work almost every day,” says Juha Latikka, Senior Science Adviser at the Research Council for Natural Sciences and Engineering.
The panels involve presentations and discussions that allow the panellists to make their assessments and rate each application. The presentations are intended to provide a sound basis for the panellists’ discussions on the applications, which they will always study ahead of time.
“Each application is assigned to at least two panellists for a more in-depth reading. It’s the role of these panellists to prepare preliminary reviews in which they weigh and assess the application in advance, before the panel convenes. However, every expert has access to all applications and all preliminary reviews. If there are 40 applications and eight panellists in a panel, each panellist will have a list of ten applications on which to prepare a presentation. The aim is always to assign applications to panellists who have the best possible expertise in the subject matter concerned,” Latikka explains.
Tailored panels well suited for reviewing multidisciplinary applications
“One review method used in other countries is to have fixed-composition panels, with applicants invited to submit their application to the panel of their choice. As the experts on these panels are appointed in advance, they won’t necessarily have the best possible expertise for the full range of applications they will be reviewing. The Academy’s approach of handpicking the experts once the applications have arrived ensures that even multidisciplinary applications receive a fair and balanced assessment. The feedback we’re getting suggests that the panellists themselves feel their expertise provides excellent coverage of all the applications,” Latikka says.
The panel’s review reports are entered in an review form to ensure consistency in length and content. As the review reports are structured and directly comparable, it’s easier for the research council to use them in its decision-making. All applicants can access and read the reports once the final funding decisions have been made. In the application process, therefore, even projects that are turned down receive valuable feedback on their work.
“We’re very open about how the applications are reviewed. The guidelines for preparing a research plan clearly state what elements the plan should include, and the application review forms are all posted online,” Latikka says.
Read the other articles in this series: Senior Science Advisers Aki Salo and Hannele Kurki explain what happens in the first stage of the application process, and Anneli Anttonen, Chair of the Research Council for Culture and Society, describes a council’s role in the third stage of the decision-making process.