NSERC peer review system: broken and unreliable

Jim Colliander has a post on the results of the 2011 NSERC Discovery Grants competition in mathematics. The official results have not yet been announced publicly, but the data that Jim has collected in his department does not bode well for mathematics in Canada. Judging by the outcomes, the new peer review system is dysfunctional, unreliable and unstable.

Please read Jim’s post – based on the information currently available, I agree 100% with his conclusions.

I would like to join him in asking NSERC officials and the Evaluation Committee members to step up and explain how this happened.

I also would like to ask NSERC to explain how they could possibly expect this system to function in terms of graduate and postgraduate training. Many senior researchers have seen their 40-50K grants cut to 15-18K per year, a pitiful amount that’s just about enough to pay for 1 graduate student at a bare subsistence level (8K at UBC), incidentals such as books and computer equipment, plus 2-3 conferences and research visits, maybe 4 if they plan carefully. If that’s how they spend the money – and you can’t really cut the travel and collaboration expenses any further without serious damage to your career – their meagre record of graduate training might not get them ranked highly enough next time around to maintain even that level of funding. A Catch-22 situation if there ever was one.

I’d like to know how this is consistent with NSERC’s own estimation (along with SSHRC and CIHR) of how much money our top graduate students and postdocs should be making. Vanier Graduate Scholarships are valued at 50K per year. Banting Postdoctoral Fellowships are valued at 70K per year. At that rate, it would take 10 mathematics faculty with 15-18K grants (with the travel and collaboration allowance subtracted) to fund a single postdoctoral fellow, and 7 or 8 such faculty to fund a graduate student.

A couple more things from my own experience of being isolated research-wise in my department for a long time (2000-06) and then having a very small group of 2 (2006-08):

Instability favours large groups. No, take that back – it doesn’t favour anyone, really – but it’s easier for large groups to survive it. One person might get cut, but then someone else might get an increase, and the total amount of money available to the group is much more likely to be stable. Of course, this only works if the group members share enough research interests to be able to work with the same postdocs for example.

Emphasis on HQP training favours large groups. A large group is likely to have a steady flow of graduate students and postdocs in the pipeline. It is likely to have regular graduate course offerings in place, making it easier to bring new students up to speed. It is able to make competitive postdoc offers to top candidates. And at the end of the day, all 4 or 5 or 7 group members can boost their HQP scores by listing every postdoc associated with the group on their application. Meanwhile, a lone supervisor has to start from scratch. Graduate course offerings have no relevance to her research program, so she tutors her graduate students one-on-one right from the basics. Aside from maybe a handful of mathematicians across Canada with the top 50-60K grants, it is very hard for a single faculty member to find the money for any postdoctoral offer at all, let alone one sufficiently competitive (in terms of salary and duration) to attract a candidate who’s actually good enough to have other choices. (NSERC’s Use of Funds restrictions on 3-year offers don’t make it any easier, either, but that’s another story.)

Can isolated researchers ever reach the top tiers of funding, regardless of their actual research excellence? Or are we going to reward success in hiring shuffles instead? Should we focus on our research and training programs, or should we fight tooth and nail to get some more faculty hired in our area because that’s the only way for us to stay afloat? How is this going to affect our research environment in the long run?


Filed under research funding

4 responses to “NSERC peer review system: broken and unreliable

  1. Thanks Izabella for highlighting my post. Mathematics in Canada faces two problems:
    1. There is not enough money in the system to run successful research activities.
    2. The peer review system is failing to distribute the limited funds strategically.

    Unless the system is fixed, we will soon see the consequence: the most talented researchers (who can leave) will leave Canada.

  2. I am glad to see that the mathematics community is speaking out against the “new and improved” NSERC. I do want to assure you all that even in Chemistry the effects of the changes to the Discovery Grant process are having similar effects.

    Our Evaluation Group took it upon themselves to raise the bar for funding in Chemistry, “in order to protect the buying power of highly ranked individuals” according to Isabelle Blain. This is code for “tax breaks for the rich paid for by service cuts to the poor”.

    On top of that, as you observe, small research groups cannot win in the current system no matter how high their quality. Quantity trumps all in the current system.

  3. Pingback: Weekly Picks « Mathblogging.org — the Blog

  4. Mike

    Thanks for this posting. It is not only mathematics but other programs are suffering as well unless you are in a group that has more representation at NSERC. Yes- NSERC seems to have degraded significantly from what it was before. For example, it is not clear how grant selection committees are selected. One that I am familiar with, has a huge number of people from Chemical Engineering (90%) and few from other disciplines. And these people evaluate researchers from Electrical and Mechanical Engineering. The reviewers have literally no say and their comments are often ignored. Many programs have been created and huge amounts of funding are given to researchers without evaluating what they do with the money while many researchers are starving for grant money.