Conferences, peer review, and political interference

Our science minister Gary Goodyear is getting involved with organization and funding of scientific conferences. Last week he asked the Social Sciences and Humanities Research Council to reconsider funding for an upcoming conference at York University. Specifically, he recommended conducting a “second peer review”. Here is an excerpt from his official statement:

It has come to my attention that following a recommendation of a peer review board earlier this year, the Social Sciences and Humanities Research Council provided $19,750 under its Aid to Research Workshops and Conferences Program to a conference at York University entitled “Israel/Palestine: mapping models of statehood and prospects for peace”.

Approval of this funding was based on an initial proposal that did not include detailed information on the speakers at the conference. Since funding was provided, the organizers of the conference have added a number of speakers to their agenda.

Several individuals and organizations have expressed their grave concerns that some of the speakers have, in the past, made comments that have been seen to be anti-Israeli and anti-Semitic. Some have also expressed concerns that the event is no longer an academic research-focussed [sic] event.

The SSHRC did request an update from the conference organizers, then issued a statement to the effect that everything is in fine order, thank you very much, and the conference will be funded as planned.

The Canadian Association of University Teachers has called on Goodyear to step down:

“It’s unprecedented for a minister – let alone a minister from the department that funds the granting councils – to intervene personally with a granting council president to suggest that he review funding for an academic conference,” said CAUT executive director James Turk. “This kind of direct political interference in a funding decision made through an independent, peer-reviewed process is unacceptable and sets a very dangerous precedent.”

This blog is not an appropriate venue to discuss the Israeli-Palestinian situation, for more reasons than I could list here (and please keep that in mind if you would like to comment). Neither do I want to discuss the “hate speech” accusations such as those quoted in this article. The conference abstracts are over here. If any of them qualify as hate speech under Canadian law (see here), then there are appropriate procedures in place. Intimidation by political interference in the peer review process isn’t one of them.

I do want to repeat what I said in an earlier post: that academic freedom applies to all views expressed in the context of academic dialogue, including those we disagree with. Especially those we disagree with. That’s pretty much the point of it. And ultimately, academic freedom leads to better science. The correctness and significance of scientific ideas isn’t always clear right away and we’re better off if all such ideas are allowed to compete on their merits.

Of course, if an academic conference became a political event instead, then that would be a problem. However, a political science conference does not become a political event just because opinions about politics are being expressed. After all, that’s what political scientists do for a living. Political action – now that would be another matter. I don’t think, though, that we’ve seen any evidence of that.

But the main purpose of this post is to clarify several aspects of the organization and funding of academic conferences for those readers who have never been involved with that.

Conference grants are usually awarded as follows. The organizers submit a proposal containing a description of the subject matter, names of possible speakers, the format of the conference, the venue, duration and tentative dates. The proposal is then subject to peer review by a selection committee, external reviewers, or some combination of both. Based on the recommendations of the reviewers, the proposal may be funded as requested, funded at a lower level, or not funded.

A 20K amount is substantial, but not outlandish. For a 3-day workshop in Toronto (with most participants staying 4 nights), it is roughly enough to pay for accommodation and per diem for about 40 participants. However, we must also consider the travel costs of the participants. I have counted over 50 confirmed speakers, including about 15 from Israel and several more from overseas, on the conference web site. With that in mind, the given level of funding does not look very high. It covers only a fraction of the cost and I’m assuming that many participants will pay part of their expenses from their own grants.

Once a positive funding decision is made, it cannot normally be reversed except where program guidelines are violated. In the latter case, the decision to withhold funding would be made by the program officers of the granting agency. A “second peer review” is completely unheard of and would be in violation of the agency’s own rules.

In rare cases, the funding level may be adjusted after the initial decision (in either direction) if there are major changes to the budget of the granting agency. I do not know, though, of any cases involving a complete reversal of the decision. I don’t think it happens much, at least not in mathematics. The peer review for such things is typically quite strict. In addition to the proposal itself, the reviewers always consider the experience and track record of the organizers. Most proposers will have some experience organizing unfunded conferences (such as “special sessions” of their professional societies where everyone comes at their own expense) before they are given their first major conference grant. Similarly, if someone has a history of organizing less than successful conferences (most of the invited speakers don’t show up, the profile of the conference is too low given the funding level, etc), that does not bode well for their next proposal. This usually prevents any serious problems before they can occur.

It should be noted that the organizers commit the grant money when they issue invitations (“we would like to invite you to give a talk and we will cover your travel expenses up to $xxx”) and the participants make their plans based on that committment. Thus a reversal of a funding decision would affect the participants just as much as it would the organizers. It would not be done lightly.

It’s perfectly normal to add new speakers to the program after the proposal is funded. In fact, it would be rather unusual not to do so. First, most proposals include a tentative list of the speakers that the conference organizers would like to invite, but there is no guarantee that all of them will accept (a 70% acceptance rate is generally very good). If some of the invitees decline then further invitations are issued. Second, proposals are usually submitted well in advance of the conference (1-2 years). Any number of developments in the field might take place between the submission of the proposal and the dates of the conference, and it makes sense to account for them by inviting additional speakers. Third, many conferences include both invited talks and contributed talks. The latter means that there is a call to submit abstracts for consideration, then the organizing committee (or another reviewing body) selects the best submissions for presentation at the conference. Nobody can really predict who might or might not submit an abstract. And fourth, junior participants are often selected very shortly before the conference. That typically includes postdocs or finishing graduate students who were busy taking their quals and might have not chosen a research direction yet when the proposal was being written.

Under the program guidelines, “approval for major program changes (e.g., changing the theme or focus of the event) must be requested in writing to SSHRC”. A major program change, as I understand it, is for example if I submit a proposal for a conference in algebraic geometry and then have a conference in harmonic analysis instead. Of course, if I did actually submit a proposal for an algebraic geometry conference, the chances of that getting funded would be exactly zero. Caveat emptor. Adding a new topic or a few more speakers does not constitute a major change and the SSHRC guidelines recognize that explicitly.

Hope that this was useful.

Advertisements

6 Comments

Filed under academia, politics, research funding

6 responses to “Conferences, peer review, and political interference

  1. Random blogreader

    I have no idea what the rules are for such conferences, but these one-state solution people are advocating [redacted]

  2. I redacted the above comment because I’m not under any obligation to host a discussion of the Israeli-Palestinian situation in general or the one-state solution in particular. I said explicitly in my post that I was not interested in doing that. I wouldn’t even try to assess the scientific value of the conference – I just don’t know enough about the subject. However, the proposal was evaluated by a committee of experts who decided that it was valuable enough to award it a limited amount of funding. Either we have to trust the judgement of such committees, or else we might as well not have them.

    Once there is an established precedent for political intervention in the peer review process, fruit fly conferences might be next, then volcano monitoring, then all bets are off. Not to mention that there’s a good number of people who believe that conferences on global warming are unscientific.

    I also would like to take this opportunity to link to these two posts. Make sure that you read the discussion in comments, too. I don’t think I’m the only one who found this a bit unnerving.

  3. Anonymous

    I largely agree with you and certainly think it would set a terrible precedent to withdraw funding now. I know nothing about the specific case, but I think you’re being too charitable in assuming good faith on the part of grant applicants. People sometimes disguise their true motivations and plans when they submit grants. Here’s an example I’m familiar with (from time spent on NSF review panels, so I’ll write about what I know):

    Successful PIs often describe carefully and enthusiastically how they plan to work to increase diversity in mathematics (under the “broader impact” part of the NSF review criteria). With so much apparent enthusiasm, you’d think we’d have solved the issues with women and underrepresented minorities, but in fact most of the efforts are half-hearted. When minor efforts fail, people cheerfully give up. I don’t think the PIs are lying, exactly – most of them probably agree with what they are saying, in principle. However, they know what the NSF wants to hear and they describe things in those terms, even though their true motivation is rather shallow. Part of the problem is that it’s difficult for a casual observer to distinguish between people who really tried and failed and people who barely tried. I’m sure a few PIs are even outright lying, with no intention of ever making any efforts, but I can’t reliably identify who they might be.

    I imagine the same issue arises with controversial workshops in the social sciences, although this is speculation since I have no direct experience. Presumably everybody promises to make sure that both sides are represented for a constructive dialogue, that extremists don’t get an uncritical platform for their views, etc. Some people try hard to achieve this and fail, while others make only a token effort. If you suspect someone is in the latter category, of course it is galling.

    I agree with you that political meddling in the process would make things much worse, and I don’t think there’s any easy solution (peer review committees do the best they can, and nobody could possibly achieve a perfect record in hindsight). However, I don’t think it’s crazy to wonder whether grant applicants sometimes get away with things that would never have been funded if the committee had known exactly what they had in mind.

  4. That’s a very real issue and thanks for pointing it out. I’ve seen my share of such half-hearted efforts and non-efforts, both in proposals and in my own experience.

    Based on the proposal alone, it’s almost impossible to tell who will make a genuine effort and who is only telling the reviewers what they want to hear. That’s why it’s equally important to also consider the proposer’s past record. Let’s say that we’re evaluating a proposal for a math conference where the organizers promise that at least 20% of speakers will be women (a subject that I know much better than issues with social sciences funding). If I’m asked to review it, I will look up the last few conferences organized by the same people and, should it turn out that the number of female participants was more like 3%, I will want to know what exactly they will do to promote female participation that they didn’t do last time. If there’s no good answer to that, I will recommend either a lower level of funding or no funding, depending on how critical the issue is in terms of meeting the program requirements.

    I would guess that it should work similarly in social sciences. If the organizers promise to have all sides represented but then the actual conference turns out to be an almost uncritical forum for one side, I would expect that the organizers should have to answer a few questions next time they apply for a conference grant. If this happens more than once, there’s a good chance that the organizers aren’t trying that hard and shouldn’t be funded. (This is not a statement about the conference in question. I’ve said already that I’m not qualified to make such judgements.)

    That of course assumes that the reviewers actually care enough about the issue – that they will take the time to look up the organizers’ past record, that they will be willing to raise the issue, and of course that the rest of the committee will listen. I’m not convinced that this can be always taken for granted. Still, I don’t see a better way to do it.

  5. Random blogreader

    My point was not that the Canadian government should be forced to withdraw the money, or that people should be punished for saying controversial things. I wasn’t even really commenting on the Arab-Israeli conflict, just describing what goes on at those things. What I’m saying is that even if they have the right to spout any kind of thing they want in the conference, one should not lose sight on the content of the gathering. One shouldn’t just decide that because it’s legal, and free speech, that there’s nothing more to discuss. If there is a conference denigrating any group in a nonpolitical way (say the anti-Semitic cariciatures passed out to everyone at Durban I) I feel people have the obligation to object, regardless if the denigrators are behaving legally, or even if there is some true injustice being carried out against the people who they belong to. There’s a difference between a political event and a hatefest. I don’t consider this a matter of politics.

    Redact away 😉

  6. Nobody is saying that there’s nothing more to discuss. Individuals or groups who find the conference objectionable can express their opinions in any number of ways, for example by communicating that view to the media as they indeed have done. That’s how democracy works and I don’t see anyone protesting the fact that the public discussion is taking place. The point here is that scientific peer review should be independent of any such discussions. There are procedures in place to that effect and they should be respected in all cases, including the controversial ones.

    “Hatefest” is a strong word. Any conference that turned out to be a hatefest, against whichever group, would quite likely run afoul of the Canadian law. Likewise, any hatefest accusations that turned out to be exaggerated might qualify as libel. So, I’m very glad to see that you only use that word as a general reference term.