A Blog Around The Clock

About a week ago, my brother sent me a couple of interesting papers about funding in science, one in Canada, the other in the UK. I barely had time to skim the abstracts at the time, but thought I would put it up for discussion online and come back to it later. So I posted the link, abstract and brief commentary a few days ago to the article: Cost of the NSERC Science Grant Peer Review System Exceeds the Cost of Giving Every Qualified Researcher a Baseline Grant:

Abstract: Using Natural Science and Engineering Research Council Canada (NSERC) statistics, we show that the $40,000 (Canadian) cost of preparation for a grant application and rejection by peer review in 2007 exceeded that of giving every qualified investigator a direct baseline discovery grant of $30,000 (average grant). This means the Canadian Federal Government could institute direct grants for 100% of qualified applicants for the same money. We anticipate that the net result would be more and better research since more research would be conducted at the critical idea or discovery stage. Control of quality is assured through university hiring, promotion and tenure proceedings, journal reviews of submitted work, and the patent process, whose collective scrutiny far exceeds that of grant peer review. The greater efficiency in use of grant funds and increased innovation with baseline funding would provide a means of achieving the goals of the recent Canadian Value for Money and Accountability Review. We suggest that developing countries could leapfrog ahead by adopting from the start science grant systems that encourage innovation.

A long and interesting discussion ensued in the comments, with the author of the paper himself showing up and offering to send reprints to those who are interested. More discussion also happened on FriendFeed here and here.
Several other bloggers also posted about it, and discussions happened on their posts as well. T. Ryan Gregory posted about it both on his Nature Network blog Pyrenaemata and on his indy blog Genomicron.
Larry Moran was largely in agreement with the article, but some commenters were not, including Rosie Redfield whose comment motivated T. Ryan Gregory to post again, just to explain his disagreement with Rosie.
Jonathan Eisen pointed out to a related post of his and Cameron Neylon to a related post of his. Finally, Zen Faulkes used it as a starting point for three posts here,
here and here.
I have finally managed to find time to read the paper myself so I think I can say something semi-intelligent about it. It became obvious that many who commented have not actually read the paper, just the Abstract, and thus were not in the position to respond to it intelligently (the paper actually answers, clearly and in detail, all the questions and complaints voiced by the commenters). The abstract is just, …well, an abstract. The paper is full of thought-provoking ideas and really needs to be read in its entirety.
Finally, my brother showed up in the comments and I would like to use his comment as a starting point today. That is – once you read the actual paper (ask for a reprint if you cannot access it), the linked blog posts and comment threads. I’ll be right here, waiting for you to come back….
I am assuming that the Canadian funding system is not very dissimilar to that in many other countries, including the USA – there is a central governmental body that gets its budget from the government and uses committees of unpaid peer-reviewers to decide how the money will be allocated to the researchers. The paper explains in detail at least a dozen reasons why and how this system is flawed: how it stifles truly innovative science, repels students from entering science, disproportinately pushes women out of science, encumbers students and postdocs with tasks they are not supposed to be doing (e.g., clerical, or technical), introduces an element of uncertainty about one’s livelihood, gives universities excuses to completely get out of research funding, shafts teaching and outreach as criteria for promotion, etc. But the clincher, for politicos at least, is that this system costs more than if a set sum of about $30,000 (Canadian) was given to every academic employed by a Canadian university who asks in any given year.
Yes, giving every Canadian scientist who already has a job and a lab this small amount of money no-questions-asked, geared toward innovative exploratory research, costs the government less than going through the peer-review system that gives money to some and no money to others (not to mention the reinforcement of the Old Boys Club this way).
This does not mean, in their proposal, that all of the Canadian money earmarked for science would be given this way – this is still just a small part of it. If you have a big lab or do expensive research and need to apply for much bigger grants, that would be done by the traditional peer review. But in order to get to the point where you have a good proposal, you need to have some neat stuff done (the “preliminary data”). With the proposed system, that preliminary data can be really exciting or revolutionary, something that, as an initial proposal, would never fly by peers.
Would people send out proposals for crap? Some would, I’m sure, but that doesn’t matter. Most would not. Scientists are curious about nature and would like to test their hunches. Some will flop, some will be amazing – it is the latter that this new system is worth doing for, as they may never be done otherwise. Anyway, how many $5,000,000 grants produced amazing stuff? All? I.Don’t. Think.So.
Where does quality control come from? First, it already came from universities who hired these researchers out of hundreds of applicants for each position. Aren’t they going to trust those best-of-the-best they hired? Second, the research itself will be judged after it is done – at conferences, in journal articles, and in post-publication metrics (citations, downloads, online chatter, etc., including perhaps a Nobel Prize here and there). If it is not up to standards, $30,000 Canadian dollars is not a big price to pay, and even the negative or inconclusive results can be useful to others if the thinking is original. If it is up to standards or more, that person will now have something exciting to base a bigger grant proposal on.
This also goes back to something I like to rant about (oh yes, go read that again!) – the bandwagon of Big Science. Biology, for example, does not equal running gels (hmmm, that’s chemistry, isn’t it?). But many people are given that appearance. “No gels – no grants, no papers, no career” (yup, I was told that a few years ago). Unless you already have a big molecular lab, this small grant will not build you one. Instead, you can do some really cool stuff at other levels – from tissues up to ecosystems and everything in-between, including computer modeling. You can use it to travel to some jungle that has never seen a Westerner and see what species live there – not hypothesis-testing, exploratory and exciting, definitely useful, but not something that is easily funded with a current system. If your proposal includes research on live vertebrates, you first have to get an IACUC proposal, something that will take 6-9 months of extremely frustrating fighting and proposal-modification – getting an IACUC proposal is the toughest peer-review known to science: if they say Yes to your proposal, no other committee of peers can add any more wisdom to it. And if you decide to work on invertebrates – it is much cheaper.
Another paper looks at this from another perspective – four stages of science. The grants, especially the big ones, disproportionately target science in Stage 3. The small baseline grants would target primarily Stage 1, the exciting, innovative stage – and this is a Good Thing. They could also more easily fund research in Stage 2 and Stage 4, also a Good Thing – from the article:

In this article I propose the classification of the evolutionary stages that a scientific discipline evolves through and the type of scientists that are the most productive at each stage. I believe that each scientific discipline evolves sequentially through four stages. Scientists at stage one introduce new objects and phenomena as subject matter for a new scientific discipline. To do this they have to introduce a new language adequately describing the subject matter. At stage two, scientists develop a toolbox of methods and techniques for the new discipline. Owing to this advancement in methodology, the spectrum of objects and phenomena that fall into the realm of the new science are further understood at this stage. Most of the specific knowledge is generated at the third stage, at which the highest number of original research publications is generated. The majority of third-stage investigation is based on the initial application of new research methods to objects and/or phenomena. The purpose of the fourth stage is to maintain and pass on scientific knowledge generated during the first three stages. Groundbreaking new discoveries are not made at this stage. However, new ways to present scientific information are generated, and crucial revisions are often made of the role of the discipline within the constantly evolving scientific environment. The very nature of each stage determines the optimal psychological type and modus operandi of the scientist operating within it. Thus, it is not only the talent and devotion of scientists that determines whether they are capable of contributing substantially but, rather, whether they have the ‘right type’ of talent for the chosen scientific discipline at that time. Understanding the four different evolutionary stages of a scientific discipline might be instrumental for many scientists in optimizing their career path, in addition to being useful in assembling scientific teams, precluding conflicts and maximizing productivity. The proposed model of scientific evolution might also be instrumental for society in organizing and managing the scientific process. No public policy aimed at stimulating the scientific process can be equally beneficial for all four stages. Attempts to apply the same criteria to scientists working on scientific disciplines at different stages of their scientific evolution would be stimulating for one and detrimental for another. In addition, researchers operating at a certain stage of scientific evolution might not possess the mindset adequate to evaluate and stimulate a discipline that is at a different evolutionary stage. This could be the reason for suboptimal implementation of otherwise well-conceived scientific policies.

Now, the proposal in this paper is quite definitive about allowing only researchers employed by universities to apply for such grants. But my mind instantly started thinking about those outside. How about amateur scientists? How about people not affiliated with the academia? How about distributed citizen science projects? Those are usually Stage 1 or Stage 2 projects, attractive to a particular kind of researcher (myself included – don’t try to lure me into a big Stage 3 lab). If I wanted to get some crayfish or spiders (or even birds, if a local IACUC would let me) and do experiments at home, this kind of a small grant would be just ideal. Could I have a local University, or some peers, write a letter in support of my proposal? Would that fly?
The paper also mentions, in a couple of places, similarities and differences between peer-review of grants and peer-review of manuscripts, including the importance of Openness to science. In one place, it mentions new journals “where ideas may be published initially unreviewed, but anyone may append public discussions to each article”. I am hoping this refers to arXiv and Nature Precedings, or even the concept of Open Notebook Science, but it smells too much like one of the pernicious myths spread by the enemies of Open Access about PLoS ONE which is, as readers of this blog are aware, stringently peer-reviewed.
One thing that the article mentions is that the current granting system allows researchers to buy time for research away from their teaching time. They note this as bad for teaching, true, but there is another angle to it. As danah writes in regard to the new proposed NSF funding of qualitative research, this kind of work does not require much in terms of equipment, but much in terms of time. It is essential for people, especially in social sciences, who do qualitative research, to be able to buy the time they need to do their research correctly.
Oh, and I mentioned at the beginning that my brother sent me two papers, yet we talked here only about one of them. The other one, if you are interested in starting a whole new discussion, is this one: Life after death? The Soviet system in British higher education

Recent studies of British higher education (HE) have focused on the application of the principles of the ‘new managerialism’ in the public sector, ostensibly aimed at improving the effectiveness of research and teaching, and also on the increasing commercialisation of HE. This article examines HE management in the light of the historical experience of the Soviet system of economic planning. Analogies with the dysfunctional effects of the Soviet system are elaborated with regard to financial planning and the systems of quality control in academic research and teaching. It is argued that Soviet-style management systems have paradoxically accompanied the growing market orientation of HE, undermining traditional professional values and alternative models of engagement between HE institutions and the wider society.

A FriendFeed discussion has started. Read the entire paper before chiming in, of course – we are scientists here!
Gordon, R., & Poulin, B. (2009). Cost of the NSERC Science Grant Peer Review System Exceeds the Cost of Giving Every Qualified Researcher a Baseline Grant Accountability in Research, 16 (1), 13-40 DOI: 10.1080/08989620802689821