I was waiting until the last installment was up to post about this. Revere on Effect Measure took a recent paper about a mathematical model of the spread of anti-viral resistance and wrote a 16-part series leading the readers through the entire paper, from the title to the List of References and everything in between. While the posts are unlikely to garner many comments, this series will remain online as a valuable resource, something one can use to learn – or teach others – how a scientific paper is to be analyzed.
As you can see, it takes a lot of time to read a paper thoroughly. It also requires some background on the topic of the paper. A journalist on deadline is unlikely to have either the time or the necessary background to be able to read a paper in this manner before writing an article. And that is just one paper per week.
Scientists themselves rarely read all the papers as thoroughly as this. If you, like I do, go through dozens of papers per week, you find your own method of cutting down the necessary time. You skim through the abstract, figures and figure legends, perhaps some of the Discussion and – this is it. You make a mental note what the paper is about and move on. But that is reading for one’s own information only. It is not the way to read a paper one is to comment on – or write an article about. For that, one has to do it throughly, like Revere did.
If a paper is in my narrow field, or a field I am very familiar with, the first place I look is the list of references. This tells me from what tradition the paper comes from, what group of people, what mindset, what research goals and questions. That is, actually, already a LOT of information about the paper. Then I read the abstract, look at the figures and figure legends and, if necessary, scan the text of the Results section to find relevant passages connected to the figure I am interested in. Then I dig deep through the Materials and Methods because that is where flaws, if any, will be discovered. Introduction can usually be skipped – that is mainly for the readers outside of the narrow field. Then I read the Discussion carefully in the very end, by which time I already have a very good idea what the data really say so I can spot if the authors overreach in their conclusions.
As a science blogger, I would not want to write a post about a paper I have not read as throughly as that. I may post a link to it and let you evaluate it for yourself, or point out if some other blogger wrote a good review, but I would not go into a critique of my own if my familiarity with the paper was only superficial, or if it is in a field I do not have a good background in – thus, no reviews of physics papers here!
As this process takes a lot of time and effort, it is not surprising that science bloggers do not post such in-depth reviews very often. I may do one a week if Real Life allows. It is easier, quicker and gets more comments and traffic to write posts that do not require as much expertise and as much time and effort.
But doing it once in a while is still worth the effort. See this latest post on Pharyngula. It is stunning, beautiful, exciting! Yet, this was probably the post that took PZ most time and work to write of all of his many posts this week. And it is likely to get less comments, links and traffic than any of the other posts. But, unlike the commentary about current issues or the daily anti-religion screed (which are all eloquent and lovely and useful and have to be done), this post will not dissappear into the depths of his archives forever. It will remain online (and likely high on Google searches) as a resource that will be linked again and again, for years to come, by other bloggers as well as people who want to use it when teaching biology in the real-world classrooms. The same goes for Revere’s series, or for that matter every serious science post that goes into detail of an area or a single paper and explains it (and perhaps criticizes it) in plain language. There has to be room for all kinds of science blog-posts, each serving a different purpose.
So, bookmark Revere’s series, read it, and save it somewhere handy for future reference.