Over the last couple of weeks, I have really started to agree with the proponents of the “Peer-Review is Broken” notion. \u00a0Well, perhaps not broken, but sickly and in need of an overhaul.<\/p>\n
A poster here directed me to an article in the Daily Mail (yeah, I know). \u00a0So I looked at it. \u00a0Then I spent some time tracking down the article. \u00a0The article was interesting (and, I admit, problematic to my position on something). \u00a0The particular journal has a hyperlink system for comments, citations, and citing articles (in the same journal presumably).<\/p>\n
Here’s where I run into the problem. \u00a0The comments are also paywalled. \u00a0With some moderate effort, the original articles can often be found on the internet. \u00a0If not, then the authors can be contacted as well. \u00a0But the comments… those are nearly impossible to find.<\/p>\n
In the case of this article, the comments were critical to the reception of the paper. \u00a0Until certain issues raised in the comments are addressed by the authors, then the study is useless. \u00a0It may be right, it may be wrong, but until the issues are resolved, we won’t know.<\/p>\n
But how will we know when the issues are resolved? \u00a0Indeed, if it wasn’t for the journal itself (charging $35 even for the comments), then I wouldn’t know that there were any issues.<\/p>\n
Peer-review, just like anything else, should be taken skeptically. \u00a0Recent incidents (Seralini, for example) show that just because something is peer-reviewed, it’s not automatically correct.<\/p>\n
The problem is that our current system promotes the idea (even if unconsciously) that peer-review is the end. \u00a0I’ve not been critical of this process until recently. \u00a0Between Seralini, other anti-GMO scientists, creationists, and global warming deniers, it’s exceedingly difficult to parse out who is trustworthy.<\/p>\n
What’s worse, it’s becoming increasingly easy for deniers to trot out peer-reviewed research to support their position. \u00a0And non-experts can’t know if those papers are any good or not. \u00a0The Seralini paper was easy to deconstruct. \u00a0Almost anyone with any kind of rational thinking ability could see the gaping flaws in that paper. \u00a0This paper, not so much. \u00a0Unless one is a soil toxicology expert, then the issue would have gone unnoticed. \u00a0I sure would have missed it.<\/p>\n
Open access is a fine idea, but the denier groups are becoming more sophisticated. \u00a0This is especially true when there is financial incentive for a scientist to promote or refute a certain position (I’m looking at you Wakefield).<\/p>\n
A start of a solution would be for journals to allow open access to all the comments and responses to a paper, with the paper itself (not separately). \u00a0I’d say that even if one downloads a PDF of a paper, it should include the comments and responses (all of them). \u00a0There’s been some movement toward a clearinghouse of papers and data. \u00a0I’d like to see the same for responses and comments to papers.<\/p>\n
I fear that we’re beginning to see the backlash of simple information overload. \u00a0In 2009, there were over 200 papers published on Origins of Life research. \u00a0That’s a tiny section of a large field (biochemistry) of a large content area (biology). \u00a0I still haven’t read them all… and they are now almost 5 years old.<\/p>\n
It’s exceedingly difficult for non-experts to stay even partially aware. \u00a0This is only made worse with all the pop-science and denial websites, journals (yes, they have journals… sometimes), and blogs.<\/p>\n
If I wanted to, I could easily get into a massive argument on some minutiae of just about anything. \u00a0While I don’t have the background for it (and I accept that), the deniers also don’t have the background, but don’t accept it. \u00a0The book review of Stephen Meyer has shown us that anyone can talk confidently, even when they have no idea what’s going on (or are purposefully misrepresenting science).<\/p>\n
And confident talk, is more important than facts and evidence in our perception-based culture.<\/p>\n
I would very much like to hear other’s thoughts on this subject. \u00a0It’s been mentioned several times here at SIN, so please leave a comment with your thoughts.<\/p>\n","protected":false},"excerpt":{"rendered":"
Over the last couple of weeks, I have really started to agree with the proponents of the “Peer-Review is Broken”…<\/p>\n","protected":false},"author":27,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[597,89,6],"tags":[565,78,937],"class_list":["post-1603","post","type-post","status-publish","format-standard","hentry","category-featured","category-research-2","category-science","tag-peer-review","tag-research","tag-science"],"_links":{"self":[{"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/posts\/1603","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/users\/27"}],"replies":[{"embeddable":true,"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/comments?post=1603"}],"version-history":[{"count":0,"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/posts\/1603\/revisions"}],"wp:attachment":[{"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/media?parent=1603"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/categories?post=1603"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/skepticink.com\/smilodonsretreat\/wp-json\/wp\/v2\/tags?post=1603"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}