For reasons that I can’t fully articulate, I’ve been frustrated with the quality of publications in many of our supposedly leading journals. Two decades of publishing my own work have taught me that there is huge variability in reviewer quality and one might be very surprised to learn that some of the top-ranked journals in IS, for example, provide the worst examples (by which I mean not rejections but comments that are extremely limited, occasionally incorrect or based on clear oversight of the paper’s true contents, or sometimes wilful ignorance). I am open to the idea that fully open reviewing might help though part of me clings to the ideal that double-blind reviewing is ‘fairer’. I have no doubt that the business of publication has taken on a new urgency as scholars try to establish their research credentials and publishers need to ensure sufficient content but it’s hard to overlook the fact that one does not have to look too hard at the literature to find methodologically or statistically-flawed studies, and that is before we cover the waterfront of trendy, incomplete and work uninformed by prior relevant scholarship that is presented as ‘cutting-edge’.
Seems the problems are not limited to our discipline. Prompted to read a couple of recent papers on the struggles of Computer Science to improve its publications quality, I found this blog entry at the CACM site, reporting that Moshe Vardi, editor in chief of Communication of the ACM feels it’s time for a change too. Apparently his keynote at the Informatics Europe conference called attention to shoddy refereeing practices in Computer Science. Bertrand Meyer. the blogger above, has a detailed strategy that he uses in his reviews to push this cause and it’s hard to object. I realize editors are supposed to maintain reviewing standards but with limited time and a reliance on completely volunteer faculty members to serve as your reviewers, the system as structured can be seen to have built-in problems.