Tag: peer review

One of the joys and challenges of being an academic is being able to help to shape the future of scholarship through the peer review process. Much has been written about the issues with academic peer review, most notably the limited incentives to spend time reviewing submissions and the increasing length of time between when an academic submits a paper to a journal and when they finally receive feedback. Heck, I wrote about this issue five years ago when The Review of Higher Education stopped accepting new submissions for about a year and a half due to this imbalance.

Throughout my ten years as a tenure-line faculty member, what I give to and take from the peer review system has changed considerably. When I was first starting on the tenure track, I was reliant on relatively quick reviews on my own submissions and was receiving 5-10 requests to review each year from legitimate journals. And since I keep a spreadsheet of the details of each journal submission, I can see that I received decisions on many articles within 2-4 months. I have never missed a deadline—typically around 30 days—to submit my thoughts as a reviewer, and have tried to accept as many requests as possible.

The peer review system changed considerably in the late 2010s. As I got closer to tenure, I received more requests to review (25-30 legitimate requests per year) and accepted them all because I was in a position to do so. Decisions on my article submissions moved more toward the 4-6 month range, which was frustrating but not a big deal for me because I figured that I had already met the standards for tenure and promotion. My philosophy at that point became to be a giver to the field because of the privileged position that I was in. I needed to review at least 2-3 times as many submissions as I submitted myself to account for multiple reviewers and so grad students and brand-new faculty did not need to review.

Going through the tenure and promotion process exposed me to another crucial kind of reviewing: external reviews of tenure applications. Most research-focused universities expect somewhere between three and eight external letters speaking to the quality of an applicant’s scholarship. I am grateful to the anonymous reviewers who accepted my department chair’s invitation to write, and now a part of my job most years as a department head is soliciting letters from some of the most accomplished (and busiest) scholars in the world.

All of this is to say that being a full professor in a field that loses a lot of full professors to full-time administrative positions (the joy of specializing in higher education!) means that my priorities for external service have changed. I am focusing my reviewing time and energy in two areas that are particularly well suited for full professors at the expense of accepting the majority of journal review requests that I receive.

The first is that I just started as an associate editor at Research in Higher Education and am thrilled to join a great leadership team after being on the editorial board for several years. I took this position because I am a big fan of the journal and I believe that we can work to improve the author experience in two key areas: keeping authors updated on the status of their submissions and quickly desk rejecting manuscripts that are outside of the scope of the journal. Researchers, please send us your best higher education manuscripts. And reviewers, please say yes if at all possible.

The second is to continue trying to accept as many requests as possible for reviewing faculty members for tenure and/or promotion. I am doing 6-8 reviews per year at this point, and it is a sizable task to review tenure packets and relevant departmental, college, and university standards. But as a department head, I am used to doing faculty evaluations and rather enjoy reading through different bylaws. It is an incredible honor to review great faculty from around the country, and it is a job that I take seriously. (Plus, as someone who solicits letters from colleagues, a little karma never hurts!)

As I prepare to enter my second decade as a faculty member, I wanted to share my thoughts about how my role has changed and will continue to change. My apologies to my fellow associate editors and editors at other journals (I will complete my term on the editorial board at The Review of Higher Education and continue to be active there), but I will say no to many of you where I would have gladly accepted a few years ago. I hope you all understand as I rebalance my scholarly portfolio to try to help the field as much as possible.

Like most research-intensive faculty members, I receive regular requests to review papers for legitimate scholarly journals. (My spam e-mail folder is also full of requests to join editorial boards for phony journals, but that’s another topic for another day.) Earlier this week, I was working on reviewing a paper submitted to The Review of Higher Education, one of the three main higher education field journals in the United States (Journal of Higher Education and Research in Higher Education are the other two). I went to check one of the submission guidelines on the journal’s website and was surprised to see that the journal is temporarily closed for new manuscript submissions to help clear a backlog of submissions.

After I shared news of the journal’s decision on Twitter, I received a response from one of the associate editors of the journal. Her statement astonished me:

This sets off all kinds of alarms. How can a well-respected journal struggle so much to get qualified reviewers, pushing the length of the initial peer review process to six months or beyond? As someone who both submits to and reviews for a wide range of journals, here are some of my thoughts on how to potentially streamline the academic peer review process.

(1) Editors should ‘desk reject’ a higher percentage of submissions. Since it can be difficult to find qualified reviewers and most respectable journals accept less than 20% of all submissions, there is no reason to send all papers out to multiple external reviewers. If a member of the editorial board glances through the paper and can easily determine that it has a very low chance of publication, the paper should be immediately ‘desk rejected’ and quickly returned to the author with a brief note about why it was not sent out for full review. Journals in some fields, such as economics, already do this and it is sorely needed in education to help manage workloads. It is also humane to authors, as they are not waiting several months to hear back on a paper that will end up being rejected. I have been desk rejected several times during my career, and it allowed me to keep moving papers through the publication pipeline as a tenure-track faculty member.

(2) Journals should consider rejecting submissions from serial free riders. The typical academic paper is reviewed by two or three external scholars in the peer review process, with more people potentially getting involved if the paper goes through multiple revise and resubmit rounds. This means that for every sole-authored paper that someone submits, that person should be prepared to review two or three other papers in order to maintain balance. But in practice, since journals prefer reviewers with doctoral degrees and graduate students need to submit papers in order to be eligible for academic jobs, those of us with doctoral degrees should probably plan on reviewing 3-4 papers for each sole-authored paper we submit. (Divide that number accordingly based on the number of co-authors on your submissions.) It’s okay to decline review invitations if the paper is outside your scope of knowledge, but otherwise scholars need to accept most invitations. Declining because we are too busy doing our own research—and thus further jamming the publication pipeline—is not acceptable, particularly for tenured faculty members. If journals publicly commit to rejecting submissions from serial free riders, there may be fewer difficulties finding reviewers.

(3) There needs to be some incentive for reviewers to submit in a timely manner. Right now, journals can only beg and plead to get reviewers to submit their reviews within a reasonable time period (usually 3-6 weeks). But in my conversations with journal editors, reviewers often fail to meet that timeline. In an ideal world, journal reviewers would actually get paid for their work like many foundations and scholarly presses pay a few hundred dollars for thorough reviews. Absent that incentive, it may be worth establishing some sort of priority structure that rewards those who review quickly with quick reviews on their own submissions.

(4) In some cases, there needs to be better vetting of reviews before they are sent to authors. Most reputable academic journals have relatively few problems with this, as this is the job of the editorial board. Reviews generally come with a letter from the editor explaining discrepancies among reviewers and which comments can potentially be ignored. But the peer review process at academic conferences has more quality control issues, potentially due to the large number of reviews that are requested (ten 2,000-2,500 word proposals is not uncommon). It seems like reviewers rush through these proposals and often lack knowledge in the subject matter. Limiting the number of submissions that any individual can make and carefully vetting conference reviewers could help with this concern.

By helping to restrict the number of items that go out for peer review and providing incentives for people to fulfill their professional reviewing obligations, it should be possible to bring the peer review timeline down to a more humane 2-3 months rather than the 4-8 months that seems to be the norm in much of education. This is crucial for junior scholars trying to meet tenure requirements, but it will also help get peer-reviewed research out to the public and policymakers more quickly. Journals such as AERA Open, Educational Evaluation and Policy Analysis, and Economics of Education Review are models in quick and thorough peer review processes that the rest of the field can emulate.