Much ink has already been spilled about Strumia (2021). Shortly after QSS agreed to publish the article, Cassidy Sugimoto—current ISSI president and QSS editorial board member—told Science the article was “methodologically flawed,” contained “several unsubstantiated claims,” and “fails to meet the standards of the bibliometric community” (Chawla, 2019). Following widespread criticism, QSS published four rejoinders—including from two additional editorial board members—identifying substantial empirical flaws and unfounded claims in Strumia (2021) (Andersen et al., 2021; Ball et al., 2021; Hossenfelder, 2021; Thelwall, 2021). This letter is not intended to further relitigate Strumia (2021). Instead, it focuses on subsequent editorial statements by QSS editor-in-chief Ludo Waltman and MIT Press that reveal discomfiting philosophies about academic gatekeeping, community building, and scientific communication.
Waltman (2021) offers vacillating justifications for publishing Strumia (2021). Waltman engages in hedging and backside-covering by vaguely claiming that he and the reviewers did not necessarily agree with all of the article, yet still somehow believed the article was meritorious enough to be published in QSS. This approach is reminiscent of the editors of the postmodernist journal Social Text responding to publishing the infamous Sokal hoax with a mixture of criticism and defense of the article, and a naïve hope that the ensuing debate about the piece would be productive (Robbins & Ross, 1996). Waltman also leaves unanswered exactly what he found meritorious about Strumia (2021), which is an important question, as the empirics, theory and implications of the article were all widely criticized. Waltman’s statement is a motte-and-bailey1, as Waltman can claim that he and QSS are merely engaging with “a diversity of perspectives” (motte), while at the same time legitimating and propagating Strumia’s views (bailey). This is a particularly bad omen for QSS on gender and science issues, given that a precedent of questionable quality control has now been established. Scholars working on gender issues in science may now rightfully question what the editor-in-chief and some of the QSS reviewer pool believes—or at least tolerates—when considering submission outlets.
Waltman’s judgment should also be questioned regarding his choice of peer reviewers. It is disappointing (and telling) that the peer reviewers Waltman selected refused to anonymously release their peer review reports. If the QSS peer reviewers actually felt strongly about the merits of their reviews and Strumia (2021), it is cowardly and unproductive of them to deny the anonymous publication of reports. This refusal is also contrary to the spirit of QSS, which has since venerably implemented a pilot program of online sharing of peer reviews. Following a controversial 2020 Nature Communications article about gender and academic mentorship (later retracted by the authors), releasing peer review reports revealed reservations of peer reviewers (Wessel, 2020), holding editors accountable for publication decisions and enabling stakeholders to identify where mistakes may have occurred in the gatekeeping process.
There is a reasonable argument that it is best not to legitimate bad articles with attention. However, as QSS legitimated and published Strumia (2021), it is perhaps not possible to ignore the article. Even retracted articles still get cited (Schneider et al., 2020). The institutionalized legitimation of an article by an academic journal is socially powerful. Thus, journal gatekeeping is an important responsibility, even if postpublication debate is possible. Publishing questionable articles based on the notion that they can be debated publicly in the future risks devaluing the imprimatur of an academic journal.
Future citation and altmetric analyses of Strumia (2021) will be revealing. Will the article be seriously used by reputable academics conducting solid research on gender issues, or will it find favor with men’s rights activists, gender essentialists, and/or alt-right outlets? I believe the latter possibility is more likely and entails irresponsible intellectual and social contributions by QSS. Publishing Strumia (2021) also placed demands on scholars who spent considerable time and effort rebutting a bad—if not also damaging—article. I would rather scholars spend their time innovating and creating new science, as opposed to having to rebut bad articles2.
MIT Press appended a statement to Waltman’s editorial piece. I was also disappointed with this response. MIT Press stated by publishing Strumia (2021) with critical rejoinders, they “welcome the opportunity to publish these wide-ranging views,” hoping “this approach sheds more light than heat on gender bias in the sciences.”3 This response is either naïve or disingenuous. Strumia has an infamous track record with his public conduct and statements about gender and science, which precipitated his removal from European particle physics research center CERN in 2019. Thousands of physicists signed a petition condemning Strumia’s work and ideas (Banks, 2018). It is unlikely that someone with this history will generate “more light than heat,” especially based on a widely panned QSS article. What light can be found in publishing and legitimating a bellicose ideologue pushing ideas that women are inferior at physics, and physics was “invented and built by men”?
As a general principle, “wide-ranging views” are desirable. Academia tends to skew liberal ideologically (Gross, 2013), which can be an intellectual and sociopolitical liability. However, controversial perspectives require careful judgment, and can entail dangerous slippery slopes. There are many “wide-ranging views” on topics such as the virtues of African colonialism (Gilley, 2018), the competency of LGBT parents (Regnerus, 2012), eugenics and the intellectual inferiority of people in developing countries (Lynn & Vanhanen, 2006), and the public health and economic benefits of permitting COVID-19 spread to achieve herd immunity (Kulldorff, Gupta, & Bhattacharya, 2020). That does not mean those “wide-ranging views” should be promoted in any reputable academic journal! To analogize based on the title of Waltman’s editorial response, understanding astronomy does not require the “diverse” and “controversial” perspectives of Flat Earthers.
Science is a human endeavor. Peer review is flawed (Smith, 2006). None of us—whether individuals or institutions—are perfect. Given the overwhelmingly negative response to Strumia (2021), I am astounded that neither Waltman nor MIT Press publicly acknowledged the mere possibility that mistakes or shortcomings may have occurred in peer review and gatekeeping at QSS. The stubbornness—if not arrogance—in Waltman’s statement is disappointing. Mistakes are inevitable, but failing to acknowledge and learn from them means they will likely be repeated.
A 2019 BBC article chronicling Strumia’s removal from CERN noted, “[Strumia’s] analysis was quickly dismissed as being unscientific and incorrect by experts in the impact of research. But Professor Strumia told BBC News that his analysis would eventually be proved correct if scientific journals would allow him to publish his research” (Ghosh, 2019)4. Since QSS allowed Strumia to publish his research, this bestows him and his ideas with a veneer of academic legitimacy. The internet is vast; “wide-ranging” and controversial views—including Strumia’s—are omnipresent and easily accessed. QSS does not have to endorse questionable research and ideas to enable “controversial” debate.
Having published five issues in just over 1 year, QSS is still a new, developing institution. Nascent periods are especially important for identity development and survival prospects of new institutions (Aldrich & Fiol, 1994). QSS is not currently indexed in Scopus or the Web of Science, nor does it have an impact factor. The relative place of QSS in the hierarchy of similar journals is uncertain5. There are many good people affiliated with QSS, but a successful launch is not assured. Publishing a widely panned article is bad enough, but it is especially perilous when a journal is still making influential first impressions. Growth and development are path dependent. Early missteps can inflict exponential long-term damage.
Hartley et al. (2019) characterized academic journals as clubs. Institutional networks and affiliations are powerful social signals. Now that QSS includes Strumia, and some questionable values and leadership, will the journal be a desirable club to patronize? It is easy to play games with “controversial” ideas when one’s social location inures one from negative consequences of legitimating and propagating those ideas. However, I would argue that Waltman harmed his reputation as a gatekeeper, as well as QSS more broadly with his decisions and responses regarding Strumia (2021).
COMPETING INTERESTS
The author has no competing interests.
FUNDING INFORMATION
Funding was received from the Alfred P. Sloan Foundation (grant G-2020-12678).
Notes
The motte-and-bailey is an informal fallacy coined by philosopher Nicholas Shackel (2005). The arguer advances two positions: one modest (motte) and one controversial (bailey). After facing criticism for the controversial claim, the arguer retreats to the modest position. The bailey goes unrefuted and the motte can be defended by the arguer claiming critics are unreasonable for equating the motte with the bailey.
I would rather be doing other things than writing this letter.
No names were appended to the MIT Press statement, indicative of a lack of accountability. Presumably, specific humans conceived and wrote the statement, as opposed to the entirety of a large, anonymous institution.
Strumia’s apparent frustration with journal editors not “allowing” him to publish his research seems to suggest that before he published in QSS, more prudent gatekeepers may have rejected his work.
Whether you like academic rankings or not, they influence journal reputations and are often directly linked to the hiring, tenure, and promotion reward structures of universities. Higher rankings (and merely being ranked at all) incentivize academics to submit better articles to journals. For example, in the Chartered Association of Business Schools (2018) Academic Journal Guide, proximate journals to QSS have a variety of rankings, ranging from 1 (lowest) to 4* (highest): Research Policy (4*), JASIST (3), Scientometrics (2), Social Studies of Science (2), Journal of Informetrics (1). Where—if at all—will QSS be ranked relative to those journals in the next edition? Even outside formal rankings, journal reputations and hierarchies matter, influencing where articles are—and are not—submitted.
REFERENCES
Author notes
Handling Editor: Staša Milojević