Impact of peer review type and editor gender on women authorships in mathematics

https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00318

Recent years have seen a growing number of studies attempting to identify factors that contribute to—or help explain—a persistent underrepresentation of women in academic publications and other critical outputs (e.g., grants, patents) in scientific disciplines including engineering and medicine, for example Belz, Graddy-Reed et al. (2022), Helmer, Schottdorf et al. (2017), Royal Society of Chemistry (2019), and Severin, Martins et al. (2020).

Aspects of the peer review process—including its actors (e.g., editors, reviewers), structure (e.g., peer review type), and outcomes (e.g., acceptance rates, reviewer scores, or feedback) are increasingly studied to help mitigate biases in general and better understand representation gaps, including the gender gap in women authorships in scientific publications (Guillopé & Roy, 2020).

In mathematics, a field where men editorships are overrepresented and Single Anonymized Peer Reviews (SAPR—reviewer identity concealed from author) are still the norm (Palus, 2015), persistent gaps in women authorships (Mihaljević-Brandt, Santamaría, & Tullney, 2016), and women editorships (Topaz & Sen, 2016) have been well documented but remain underinvestigated and insufficiently explained.

We aim to lessen this gap by tackling what is possibly the first study on the impact of editor gender and peer review policies on women’s publication output in mathematics. More specifically, we study changes in authorships straddling the year 2015, which marked a change in both peer review type and editor gender (man editor and SAPR to woman editor and Double Anonymized Peer Reviews, DAPR—author identity also concealed from reviewer) in the American Mathematical Monthly (henceforth, the Monthly) one of the oldest mathematics journals in the United States.

To understand whether or not changes in the Monthly could be explained by underlying trends in women authorships not particular to the journal, we compare Monthly authorship changes with parallel changes in three other journals similar to the Monthly in focus, scope, rankings, and reach (see Section 6), yet contrasting with the Monthly as far as editor gender/review type. All comparison journals maintained SAPR over 2011–2019, and all had men editors throughout this period. The Monthly had men editors and utilized SAPR up until 2015; it elected its first-ever woman editor and transitioned to DAPR in 2015 (Chapman, 2015; Nagy, 2015).

Our results reveal a significant increase in women authorships in the Monthly after 2015, the year of the editor gender/review type switch. Our results also suggest this increase cannot be explained by an underlying growth in women authorships in the field (as represented by the systematically selected group of comparison journals).

Additionally, our analysis suggests that the Monthly changes did not impact the share of newcomer authors to the Monthly (i.e., first-time authors to this journal, though not necessarily novices to mathematics publishing more generally), and that men abbreviate first names more often than women, counter to the hypothesis that women use initials to conceal the (binary) gender implied by given names.

Our study explores the impact of peer review factors known to positively contribute to women representation in science (DAPR and women editorships) in increasing the share of women authorships in mathematics, a field where SAPR is still the norm and where women are significantly underrepresented in editorship roles (Topaz & Sen, 2016). To frame our analysis, we discuss measured gender gaps in mathematics, related STEM fields, and medicine. We then segue to studies that explore (a) editors’ and reviewers’ (binary) gender, and (b) peer review practices as a factor that contributes to explaining measured gaps, or mitigating biases in science.

In 2016, two large data-backed studies offered, for the first time, global-scale analyses of gender representation in mathematics publications (Mihaljević-Brandt et al., 2016) and journal editorships (Topaz & Sen, 2016). Using large data sets and a combination of automated and manual methods to infer author/editor (binary) gender, both studies quantified the extent to which women remain underrepresented as mathematics authors and editors. Mihaljević-Brandt et al. (2016) showed that, in spite of improvements over time, women’s publication output falls consistently behind men’s, with larger gender gaps in top-ranked mathematics journals and pure fields, for example.

Topaz’s and Sen’s (2016) analyses of global trends unveiled an even wider gender representation gap in editorships, as well as examples of countries where gender parity (meaning the proportion of women’s editorships is at least as high as men’s) exists. Their work also found a significant positive association between a journal’s impact factor (a measure of the citation frequency of a journal’s publications) and the share of its editorships held by women.

Such data-based studies are useful and important for documenting global trends, change over time, and field-specific differences in gender representation in mathematics. Yet, investigating possible explanatory factors for observed gender gaps in mathematics, as this study aims to do, and considering actions that might mitigate potential inequities, is also critical.

Outside mathematics a number of studies have investigated women’s underrepresentation in scientific publications (Guillopé & Roy, 2020) and other academic and professional realms considered proxies for career success (and critical for career advancement). Thomas, Bamini et al. (2019) for example, utilized a matched case-control study design to show that women are significantly underrepresented as authors of invited commentaries in medicine despite expertise, productivity, and seniority commensurate with the men who predominantly author them. Belz et al. (2022) showed that women are significantly underrepresented in scientific grant programs aimed to increase women’s participation, and that women score more poorly than men even when controlling for proposal quality, for example.

Other large, recent studies with access to peer review scores, have reported findings similar to Belz et al. (2022). Severin et al.’s (2020) analysis of over 38,000 peer reviews submitted to the Swiss National Science Foundation (SNSF) found that women were underrepresented as principal applicants in all areas and received lower scores than men overall, with differences in representation and scores widening in mathematics, physics, engineering, biology, and medicine. Fox and Paine’s (2019) analysis of six journals in ecology and evolution also found that women garnered lower peer review scores than men overall and were invited less often than men to revise or resubmit manuscripts, for example.

Several large studies with data on the identity of key actors in the peer review process have further nuanced our understanding of the role the (binary) gender of editors and reviewers may play in explaining aspects and outcomes of peer review in scientific disciplines. Helmer et al.’s (2017) analysis of Open Peer Review (OPR) data on the identity of over 9,000 editors, 43,000 reviewers, and 126,000 authors across 142 Frontiers journals spanning multiple science disciplines, found homophily (same-gender preference) to be widespread among men editors (men editors select men reviewers), but only sparsely yet strongly present in very few cases among women editors. These homophilic trends imply (Helmer et al., 2017) that women are more severely underrepresented at various levels of the peer review process than their numerical underrepresentation would warrant. Differential homophilic tendencies between men and women editors (Helmer et al., 2017) could result in substantially more women reviewers selected by women editors and substantially fewer women reviewers selected by men editors.

Severin et al.’s (2020) study of a large data set of SNSF proposals also spanning multiple disciplines, including mathematics, found evidence of an interaction between reviewer and applicant gender consistent with the homophilic tendencies noted in Helmer et al. (2017) at the editor/reviewer level. More specifically, while men reviewers tended to score all applicants higher than women reviewers, men reviewers—unlike women reviewers—rated men applicants systematically higher than women applicants. Squazzoni, Bravo et al.’s (2021a) study of peer review process and outcomes across 145 journals, which found no evidence that women-authored manuscripts are penalized by the peer review/editorial processes, also found that women-authored manuscripts were more likely to be reviewed by women across all fields. Squazzoni et al.’s (2021a) study included 50 physical/life sciences journals, including computer science and engineering; it is not stated whether or not it included mathematics. Their study did not consider editor gender or its potential impact on the selection of reviewers or other editorial decisions.

Fox, Burns, and Meyer’s (2016) analysis of editor and reviewer gender impact over 10 years in a single ecology journal found no differences in how women/men reviewers and editors assessed papers, concluding that editor and reviewer gender did not affect peer review outcomes. Yet, similar to other recent reports (Helmer et al., 2017; Severin et al., 2020), Fox et al. (2016) did find that the share of women selected to review differed substantially with editor gender and widened with editor seniority, with a growing proportion of women editors driving the increase in women reviewers.

Other large, data-based studies have linked women editorships to larger numbers of women reviewers (Royal Society of Chemistry, 2019) as well as increases in women authorships in academic publications, and have recommended increasing the share of women editorships as a gender gap reducing mechanism. In fact, greater diversity (including, but not limited to gender) among reviewers and editors has been proposed as a more effective bias-reducing measure than fully open or partially anonymized peer review practices (Kiermer & Mudditt, 2021).

Double anonymized peer reviews (DAPR) are rare in mathematics and the natural sciences. Single anonymized peer reviews (SAPR) remain the norm, and a host of practical and theoretical reasons have been offered to justify them. They include the tightness of specialized research communities (e.g., anonymizing is ineffective when reviewers and authors all know each other), ease of guessing authors’ identities (e.g., authors names listed at online preprint repositories (Lamb, 2018); ineffective anonymizing (Hill & Provost, 2003; Katz, Proto, & Olmsted, 2002)), mixed evidence on the impact of DAPR for different populations (e.g., newcomers to a field (Seeber & Bacchelli, 2017); women (Blank, 1991; Borzuk, Aarssen et al., 2009; Budden, Tregenza et al., 2008a); international authors (Kalavar, Watane et al., 2022)); venues (e.g., conferences (Tomkins, Zhang, & Heavlin, 2017a); grants (NASA, 2020); publications (McGillivray & De Ranieri, 2018)), and/or disciplines (e.g., ecology (Budden et al., 2008a); computer science (Seeber & Bacchelli, 2017; Tomkins, Zhang, & Heavlin, 2017b); medicine (Okike, Hug et al., 2016)).

While DAPR and sometimes triple anonymized reviews (authors’ and reviewers’ names are not known to the editor) are the norm in certain disciplines (Palus, 2015), discussions on using DAPR in mathematics have been mostly theoretical in nature (Lamb, 2018). At the time of writing, only some journals, focused on mathematics education (e.g., Journal of Urban Mathematics Education) and journals published by the Mathematical Association of America—which include the Monthly yet are generally focused on specialized purposes or audiences (e.g., Mathematics Magazine, targeting undergraduate readerships; The College Mathematics Journal, aimed at early college instructors)—require DAPR. Literature searches suggest that some mathematics journals based outside the United States and Europe also utilize DAPR. By and large, however, peer review in mathematics means SAPR.

Broadly speaking, and despite mixed empirical and experimental evidence, arguments favoring DAPR recognize the pervasiveness of implicit biases in all human endeavors, including peer review. Such arguments are motivated by evidence that bias can impact review processes (e.g., Belz et al., 2022; Fox et al., 2016; Helmer et al., 2017; Severin et al., 2020) and outcomes (e.g., Fox & Paine, 2019; Huber, Inoua et al., 2022; Seeber & Bacchelli, 2017) and a growing commitment to equity and best practices (e.g., Mainguy, Motamedi, & Mietchen, 2005; Royal Society of Chemistry, 2019).

Over the last decade, DAPR practices in science and mathematics have not substantially grown or been implemented in ways conducive to greater equity (e.g., making DAPR optional was found to favor prestigious authors; McGillivray & De Ranieri, 2018). Yet, both experimental (Huber et al., 2022) and data-backed studies continue to suggest the practice may substantially enhance equity and diversity in particular scientific disciplines or contexts, including medicine (Okike et al., 2016) and computer science (Seeber & Bacchelli, 2017; Tomkins et al., 2017a).

Practically speaking, few studies have addressed the impact of DAPR vs. SAPR on women’s academic output in science (e.g., Budden et al., 2008a; Budden, Tregenza et al., 2008b; Roberts & Verhoef, 2016; Tomkins et al., 2017b; Webb, O’Hara, & Freckleton, 2008) and none, to our knowledge, in mathematics. This article tackles what is possibly the first study of the effects of a combined switch in editor gender and review policy (man editor and SAPR to woman editor and DAPR) on women’s publication outputs in mathematics.

This study explores and aims to explain changes in authorships in the Monthly (see RQ1–4 in Section 3.2) straddling the journal’s change to its first ever woman editor and DAPR in 2015 (Chapman, 2015; Nagy, 2015). We build upon preliminary results presented at the Conference on the Global Gender Gap in Science (Barr & Lozano, 2017; Lozano, 2019).

The choice to focus on the Monthly was informed by research goals and circumstances. First, to our knowledge, the Monthly is the only journal publishing mathematics research to have switched from SAPR to DAPR. The switch happened on January 1, 2015, a year ahead of the SAPR to DAPR transition mandate’s date, which was slated to coincide with the new editor’s term (Chapman, 2015). Then editor Chapman noted no deviation from the norm during the transition year to DAPR (Chapman, 2016), ahead of the new editor taking office. Despite its focus on expository mathematics, the Monthly’s longevity (first published in 1894), international reach, and reputation further made it a compelling candidate for our study.

Second, the Monthly elected its first ever woman editor the year it switched to DAPR (Chapman, 2015; Nagy, 2015) allowing us to study the coupling of two key factors known to enhance equity in women’s academic publication output. According to Chapman (2015), the newly elected Monthly editor began handling submissions on January 1, 2016, rendering 2015 a robust editor/review type transition year for our study’s purpose.

Finally, the first author’s involvement in the Global Gender Gap in Science project (Guillopé & Roy, 2020), yielded both access to quality bibliographic data (from zbMATH, formerly Zentralblatt MATH, a major European reviewing and abstracting service), and an opportunity to carry out a study that could help explain and reduce women’s underrepresentation in mathematics publications (Mihaljević-Brandt et al., 2016; Mihaljević & Santamaría, 2020).

A first analysis of Monthly zbMATH data was done as part of an undergraduate independent study and presented at the Global Gender Gap in Science conference in Trieste (Barr & Lozano, 2017; Lozano, 2019). That work suggested an increase in women authorships and a decrease in authorships bearing fully abbreviated first names after the Monthly’s editor and review type switch in 2015. Yet limitations in both data and methods led to a redesigned research approach, and analysis centered on four questions:

• RQ1. 

  How did the share of women authorships change, straddling the Monthly’s editor/review switch year?

• RQ2. 

  To what extent are observed women authorship changes in the Monthly exceptional relative to changes in comparable journals in the field?

• RQ3. 

  How did the share of authorships bearing fully abbreviated first names change after the Monthly’s editor/review switch year?

• RQ4. 

  How did the share of authorships from newcomer authors change after the Monthly’s editor/review switch year?

To answer RQ1, RQ3, and RQ4 we compare Monthly authorship data four years before (2011–2014) and four years after (2016–2019) the year 2015 (henceforth, 2011–2019*). We exclude the year 2015 to accommodate for the transition in editor/review type, as is customary in this type of study. To answer RQ2, we systematically identify a set of comparable journals in focus, scope, reach, and standing (see Section 6) and compare the change in women authorships in said journals relative to the Monthly using a suitable Generalized Linear Mixed Model (GLMM) over the period of interest. As detailed further below, the three comparison journals had men editors and utilized SAPR (both field standards) throughout the period of interest.

Generally, as in this paper, publication output is measured in terms of authorships. An authorship is an instance of a specific publication by a specific author (e.g., a single article authored by three people gives rise to three authorship instances, one for each author). Depending on study aims and discipline standards, certain authorship instances may be more important than others (e.g., first, last, or corresponding author). For our purposes, we consider all authorship instances by a single author equally; we do not consider author position (which has not always served as a robust proxy for authors’ roles or responsibilities in mathematics).

The problem of genderizing—or inferring an author’s gender—is complex and remains open on various fronts (e.g., Mihaljević, Tullney et al., 2019). Most present-day studies utilize an author’s first name coupled with automated (e.g., genderize.io) and data-backed processes to infer (binary) gender. As described in Section 5.1, our initial data set included a robust genderization we later improved upon using manual methods. This initial genderization was carried out by Mihaljević-Brandt et al. (2016), leveraging zbMATH author profiles, each represented by a unique author ID. For more on zbMATH author profiles including the method originally utilized to genderize our data, see Mihaljević-Brandt et al. (2016) and Mihaljević and Santamaría (2020).

Compared to collecting gender information directly from authors, automated genderization approaches have substantive and technical limitations (Santamaría & Mihaljević, 2018), a thorough discussion of which is beyond the scope of this paper. Among the technical limitations is the impossibility of accurately inferring (binary) gender for certain classes of names, including Asian and gender-neutral first names. Among the substantive limitations is the implicit validity of a binary gender assignment in the first place.

Monthly data (see Section 5.1) was prepared for analysis using a combination of manual and automated methods to ensure the accuracy and completeness of records over 2011–2019*. To address RQ4 we leveraged the full available record of Monthly authorships, dating back to 1898.

Our final data for the Monthly consisted of 1,163 authorship instances from 2011–2019*. Most of these authorships (1,116) came from a data set shared by Helena Mihaljević for this study (personal email communication, December 1, 2019).

That initially shared Monthly data set consisted of 12,346 instances of Monthly authorships corresponding to the full record of Monthly publications between 1898 (Vol. 5) and July 2019 (Vol. 126, Issue 7) as indexed by zbMATH. This data included a rigorous genderization (Mihaljević-Brandt et al., 2016; Mihaljević & Santamaría, 2020) relying largely on zbMATH-constructed author profiles, which uniquely identify an author by combining “rule-based, manual, and collaborative methods” (Mihaljević & Santamaría, 2020, p. 92). These zbMATH unique identifiers usually include an author’s first name, enabling the genderization of many authorship instances in which the author’s name is abbreviated in print.

Because the initial data set was missing Monthly data past July 2019, we manually augmented it with all relevant authorship instances (N = 47) over July–December 2019 (Vol. 127, Issues 8–10) using published tables of contents (Taylor & Francis Online, 2020). We genderized these new authorship entries, and also manually improved the genderization included with the initial data set to improve the gender assignment rate over 2011–2019* (see Section 5.4).

A total of 251 instances of authorships over our period of interest were excluded from our final Monthly data set due to one or more of the exclusion criteria described below. Authorship data preceding the year 2011 served to identify new Monthly authors over the analysis period (RQ4) but was otherwise not part of our analysis.

Not all Monthly content is subject to peer review. Personal communications with the editor confirmed that book reviews, editor(s) letters and endnotes, problems and solutions, and articles reporting on awards (e.g., William Lowell Putnam Mathematical Competition reports) or recognizing awardees (e.g., Yueh-Gin Gung and Dr. Charles Y. Hu Award for Distinguished Service to Mathematics) are not subject to peer review (S. Colley, personal communications, October 26, 2017, November 5, 2020). Such authorships were removed prior to analysis, as were occasional reprints, errata, and addenda. Overall, 135 (∼12%) authorships in these categories were removed from the 2011–2019* Monthly data.

Another type of Monthly content excluded from analysis were the Monthly Mathbits (so-called “filler content”), pieces shorter than a page whose content is often purely formulaic, visual, or even poetic. Such content is not subject to the same review process as Monthly articles and notes (S. Colley, personal communication, November 5, 2020). Overall, 61 (∼5%) instances of Mathbits authorships were removed from the shared zbMATH-sourced data.

Additionally, a few erroneous authorship attributions included in the shared zbMATH-sourced data but absent on published tables of contents (Taylor & Francis Online, 2020) were also removed from the data. In each of these instances, authorship was assigned to a person whose name was mentioned in an article but who was not one of the article’s authors (e.g., Galois, Legendre, Ramanujan, Torricelli). A total of 21 (∼2%) instances of erroneous authorship attributions were identified since 2011 and removed from the data prior to analysis.

Finally, all authorships from Monthly special issues (36 additional authorships, from three special issues over 2011–2019*, each with its own guest editors) were excluded from our final data set prior to analysis.

While it has no bearing on our analyses, examination of the zbMATH-sourced data suggests inconsistencies and errors in indexed data that might potentially skew gender representation work if left undetected. We found indexing errors (i.e., authorship assignments to clearly nonauthors) in both Monthly and comparison journals authorship data (see Section 6.3). We found indexing inconsistencies (i.e., authorships tied to awards, book reviews, and nonpeer reviewed content indexed over some years, but not over others) primarily in Monthly data, but also for other journals. For example, zbMATH-sourced data included some Problems & Solutions authorships between 1933 and 2007 and in 2010, but excluded such authorships over other time periods (e.g., 2011–2015). Such inclusion/exclusion inconsistencies were also found for Mathbits authorships.

We improved authorship genderization in our final data (see Figure 1) as done elsewhere (Santamaría & Mihaljević, 2018; Topaz & Sen, 2016). We utilized author bios and matched author emails included in the Monthly with online search results (e.g., gender-specific pronouns, publications, and other information in professional organizations sites or university websites). We improved our genderization rates from 92% to 98% for the 2011–2014 authorship data (N = 547), and from 90% to 99% for the 2016–2019 data (N = 616). Sixteen authorships were excluded from the final set due to inconclusive gender assignment.

Addressing RQ2 required establishing a baseline for women authorship changes in comparable mathematics journals over 2011–2019, and then ascertaining if changes in the Monthly could be deemed exceptional relative to this baseline. Because the share of women authorships in mathematics has continued to grow slowly over time and this growth rate is subfield-dependent (Mihaljević-Brandt et al., 2016), we set out to systematically select a group of journals comparable to the Monthly. We first created a suitable journal profile for the Monthly, then searched for journals with comparable focus, scope, rankings, and reach based on specific inclusion/exclusion criteria (see Figure 2). That search yielded three journals comparable to the Monthly along these dimensions, yet with uniformly contrasting characteristics in what concerns editor gender (all men editors) and peer review type (all SAPR) over 2011–2019.

Our journal profile for the Monthly considered four main dimensions: Focus—the mathematics subject areas most represented in Monthly articles over 2011–2019* (Mathematics Subject Classification—MSC2020—codes); Scope—the journal’s focus on expository mathematics; Ranking—the journal’s standing based on specific journal rankings (ARC and JCI ratings); and Reach—global vs. local base in author participation (a sense of country representation among authorships).

The Mathematics Subject Classification (MSC), a coding system maintained by Mathematical Reviews and zbMATH, is widely utilized to classify mathematics publications according to the subject(s) they represent (Dunne & Hulek, 2020). Typically, authors submit one primary MSC code (required) and one or more secondary (optional) codes. Leveraging zbMATH Open (zbmath.org) we generated the list of primary MSC subject classification codes for Monthly articles over 2011–2019*.

Apart from MSC codes 00, 01, and 97 (General mathematics, History of Mathematics, and Mathematics Education), discussed separately below, we found MSC codes 11, 05, 26, and 51 (Number Theory, Combinatorics, Real Functions, and Geometry) were the most frequently used. MSC 11 was by and large the most popular primary field code (used 233 times or in 16% of the articles); MSC 05 was the second most popular code (used 110 times, or in 7% of the articles); MSC 26 and 51 came 4th and 5th (each in roughly 6% of the articles). Our Monthly focus profile hence leads with Number Theory (MSC 11) and also foregrounds Combinatorics, Real Functions, and Geometry (MSC 05, 26, 51) as secondary mathematics research areas. These profile features inform inclusion/exclusion Criteria 1–2 (see Figure 2).

Reflecting in part the journal’s call for expository articles, MSC codes 00, 01, and 97 are also among the 58 distinct MSC primary codes appearing in our Monthly list for 2011–2019*. MSC code 97 (Mathematics Education) is the third most common (105 articles), roughly on par with MSC 05 (Combinatorics). MSC 00 and 01 (General Mathematics and History of Mathematics) are the 6th and 7th most common, respectively. The relative prevalence of MSC codes 97, 00, and 01 in the Monthly’s profile (common, but at most half as common as the top code, MSC 11) informs inclusion/exclusion Criterion 3 (Figure 2).

Apart from content not subject to peer review and thus excluded from our data (e.g., book reviews, problems and solutions, awards—see Section 5.2), the Monthly publishes expository research articles in all areas of mathematics. The journal favors quality expository writing over research originality, as explicitly conveyed in the journal’s Aims & Scope. Criterion 4 (Figure 2) assesses comparability to the Monthly in this dimension based on a journal’s explicit commitment to exposition, as noted in the official description of the journal’s Aims & Scope.

The Monthly was ranked by the (now phased out) Australian Research Council (ARC) journal ranking system (Australian Research Council, n.d.; Creagh, 2011) and has been assigned a Journal Citation Index (JCI) rating—the JCI is a new field-sensitive measure of journal impact based on citations (Szomszor, 2021). The Monthly holds a B ranking under ARC and a 0.55 under JCI (as of 2021). We utilized both ARC and JCI ratings to assess comparability to the Monthly. Our aim is to leverage Monthly values under these markers to assess comparability, not to evaluate the journal or either measure. These rankings inform inclusion/exclusion Criteria 6–7 (see Figure 2).

We leveraged available zbMATH data on authorships with country information to garner a sense of the Monthly’s global reach (H. Mihaljević, personal email communication, December 17, 2022). Our aim was to establish a profile for global reach vs. more local (e.g., country-specific) reach, that would serve as an additional dimension for selecting comparable journals. About 22% of the Monthly authorships indexed by zbMATH (2011–2021) carry country information; 82% of these authorships come from the United States, with Canada, China, Germany, and Spain claiming a fairly equal split of the rest (see Table 1, Line 1). Reach informs inclusion/exclusion Criterion 8 (see Figure 2).

Our systematic search for comparable journals followed a sequence of inclusion/exclusion criteria informed by the Monthly’s profile just described. The process, summarized in Figure 2, was carried out using zbMATH Open data and tools, yielding at each stage a reduced set of journals with comparable characteristics to those of the Monthly.

We required that MSC 11 (Number Theory) and at least two among MSC 05, 26, 51 should appear among the journals’ MSC codes for 2011–2019* (Criterion 1, Figure 2). We further required that MSC 11 be among the three most common MSC codes for that period (Criterion 2). This yielded 40 journals, five of which had either MSC code 00, 01, or 97 as their most common primary field over our period of interest. To reflect the relative prevalence of those codes in the Monthly (at most half as common as the most common code, MSC 11—see Section 6.1.1) we retained the 35 journals for which none of those codes were their most common (Criterion 3). Of those 35, only journals with an explicit commitment to expository articles were retained (Criterion 4—see Section 6.1.2). In keeping with the Monthly’s explicit interest in all areas of mathematics and broad readership aim, we eliminated journals dedicated to particular research fields or particular audiences (Criterion 5). We then retained journals deemed comparable based on ARC and JCI rankings (Criteria 6 and 7) and global reach (Criterion 8, Table 1).

Finally, we required that the journals maintained SAPR (Criteria 9) and had men editors (Criteria 10) throughout 2011–2019, the standard in mathematics in general. When the journals or their official pages did not explicitly offer the information, we verified utilization of SAPR and the list of editors’ names over 2011–2019 by contacting the 2022 editors by email. The SAPR condition (Criteria 9) did not further reduce the comparison group. The men editors’ condition (Criteria 10) eliminated one last journal, Archiv der Mathematik (henceforth, Archiv), from the comparison group. Interestingly, Archiv switched to a coed editorship—a woman and man pair of managing editors—the same year as the Monthly editor/review switch (Archiv der Mathematik, 2016). This excludes Archiv from the comparison group but makes it an interesting special case for future research, as we discuss further below.

In all, inclusion/exclusion Criteria 1–10 yielded three comparable journals: Expositiones Mathematicae (henceforth, ExpositM), L’Enseignement Mathématique, 2e Série (henceforth, LEnseigM), and Rocky Mountain Journal of Mathematics (henceforth, RMJ).

We obtained complete authorship data over 2011–2019 for each of the comparison journals from zbMATH through the first author’s involvement in the Global Gender Gap in Science project (Guillopé & Roy, 2020). As with the Monthly data, comparison journal data included a rigorous genderization (Mihaljević-Brandt et al., 2016; Mihaljević & Santamaría, 2020), leveraging zbMATH-constructed author profiles (see Section 5.1).

We cleaned up authorship data for each new journal, applying the same inclusion/exclusion criteria utilized for the Monthly. Specifically, we removed content not subject to peer review (e.g., obituaries, awards, book reviews, errata, special issue articles) and misassigned authorships (e.g., authorships clearly assigned to nonauthors). As done elsewhere (Mihaljević-Brandt et al., 2016), in cases where one authorship instance was assigned multiple unique zbMATH identifiers without a uniform gender assignment, we removed all instances of such authorship, unless we were able to identify the actual author using manual methods.

We did not, however, improve on Mihaljević-Brandt et al.’s (2016) already robust genderization for the comparison journal data. Table 2 summarizes final genderization rates, authorship counts and other journal-level data for the comparison group and the Monthly. As noted by Mihaljević-Brandt et al. (2016), the actual share of women authorships is less than or equal to the estimated share of women authorships based on conclusively genderized data.

We first report on RQ1, RQ3, and RQ4 (changes in women, abbreviated names, and newcomer authorships; see Section 3.2), which concern Monthly data only. We then report on RQ2, which looks at whether the increases in Monthly women authorships (RQ1) were exceptional.

Regarding RQ1, crosstabulation of Monthly men and women authorships (Table 3) straddling the year 2015 shows women authorships increased by about 4.3 percentage points (6.5% to 10.8%) after 2015. A chi-squared test suggests that, albeit a small effect (r = .08), the observed increase in Monthly women authorships after 2015 was significant (p = 0.011 < 0.05), meaning unlikely due to chance. As our results for RQ2 show further below, the increase is also exceptional relative to the growth of women authorships in comparable journals under men editors and SAPR before and after 2015.

Regarding RQ3, crosstabulation of Monthly authorships with fully abbreviated names (FAN) and authorships bearing first name(s) (Table 4) shows FAN authorships decreased by 3.2 percentage-points (from 8.2% to 5.0%) after 2015. A significant (p = 0.028) chi-squared test shows that the decrease in FAN authorships is unlikely due to chance. This could suggest authors were less likely to fully abbreviate first names after 2015, the year of editor gender/review type switch. Thanks largely to zbMATH author profiles (see Section 5.1), we conclusively assigned gender to 91% of all FAN authorships (69 out of 76) in our data set. Throughout 2011–2019*, 96% of FAN authorships with a conclusive gender assignment belonged to men and 4% to women. After the switch year (2015), all conclusively genderized FAN authorships belonged to men, none to women.

Figure 3 shows trends in women and FAN authorships over 2011–2019* each as a share of all yearly authorships. Trends varied closely pre-2015 and pulled apart post-2015.

Regarding RQ4, crosstabulation of authorships from newcomer authors to the Monthly versus those with at least one prior Monthly publication in the history of the journal (Table 5) shows the share of authorships from newcomer authors did not significantly change (65.1% vs. 64.8%) after 2015, as a chi-squared test confirms (p > 0.5).

To address RQ2 we looked at changes in women authorships in each of the three comparable journals individually, then used a Generalized Linear Mixed Model (GLMM) to test if women authorship changes in the Monthly were exceptional relative to the field, as represented by our group of comparable journals.

Individual analyses of each comparison journal showed that, as anticipated, the share of women authorships did at least slightly increase in 2016–2019 compared to 2011–2014 for all journals (see Figure 4). Yet, chi-squared tests convey these increases were not significant for any of the three comparison journals (ExpositM, p = 0.86; LEnseigM, p = 0.91; RMJ, p = 0.36).

To assess the exceptionality of the Monthly changes relative to the group of comparison journals (taken as a proxy for field; see Section 6), we fit a GLMM with Poisson errors and log-link, using the lme4 package in R version 4.3.1 (Bates, 2005). GLMM modeling is useful in this context due to the nonnormal distribution of our output variable (number of authorships) and the need to consider random effects.

We modeled the number of authorships as a function of gender, time (2011–2014 or 2016–2019) and whether or not the journal had editor and peer review changes (True or False), using journal as the random effect variable (Monthly, ExpositM, LEnseigM, RMJ). This model showed no significant interaction between gender and time (coefficient ± SE = 0.125 ± 0.131, p = 0.34), indicating that the increase in women authorships across 2015 cannot be explained by time. But the model did show a significant interaction between gender and editor gender/review type (coefficient ± SE = –0.902 ± 0.201, p < < 0.0001), indicating that editor gender/review type (man and SAPR vs. woman and DAPR) does help explain the increase in women authorships.

In other words, in our context, time does not contribute to explaining the growth in the share of women authorships across 2015. Yet the combined journal changes made by the Monthly (woman editor and DAPR) do in fact help explain the observed increase. Thus, our results show that the increase in the share of women authorships seen in the Monthly was indeed exceptional relative to the field, as represented by the group of comparison journals.

As noted earlier, our systematic selection method (see Section 6.2) identified one journal, Archiv, excluded from the comparison group due to not meeting the last inclusion criterion (i.e., man editor, Criterion 10, Figure 2). Archiv maintained SAPR over 2011–2019 but switched to a woman and man pair of managing coeditors at the start of 2016, in sync with the Monthly. An individual analysis of Archiv’s data (637 authorships, 87 from women over 2011–2014; 667 authorships, 118 from women over 2016–2019) showed a (borderline) significant increase (p = 0.045) in women authorships across 2015 (coupled with small effect size, r = 0.06). Archiv’s data (prepared for analysis as for the comparison journals) had a relatively low genderization rate (76.9% vs. 98.6% for the Monthly), and showed a relatively high share of women authorships from repeat authors (16.0% vs. 4.0% for the Monthly).

The significant increase in the Monthly’s share of women authorships straddling 2015 is noteworthy in several ways. First, unlike what was seen in other studies (Budden et al., 2008a; Webb et al., 2008), the increase was substantial relative to previous high and lows—the historic yearly trend in the share of Monthly women authorships oscillated around 8% in the 12 years or so prior to the switch, climbing to about 14% three years after it (Lozano, 2019).

Second, also unlike other studies (Budden et al., 2008a; Webb et al., 2008), the increase in the Monthly is exceptional relative to the overall growth in the share of women authorships in comparable journals. That is, the increase in the Monthly cannot be explained by the underlying growth over time in women authorships in the areas and foci the Monthly centrally embodies (see Section 6.1).

Third, while we cannot systematically disentangle the potential effect of DAPR vs. editor gender on the increase in women authorships for the Monthly, we can lean on relevant literature and data-based observations to discern the likely effects of each factor separately as well as together.

DAPR has been linked to increased expert newcomer participation in certain specific science contexts (Seeber & Bacchelli, 2017). In the Monthly context, however, we saw no noteworthy changes in the share of new authors straddling the switch year (results for RQ4). We also observed a negligible share of repeated women authors over 2011–2019* (i.e., only 4% of the women authorships over 2011–2019* were authorships from repeat authors). Taken together these suggest the Monthly’s switch to DAPR had no noteworthy impact on newcomer authors, including women newcomers.

In fact, editor Chapman’s preliminary assessment of the Monthly’s first year of DAPR (just ahead of its first woman editor) states “submission and acceptance statistics from 2015 are completely in line with what we saw in 2011, 2012, 2013, and 2014” (Chapman, 2016, p. 956). While agnostic about gender representation, Chapman’s statement (2016) suggests the first year of DAPR was unremarkable for the Monthly, which is consistent with our observations regarding newcomer and repeat authors. Yet evaluating over a longer period, and hence of necessity across a combined switch in editor gender and review type, reveals a significant increase in women authorships not seemingly manifest in 2015. A woman editor could help explain this increase in specific ways, as recent studies (Helmer et al., 2017; Royal Society of Chemistry, 2019) support.

Women editors are unlikely gatekeepers for women’s publication output and may in fact contribute to increasing it. Helmer et al. (2017) showed that women editors tend to choose more women reviewers than men editors, and contribute to gender diversity among reviewers more than men editors, for which homophily is widespread (Helmer et al., 2017). The Royal Society of Chemistry also observed that women editors tend to select, and are especially likely to agree with, women reviewers (Royal Society of Chemistry, 2019). In turn, women reviewers have been shown to more equitably score science proposal submissions (Severin et al., 2020) than men reviewers, suggesting women editors may enhance gender equity along the submission to publication pipeline irrespective of peer review type, possibly softening some of the effects that DAPR may bring about otherwise.

The particular case of Archiv—robustly similar to the Monthly based on Criteria 1–9 (see Figure 2) but under a coed editorship (man and woman pair) since 2016—could further hint that the presence of a woman editor serves to enhance gender equity in authorships, yet possibly preserves what are arguably “signatures” of particular peer review types, such as SAPR. This argument is consistent with Archiv’s (borderline) significant growth in women authorships straddling the year 2015, along with its high percentage of women authorships from repeat authors (16%) over 2011–2019*, a possible tell-tale mark of Archiv’s SAPR, a review type that has been shown to favor “known” authors in various science contexts (e.g., McGillivray & De Ranieri, 2018; Okike et al., 2016; Seeber & Bacchelli, 2017).

Given the particularities of mathematics publishing venues (pervasive SAPR and men editorships), detangling the effects of different peer review elements all deemed to enhance equity in the reviews, may be less valuable than designing a study to assess their effect as a whole, clustering the few cases in which practices run counter to the field’s standards as far as peer review type and editor gender, for example.

In the Monthly’s particular case, further detangling the effect of DAPR and editor gender might require waiting a few more years. Such an investigation would have to account for the (possible cumulative) impact of a global pandemic in women authorships (Squazzoni, Bravo et al., 2021b) as well as the Monthly’s 2022 transition to its second-ever woman editor.

Finally, the fact that 96% of all conclusively genderized FAN authorships over 2011–2019* (see results for RQ3) belong to men authors (counter to our initial hypothesis; Lozano, 2019) suggests the Monthly’s decrease in FAN authorships across 2015 is consistent with the increase in women authorships. Further analysis of FAN authorships in terms of gender and other author characteristics may be of interest.

Reducing the gender gap in mathematics and science is tied to slow-changing macro (cultural, societal) and meso (institutional, organizational) level norms (Lozano, 2021). In particular, evidence-based changes along the submission to publication pipeline—including its actors (e.g., editors, reviewers) and practices (e.g., peer review policies and processes) have been shown to mitigate implicit biases present within a still gendered-masculine (Acker, 1990) academy.

Our study leverages a rare occurrence—a switch in two factors associated with greater gender equity in peer review (editor gender and peer review type)—in one of the oldest mathematics journals in the United States. Working with data on accepted papers only (we did not have data on journal submissions), our results show a significant increase in women authorships explained by the switch to a woman editor and DAPR rather than time, making the increase exceptional relative to the underlying growth in women authorships in the field.

We recognize that data on submitted papers (in addition to accepted ones) and gender distribution among reviewers—both not available to us for this analysis, could lend nuance to our results in valuable ways and thus are limitations of our study. So is the relatively small size of our data set.

While the impossibility of systematically detangling the effects of editor gender and peer review type can also be seen as a limitation of our study, we argue (see Section 8) that it is perhaps the coupling of these two factors—and possibly other variations of these factors, as in the case of the coed editorship for Archiv—that maximizes the equity afforded by the approach. Given the rarity of DAPR and the scarcity of women editorships in mathematics, we propose it may be more interesting and valuable to trade the question of detangling editor gender vs. peer review type effects for one that considers whether collections of equity-aligned peer review practices help reduce bias and representation gaps better than the norm, at least in mathematics.

The first author would like to thank Helena Mihaljević for sharing the journal data for this study and useful email exchanges; Susan Colley for providing valuable information on the Monthly’s history, structure, and review policies; and Liz Rennick for performing the initial statistical analysis. The first author would especially like to thank Marie-Françoise Roy for the invitation to serve on the Advisory Board for the Global Gender Gap in Mathematics and the Natural Sciences Project, and for the multiple opportunities to contribute to this project through talks and other media. We are also thankful for Sam Dutton’s advice on the GLMM analysis.

Guadalupe Lozano: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Supervision, Visualization, Writing—original draft, Writing—review & editing. Kelly MacArthur: Formal analysis, Methodology, Writing—review & editing. Ben Barr: Formal analysis, Writing—review & editing.

The authors have no competing interests.

This research was not funded.

Journal authorship data (from Monthly, ExpositM, LEnseignM, RMJ, Archiv) is subject to a data sharing agreement between zbMATH and Mihaljević et al.; it was shared with the first author with special permission. As zbMATH is now open, such data should be retrievable from zbMATH.org, save for the genderization and possibly author profiles in the original data files (see Section 5.1). The data utilized for the comparison journal selection was retrieved directly from zbMATH.org, using the retrieval criteria described in Section 6.2.