Large publishing consortia produce higher citation impact research but coauthor contributions are hard to evaluate

The frequency and scale of academic research collaboration varies within and between fields but has grown steadily over time (Fortunato et al., 2018). Whilst arts and humanities scholars often work alone (Larivière et al., 2006; Wuchty et al., 2007), some types of experimental and applied research require large teams (de Solla Price, 1986). Interdisciplinary collaboration may be needed to solve complex real-world problems (e.g., Moore et al., 2017; Olds, 2016), in a process that has been called mode 2 science (Gibbons et al., 1994). Big teams may also be needed to operate large-scale equipment, such as the Large Hadron Collider, and collaboration may sometimes be more efficient by combining expertise. Weak or strong collaborations have also been formed to create specific resources of common value, such as the Human Genome Project (Roberts, 2001), leading to their landmark paper (International Human Genome Sequencing Consortium, 2001), epigenomics mapping (Bernstein et al., 2010), and gene product function ontologies (Gene Ontology Consortium, 2014). Nevertheless, research consortia may be formed through policy decisions or in response to research funding requirements, which may be less successful (Defazio et al., 2009) or primarily benefit less successful partners (Hoekman et al., 2013).

Close collaboration can also occur around data when a discipline collaborates to create standards and repositories to systematically exchange information (e.g., AACR Project GENIE Consortium, 2017; Buniello et al., 2018; Costa et al., 2016; Vermeulen et al., 2013; Welter et al., 2013). These are problematic to evaluate with scientometric methods because the primary output of such collaborations is not a set of journal articles.

A more recent reason to collaborate on some types of research is that competing groups using similar methods to address a shared problem may lead to a situation in which the wrong answer achieves statistical significance and is published, whereas combining the competing teams would give the correct result (Ioannidis, 2005; Munafò et al., 2017). This occurred for Genome-Wide Association Studies (GWAS), for example. In this field, multiple teams attempted to identify single nucleotide polymorphisms (SNPs) on the human genome that are associated with genetic diseases or health problems. Each team generated its own data set, typically consisting of hundreds or thousands of human volunteers and genetic information on a sample of SNPs. Only positive results were published and many negative results were discarded, which led to incorrect results being published when they accidentally achieved statistical significance, unless more stringent thresholds were used (Pe’er et al., 2008). A side-effect was the “winner’s curse,” with the effect of SNPs being overestimated in studies that found them (Garner, 2007). The solution adopted by the community included pooling data sets to improve statistical power so that the number of discarded negative results would be reduced. In this situation the incentive to form consortia was statistical and improved research validity and power (Psaty et al., 2009).

Collaboratively authored journal articles may be valued by their coauthors in research evaluation exercises because they tend to be more cited than solo articles in many fields, although the citation advantage of collaboration has diminished over time (Larivière et al., 2015). An advantage of team science is the ability to apply novel combinations of expertise, but the research produced is not more novel than less collaborative research in some fields (Wagner et al., in press). The GWAS example above illustrates that large teams may be needed to make research more predictable. Author ordering may be a problem in large collaborations, making it difficult to assess individual scholars’ contributions. Although alphabetical author ordering is not the norm in any broad field of science (Levitt & Thelwall, 2013; Waltman, 2012), it seems unlikely that a large group of authors on a long-term project would be able to precisely determine relative contributions, and partial or full alphabetical ordering would be a logical solution to this. A common biomedical solution is to order the main contributors at the start and end of the author list nonalphabetically, with the remaining authors ordered alphabetically in the middle (Mongeon et al., 2017).

Despite much evidence about collaboration in general within fields, science-wide, or from an individual funding program, no previous science-wide study has investigated the prevalence of publishing consortia, their impact, or use of alphabetical ordering. This is important because research evaluation exercises often need to evaluate the contributions of individual authors or departments, which may be difficult for large collaborative papers. This article uses a heuristic to identify publishing consortia throughout science from 1996–2018 to investigate impact and authorship order, and a manual classification of their type for deeper insights into their contributions to science.

• • 

  RQ1: Do large publishing consortia produce higher impact research?

• • 

  RQ2: Does authorship order reflect contributions in large publishing consortia?

• • 

  RQ3: Which types (fields, reason for existence) of large publishing consortia exist?

All 67,608,187 Scopus records of type journal article published between 1996 and 2018 were downloaded between November 2018 and February 2019. Articles tagged with other categories, such as review papers or editorials, were excluded to focus on a standard document type. The data included the author names, as recorded by Scopus, up to a maximum (usually) of 100. Each author was also assigned a unique ID by Scopus and these IDs were used to compare authors, even though they are likely to be sometimes incorrect, to reduce errors caused by authors having the same common name. Scopus author IDs are automatically assigned by an author name disambiguation algorithm that prioritises precision (i.e., attempts to avoid conflating different researchers with the same name; Moed & Plume, 2013). Its accuracy seems to be high, since a study of Japanese-funded authors found it to have precision of 99% (99% of papers associated with an author ID were correctly assigned) and recall of 98% (98% of an author’s papers were assigned to their ID; Kawashima & Tomizawa, 2015). Duplicate IDs occurred in these lists when authors had multiple affiliations, reducing the total number of unique authors, for articles with large sets. For example, a paper with thousands of authors would have 100 author IDs in Scopus, with as few as 30 being unique if they had about three affiliations each. Duplicate IDs were removed before subsequent processing.

Document clustering has previously been used to identify scientific topics (Sjögårde & Ahlgren, 2018; Waltman et al., 2010) and network mapping has been used to identify collaboration patterns (Xie et al., 2016), but the task here is different: to identify groups of articles created by a mostly common set of authors. A heuristic was used to identify publishing consortia. A set of articles was classified as a large consortium product when all had at least 20 (unique) authors, each article had at least 80% of authors in common with one other article in the set, and the collection contained at least three articles, building collections through a simple agglomerative clustering algorithm. These values (80%, 20 and, less critically, 3) were chosen after manual inspection of the data with the aim of getting the most generous definition that included few false consortia (i.e., one that did not self-define as a coherent group). A commonality of 80% allows for moderate changes in the composition of a consortium.

The consortium identification method used would be defeated by consortia with hundreds of authors that substantially changed author name orders, since the number of authors in common would then be hidden by the Scopus 100 author maximum threshold. No examples were found of consortia that were split into separate groups because of this, but others may not have been found. The single linkage agglomeration method allows a consortium to evolve gradually. For example, a group of authors that maintained a size of over 20 people, published at least one article every year, and had a personnel changeover rate of under 20% would appear as a single cluster of papers. This is because an article with, say, 20 authors would cluster with a second if up to four of these authors changed. It would cluster with a third paper with an overlap of less than 80% if another paper in the cluster had an overlap of at least 80% (Figure 1).

In partial validation of the method and parameters chosen, most of the consortia identified had given themselves names and so were formal research groupings. The clustering rules may still have been too conservative, however, because there is no evidence about the extent to which clusters were missed. In particular, research groupings with selective authorship (e.g., only contributing authors from a consortium pool) would be omitted.

RQ1: The Mean Normalised Log Citation Score (MNLCS; Thelwall, 2017) was calculated for each consortium to assess whether it produced research with a citation impact above the world average. This is a field normalised citation impact indicator that divides the citation score for each paper by the world average of all papers published in the same field and year. This avoids advantages for older articles (because they are normalised against contemporary articles) and for high citation fields (because they are normalised against articles from the same field). The log transformation ln(1 + c) was applied to all citation counts before any calculation to reduce the impact of individual highly cited articles so that (a) the data is less skewed and it is more reasonable to average it with the arithmetic mean and (b) the resulting figure is less subject to variability (Thelwall & Fairclough, 2017). Most articles were in multiple fields and in these cases the arithmetic mean of the values for each field was used.

RQ2: Alphabetical ordering of authors is not straightforward to assess, because a consortium may use partial or full alphabetical ordering (Mongeon et al., 2017) and authors may fall into alphabetical order by accident. Partial alphabetical ordering can also be a side effect of cultural differences in contributions because, for example, Chinese last names are more likely to start with Z than American last names. A nonalphabetic systematic ordering may also be used for articles originally written in languages, such as Chinese, that do not use the Latin alphabet (but alphabetical ordering from China is rare: Liu & Fang, 2014), and may use radical and stroke sorting or pronunciation order. Moreover, consortia may make different decisions about how to deal with issues such as prefixes (and their capitalisation), hyphenated names, double last names, accented characters in the extended Latin alphabet, cultures where the second name is the given name, cultures where the penultimate name is the family name, and whether to use the full first name(s) or the initials when ordering. Because of these issues, articles in perfect alphabetical (or equivalent) order according to the rules of a consortium may appear to be not lexically sorted based on the information available to Scopus or in the original article, in addition to any transcription errors. A heuristic was therefore used to judge the degree of alphabetical ordering. This was designed to be sensitive to partial alphabetical orderings after observations of this in some consortia. The alphabetical ordering heuristic used was to count the proportion of consecutive distinct (with different last names and/or first initials) authors that were in alphabetical order, according to their last name and first initial, as recorded in Scopus. Thus a score of 18/19 for an article with 20 authors indicates that the nth author was alphabetically before the (n + 1)th author for all consecutive pairs of distinct authors except one. On average, an article should score 50%, and values that substantially deviate from this indicate a degree of deliberate alphabetical order. For convenience of analysis, articles were split into groups based on the extent of alphabetical author ordering: close to alphabetical (90%+); partial alphabetical (between 60% and 99%); close to nonalphabetical (from 40% to 60%); and anti-alphabetic (Below 40%). A large-team biomedical article with an average sized set (60%) of alphabetically ordered middle authors and half of the remaining authors alphabetically ordered by accident (Mongeon et al., 2017) should therefore score 80% and fall comfortably into the partial alphabetical set. The Scopus limit to 100 author affiliations affects the accuracy of this for a minority of the consortia. Although the average number of authors for a consortium article was 38, consortia with over 100 and partial alphabetical ordering might have this hidden if most of the first 100 authors were in an initial nonalphabetical section, although this seems unlikely.

RQ3: The largest 250 consortia were examined (all those with at least 17 papers) and manually clustered by discipline and consortium type. This clustering was performed by examining basic properties of the consortium and, from acknowledgments, methods descriptions and web searches, identifying the scope of the grouping. This was an inductive and iterative process, with the initial classifications being revisited and subsequently reclassified into a smaller number of different types. The largest consortium of each type was then singled out for a more detailed description, based on a brief publication history and, if relevant, web explorations. Physics and astronomy consortia appeared to be very similar and were ignored after the first few.

The key data behind the results, including a list of consortia, is on Figshare (https://doi.org/10.6084/m9.figshare.8214050.v1).

Based on Scopus journal articles from 1996–2018, the largest apparent consortium has 755 articles and 3,927 consortia have at least three articles, totaling 31,340 papers. This is 0.05% of the set examined. The largest 250 consortia include between 16 and 755 articles, accounting for 14,135 articles. Thus, large publishing consortia are numerically common in academia but are very rare overall, at least as defined by the heuristic used here.

Consortium size follows a power law, which gives a straight-line shape on a dual logarithmic scale (Figure 2). There are many small consortia (in terms of the number of papers published) and few large ones, with no evidence of any size being unusually common. A power law can be produced if there is a positive feedback mechanism, but it is not clear how such a mechanism could operate here.

The 3,927 publishing consortia produced research with an average citation impact of MNLCS 1.954. On the log-normalised scale, this is almost double the impact of the world average (=1). Because of the (typically) shrinking effect of the log normalisation part of MNLCS, large publishing consortia usually publish research with several times more citation impact than the world average, although the exact ratio depends on the average (untransformed) citation counts involved.

The overwhelming majority (87.5%: 3,427) of individual consortia had an MNLCS above the world average of 1, including the largest 93. The first MNLCS below 1 was for research from a single department (in the United States) rather than a multi-institution consortium. A Spearman correlation between latest year of the first published article from a consortium and MNLCS of −0.818 (p = 0.000) confirms that consortia with a lower impact score tend to be the newer ones for which the MNLCS value is less reliable due to a shorter time to accrue citations. In fact, for very recently published articles, most have a normalised impact below the mean as an artifact of the dominance of zeros, further explaining this result (e.g., to give an extreme example, if only 1 out of 100 recently published articles had been cited, 99 of them would have a field and year normalised citation score of 0, which is below the world average of 1). All the 1,050 consortia with their first articles published before 2006 (for which citation counts are mature and hence most robust) had an average impact above the world average (i.e., MNLCS > 1). This is clear statistical evidence that large publishing consortia reliably produce research with citation impact substantially above the world average.

A tiny positive Spearman correlation between cluster size (number of articles published by a consortium) and MNLCS of 0.072 (p = 0.000) suggests that the number of articles published by a consortium has little relationship with the average impact of its research. This may be due to a variety of conflicting factors, however, such as consortium age, type and discipline.

Although two fifths (38%) of the consortia largely avoid any kind of alphabetical ordering (on average for their papers), a fifth (22%) are in perfect or close to perfect alphabetical ordering and two fifths (38%) are in partial alphabetical order (over 60% but less than 90% of consecutive different author names being in alphabetical order) (Table 1). Counting by paper rather than by consortium, three quarters of papers in consortia (75%) have at least partial alphabetical ordering (above 60%). This figure is higher than for consortia (60%) because the biggest consortia are more likely to use alphabetical ordering. The largest 19 consortia all have average alphabetical ordering rates of at least 84%, with 16 of them above 90%.

Eight types of consortia were identified from manual examinations of the papers written by the largest 250 (Table 2). The types are distinguished based on the apparent consortium purpose and broad field. This typology is one coherent way of differentiating between consortia based on their apparent purpose, rather than a definitive typology. The largest example of each type is discussed below. Most of the types of consortia are organised around a specific narrow task or equipment, although some are subcollections within a department or funded broad research groups.

The ATLAS Collaboration at CERN (atlas.cern/discover/collaboration) had 755 journal articles from 2010–2018 in high energy or nuclear physics journals (e.g., Journal of High Energy Physics: 23%). ATLAS (A Toroidal LHC ApparatuS) at the Large Hadron Collider (LHC) is a large multinational particle detector project using specialised equipment. Its publications record particle properties, behaviours, or searches (e.g., “Search for lepton-flavor violation in different-flavor, high-mass final states in pp collisions at $s$ = 13 TeV with the ATLAS detector”), including for the Higgs boson.

The role of the multinational consortium is as a fixed pool of skills to design and run the experiments. Authorship is in alphabetical order (an average of 98% alphabetical using the methods above). Although an average of 87 authors are recorded per paper, the collaboration has hundreds of authors and all are listed on all papers associated with the apparatus during their project tenure.

The HESS Collaboration (www.mpi-hd.mpg.de/hfm/HESS/pages/collaboration/) is a multinational consortium around a single stereoscopic astronomy telescope system. Its 150 publications from 2004 to 2018 record and analyse phenomena observed in space (e.g., “Discovery of very high energy γ-ray emission from Centaurus A with H.E.S.S.”).

The role of the multinational consortium seems to be again a fixed pool of skills to design and run the experiments. Authorship is always in alphabetical order (an average of 95% alphabetical using the methods above), with an average of 92 recorded in Scopus but all articles having hundreds of authors.

The Genomic Encyclopedia of Bacteria and Archaea (GEBA) project is a joint initiative by the Leibniz-Institute DSMZ–German Collection of Microorganisms and Cell Cultures and the U.S. Department of Energy (DOE) Joint Genome Institute (JGI) to systematically select and sequence bacterial and archaeal genomes (Wu et al., 2009). Its 148 Scopus publications from 2009–2015 identified for the cluster had an average of 35 authors, with orders varying substantially between articles. All except one were published in the Springer environmental biology journal Standards in Genomic Sciences and reported a complete, or near complete, genome for bacteria or archaea. It has sequenced 250 bacteria and archaea genomes (jgi.doe.gov/our-science/science-programs/microbial-genomics/phylogenetic-diversity), so the cluster is an incomplete record of its activities. Most of its articles had titles starting with “Complete genome sequence of” (125) or “Genome sequence of” (14). For example, one article was a short report that summarised key properties of the genome of an orange seawater bacterium, as well as reporting that the complete sequence had been recorded in GenBank, with wider project results in the Genomes OnLine Database (Riedel et al., 2012).

The role of the consortium is presumably to combine the multiple types of equipment and other expertise needed to efficiently sequence and analyse genomes. Since author orders and numbers vary (although a standard list is sometimes at the end (e.g., Bristow; Eisen; Markowitz; Hugenholtz; Kyrpides; Klenk) the set seems to reflect the relative contributions of the participants. Paper authors are, on average, in 49% alphabetical order using the methods above, confirming that this is not used.

The Swiss HIV Cohort Study (SHCS) has been from 1998 “a systematic longitudinal study enrolling HIV-infected individuals in Switzerland” (www.shcs.ch), gathered from multiple hospitals and clinics. It includes the Swiss Mother and Child HIV Cohort Study (MoCHiV). The 120 papers in the cluster were from 2000–2018, had an average of 57 authors and were published in many different journals, with the most common being AIDS (13%) and Journal of Infectious Diseases (12%). The publications seem to cover a wide range of HIV-related research, and appear in many journals. There is no typical paper for the consortium. As an example, one paper reports a type of cancer risk for patients (Clifford et al., 2008).

The role of the consortium is presumably to collect and manage information on the study. Author order varies by paper. The first few authors are typically not in alphabetical order, but the remainder are (an average of 85% alphabetical using the methods above). Thus, each paper seems to have a small group of main authors: perhaps those that planned the study, conducted the analysis and wrote the paper, with the remaining authors perhaps awarded coauthorship for contributions to data collection and management.

The China Antimicrobial Surveillance Network (CNINET) originates from 2005 and is an ongoing surveillance program in China for antimicrobial resistance (including antibiotic resistance). Its sister program, the China Antimicrobial Resistance Surveillance System (CARSS), focuses on regional comparisons (Hu et al., 2018) but did not appear as a consortium in the Scopus data. Only one other consortium of this type appeared in this data, from Canada, despite similar international data collection exercises (World Health Organization, 2017). CNINET’s 52 Scopus publications were almost all (51) published in the Chinese Journal of Infection and Chemotherapy from 2008–2017 had an average of 32 authors. The publications describe resistance associated with one genus, such as “CHINET 2009 surveillance of antibiotic resistance in enterococcus in China.” This focuses on a genus of lactic acid bacteria related to human medical conditions such as urinary tract infections and meningitis. Its data was from 14 hospitals in China. There are also generic surveillance papers, such as, “CHINET 2008 surveillance of bacterial resistance in China.” The value of a long-term coherent consortium for this task is presumably in standardising methods for reliable comparisons over time (Hu et al., 2016), as well as general efficiency gains.

The role of the consortium is presumably to manage access to data or testing at the different sites, because the authors have multiple hospital affiliations. Author order varies between papers, except for the general surveillance papers. For example, the first 13 authors were the same and in the same order for the 2008 and 2009 general surveillance papers. Authors are in an average of 50% alphabetical order (i.e., exactly that expected if alphabetical considerations are ignored) using the methods above.

The Department of Orthopaedic Surgery at Chiba University Hospital in Japan had a cluster of 46 papers from Scopus 2009–2017, mainly published in spine-related journals, such as Spine (37%) and European Spine Journal (20%), with an average of 22 authors. Its research includes both spinal and joint injuries and it seems to be a highly successful award-winning research group (www.ho.chiba-u.ac.jp/en/dpt/orthopaedic.html). The main author of many papers, Professor Seiji Ohtori, had two records in Scopus that accounted for 391 publications (Author ID: 7004456445, 197 publications 2009–2014; ID: 56008400700, 194 publications 2012–2019).

The cluster is a department rather than a consortium. Most papers include only authors within the department. Authors are not in alphabetical order (average: 52%). Author order varied between papers, although often with the same first author and similar sequences of last authors and two of the three professors (as listed in the department website in 2019) listed last except in four cases. Papers by the department not included in the cluster had fewer authors (e.g., 6: “Segmental Pedicle Screw Instrumentation and Fusion Only to L5 in the Surgical Treatment of Flaccid Neuromuscular Scoliosis”), or a smaller overlap than 80% due to a collaboration (e.g., 22 authors, but several were from other universities, “Use of Bioelectrical Impedance Analysis for the Measurement of Appendicular Skeletal Muscle Mass/Whole Fat Mass and Its Relevance in Assessing Osteoporosis among Patients with Low Back Pain: A Comparative Analysis Using Dual X-ray Absorptiometry”). Thus, the cluster of 46 articles represents under a quarter of the output of the department: a highly collaborative ad hoc subset.

The JACC Study (Japan Collaborative Cohort Study for Evaluation of Cancer Risk) included 46 papers published from 2002–2008, mainly in the Journal of Epidemiology (72%), mostly examining specific risk factors for a type of cancer (e.g., “Smoking, alcohol drinking and esophageal cancer: Findings from the JACC Study”). This study gave a lifestyle questionnaire to 73,424 people from 1988–1990 and tracked their health problems until the end of 1997 (Iso et al., 2002). This provided a rich data source to investigate the relationship between lifestyle factors, diseases and mortality.

The cluster is a named consortium from different universities, hospitals and health centers created around a shared, funded task. Authors are not in alphabetic order (47% of distinct author pairs in alphabetical order). Authorship presumably reflects contributions to leading the extensive shared data gathering task for the initial questionnaires or the follow-up health outcomes information. An average of 40 authors (min: 34; max: 50) wrote each paper, with varying author order (no more than four papers with the same first author), although the last-listed about 10 authors tended to be in a similar order. Thus, authorship seems to reflect roles in the data collection process, with the first authors presumably carrying out the analysis for a paper.

The IMAGEN Study is a consortium funded by a series of EU, national and other grants to follow a cohort of 2,000 young Europeans from the age of 14 to track adolescent brain development through questionnaires, brain imaging and tests (imagen-europe.com). Its publications from 2011–2016 cover different aspects of behaviour and brain development (e.g., “No differences in ventral striatum responsivity between adolescents with a positive family history of alcoholism and controls”), with an average of 26 authors per article.

The cluster contains about a quarter of the articles from the project, since its website reports 104 journal articles from 2010–2018. Author order is typically alphabetical in the middle but not at the start (presumably the main authors for the study) and end (presumably consortium leaders). There is an average of 77% alphabetical ordering using the methods above. The role of the consortium seems to be partly to maintain and update core information about the cohort and partly to get additional information for specific analyses.

This paper introduces a new method to identify large publishing consortia as well as evidence of their research impact, use of alphabetical ordering, and broad types. The results show that there are large publishing consortia in nuclear physics, astronomy, and some health-related fields. The heuristics used to identify these consortia cannot be used to estimate the prevalence of large stable research groups because others may have different publishing agreements or may produce other types of output. Thus the 0.05% of Scopus articles included in them underestimate the extent of large publishing consortium output. The highest profile huge research collaboration in the last 50 years, the Human Genome Project, was not included because of this. In addition, modifications to the heuristics (e.g., 70% authors in common; minimum 10 authors per paper) would produce different results. Nevertheless, the existence of publishing consortia of many types is confirmed by the examinations above of individual groups that found them all to be named and clearly identifiable entities, or at least collections of papers within departments or research groups.

Whilst the physics consortia seem to have been formed because of the expense and perhaps complexity of the equipment required for specific tasks, the health-related consortia mainly reflect the need to systematically gather large quantities of diverse high-quality human-related (life-related in the remaining cases) data to have sufficient statistical power or variety to investigate a health issue. As with the GWAS case discussed above, some health issues cannot be resolved on a small scale or may give statistically misleading answers from repeated small-scale experiments, so combining data is essential for progress.

The publishing consortia found have research impact that tends to be substantially, and almost universally, above the world average for the fields and years in which they published. This may be due to the consortia typically involving economically advanced nations (e.g., Sud & Thelwall, 2016) or the increased likelihood to attract self-citations (Bornmann, 2017; van Raan, 1998) from the large teams but, from the example examined, it seems more likely that these teams produce research that is valuable to their disciplines, justifying the human and financial resources that seem to have been invested in many of them. Nevertheless, the results exclude all consortia that did not create at least three Scopus-indexed consortia and might (if they had no other goals) be characterised as failing, so the conclusions should not be extrapolated to nonpublishing consortia.

From a scientometric perspective, to evaluate the impact of the work of individual academics or departments from their citations it is useful to be able to attribute an appropriate share of an output to its contributors. This is complicated by publishing collaborations, especially if partial or full alphabetical ordering is used. For example, if five different groups might have produced five competing analyses of a single medical issue (e.g., whether green tea affects cancer risks) but instead publish one more powerful combined study, then four fewer papers would be published and the combined paper would have many authors. Presumably economies of scale would apply (e.g., one analysis instead of five; one study design instead of five), so that the total amount of work for the combined paper would be less than five times the amount for any individual paper. Whilst the efficiency and power improvements are good for science, a scientometric attempt to share credit for a single paper (e.g., with fractional author counting) would be unfair in this context. Conversely, for a publishing consortium that rewards participants that manage data gathering but are otherwise not active, it might be reasonable to assign fractional value (Hagen, 2013) to the less active participants. For the previously noted biomedical phenomenon of middle author alphabetical ordering (Mongeon et al., 2017), the middle authors (if accurately identified) could potentially be awarded equal but lesser credit to the remaining authors. Thus, a scientometric exercise would need to make a reasoned judgement about how to deal with contributions from large publishing consortia. For health-related consortia, it seems to be an ethical necessity to ensure that participation in collaborations that aid public health should not be discouraged by under-rewarding participants, so evaluators should be extremely careful when considering this option.

The author has no competing interests.

No funding has been received.

The key data behind the results, including a list of consortia, is on Figshare (https://doi.org/10.6084/m9.figshare.8214050).