Although funding is essential for some types of research and beneficial for others, it may constrain academic choice and creativity. Thus, it is important to check whether it ever seems unnecessary. Here we investigate whether funded U.K. research tends to be higher quality in all fields and for all major research funders. Based on peer review quality scores for 113,877 articles from all fields in the U.K.’s Research Excellence Framework (REF) 2021, we estimate that there are substantial disciplinary differences in the proportion of funded journal articles, from Theology and Religious Studies (16%+) to Biological Sciences (91%+). The results suggest that funded research is likely to be of higher quality overall, for all the largest research funders, and for 30 out of 34 REF Units of Assessment (disciplines or sets of disciplines), even after factoring out research team size. There are differences between funders in the average quality of the research supported, however. Funding seems particularly associated with higher research quality in health-related fields. The results do not show cause and effect and do not take into account the amount of funding received but are consistent with funding either improving research quality or being won by high-quality researchers or projects.

National research evaluation initiatives and incentive schemes choose between simplistic quantitative indicators and time-consuming peer/expert review, sometimes supported by bibliometrics. Here we assess whether machine learning could provide a third alternative, estimating article quality using more multiple bibliometric and metadata inputs. We investigated this using provisional three-level REF2021 peer review scores for 84,966 articles submitted to the U.K. Research Excellence Framework 2021, matching a Scopus record 2014–18 and with a substantial abstract. We found that accuracy is highest in the medical and physical sciences Units of Assessment (UoAs) and economics, reaching 42% above the baseline (72% overall) in the best case. This is based on 1,000 bibliometric inputs and half of the articles used for training in each UoA. Prediction accuracies above the baseline for the social science, mathematics, engineering, arts, and humanities UoAs were much lower or close to zero. The Random Forest Classifier (standard or ordinal) and Extreme Gradient Boosting Classifier algorithms performed best from the 32 tested. Accuracy was lower if UoAs were merged or replaced by Scopus broad categories. We increased accuracy with an active learning strategy and by selecting articles with higher prediction probabilities, but this substantially reduced the number of scores predicted.

Research coauthorship is useful to combine different skill sets, especially for applied problems. While it has increased over the last century, it is unclear whether this increase is universal across academic fields and which fields coauthor the most and least. In response, we assess changes in the rate of journal article coauthorship 1900–2020 for all 27 Scopus broad fields and all 332 Scopus narrow fields. Although all broad fields have experienced reasonably continuous growth in coauthorship, in 2020, there were substantial disciplinary differences, from Arts and Humanities (1.3 authors) to Immunology and Microbiology (6 authors). All 332 Scopus narrow fields also experienced an increase in the average number of authors. Immunology and Classics are extreme Scopus narrow fields, as exemplified by 9.6 authors per Journal for ImmunoTherapy of Cancer article, whereas 93% of Trends in Classics articles were solo in 2020. The reason for this large difference seems to be the need for multiple complementary methods in Immunology, making it fundamentally a team science. Finally, the reasonably steady and universal increases in academic coauthorship over 121 years show no sign of slowing, suggesting that ever-expanding teams are a central part of current professional science.

This article assesses the balance of research concerning women and men over the past quarter century using the crude heuristic of counting Scopus-indexed journal articles relating to women or men, as suggested by their titles or abstracts. A manual checking procedure together with a word-based heuristic was used to identify whether an article related to women or men. The heuristic includes explicit mentions of women and men, implicit mentions, and a set of gender-focused health issues and medical terminology. Based on the results, more published articles now relate to women than to men. Moreover, more than twice as many articles relate exclusively to women than exclusively to men, with the ratio increasing from 2.16 to 1 in 1996 to 2.25 to 1 in 2020. Monogender articles mostly addressed primarily female health issues (maternity, breast cancer, cervical cancer) with fewer about primarily male health issues (testicular cancer, pancreatic cancer, health needs of men who have sex with men). Some articles also explicitly addressed gender inequality, such as empowering female entrepreneurs. The findings suggest that the androcentrism of early science has eroded in terms of research topics. This apparent progress should be encouraging for women researchers and society.

Scientometric research often relies on large-scale bibliometric databases of academic journal articles. Long-term and longitudinal research can be affected if the composition of a database varies over time, and text processing research can be affected if the percentage of articles with abstracts changes. This article therefore assesses changes in the magnitude of the coverage of a major citation index, Scopus, over 121 years from 1900. The results show sustained exponential growth from 1900, except for dips during both world wars, and with increased growth after 2004. Over the same period, the percentage of articles with 500+ character abstracts increased from 1% to 95%. The number of different journals in Scopus also increased exponentially, but slowing down from 2010, with the number of articles per journal being approximately constant until 1980, then tripling due to megajournals and online-only publishing. The breadth of Scopus, in terms of the number of narrow fields with substantial numbers of articles, simultaneously increased from one field having 1,000 articles in 1945 to 308 fields in 2020. Scopus’s international character also radically changed from 68% of first authors from Germany and the United States in 1900 to just 17% in 2020, with China dominating (25%).

Two partly conflicting academic pressures from the seriousness of the Covid-19 pandemic are the need for faster peer review of Covid-19 health-related research and greater scrutiny of its findings. This paper investigates whether decreases in peer review durations for Covid-19 articles were universal across 97 major medical journals, as well as Nature , Science , and Cell . The results suggest that on average, Covid-19 articles submitted during 2020 were reviewed 1.7–2.1 times faster than non-Covid-19 articles submitted during 2017–2020. Nevertheless, while the review speed of Covid-19 research was particularly fast during the first 5 months (1.9–3.4 times faster) of the pandemic (January–May 2020), this speed advantage was no longer evident for articles submitted in November–December 2020. Faster peer review was also associated with higher citation impact for Covid-19 articles in the same journals, suggesting it did not usually compromise the scholarly impact of important Covid-19 research. Overall, then, it seems that core medical and general journals responded quickly but carefully to the pandemic, although the situation returned closer to normal within a year.

Formal assessments of the quality of the research produced by departments and universities are now conducted by many countries to monitor achievements and allocate performance-related funding. These evaluations are hugely time consuming if conducted by postpublication peer review and are simplistic if based on citations or journal impact factors. I investigate whether machine learning could help reduce the burden of peer review by using citations and metadata to learn how to score articles from a sample assessed by peer review. An experiment is used to underpin the discussion, attempting to predict journal citation thirds, as a proxy for article quality scores, for all Scopus narrow fields from 2014 to 2020. The results show that these proxy quality thirds can be predicted with above baseline accuracy in all 326 narrow fields, with Gradient Boosting Classifier, Random Forest Classifier, or Multinomial Naïve Bayes being the most accurate in nearly all cases. Nevertheless, the results partly leverage journal writing styles and topics, which are unwanted for some practical applications and cause substantial shifts in average scores between countries and between institutions within a country. There may be scope for predicting articles’ scores when the predictions have the highest probability.

While funders increasingly request evidence of the societal benefits of research, all academics in the UK must periodically provide this information to gain part of their block funding within the Research Excellence Framework (REF). The impact case studies produced in the UK are public and can therefore be used to gain insights into the types of sources used to justify societal impact claims. This study focuses on the URLs cited as evidence in the last public REF to help researchers and resource providers to understand what types can be used and the disciplinary differences in their uptake. Based on a new semiautomatic method to classify the URLs cited in impact case studies, the results show that there are a few key online types of source for most broad fields, but these sources differ substantially between subject areas. For example, news websites are more important in some fields than others, and YouTube is sometimes used for multimedia evidence in the arts and humanities. Knowledge of the common sources selected independently by thousands of researchers may help others to identify suitable sources for the complex task of evidencing societal impacts.

Quantile regression presents a complete picture of the effects on the location, scale, and shape of the dependent variable at all points, not just the mean. We focus on two challenges for citation count analysis by quantile regression: discontinuity and substantial mass points at lower counts. A Bayesian hurdle quantile regression model for count data with a substantial mass point at zero was proposed by King and Song (2019) . It uses quantile regression for modeling the nonzero data and logistic regression for modeling the probability of zeros versus nonzeros. We show that substantial mass points for low citation counts will almost certainly also affect parameter estimation in the quantile regression part of the model, similar to a mass point at zero. We update the King and Song model by shifting the hurdle point past the main mass points. This model delivers more accurate quantile regression for moderately to highly cited articles, especially at quantiles corresponding to values just beyond the mass points, and enables estimates of the extent to which factors influence the chances that an article will be low cited. To illustrate the potential of this method, it is applied to simulated citation counts and data from Scopus.

Information about the relative strengths of scholars is needed for the efficient running of knowledge systems. Because academic research requires many skills, more experienced researchers might produce better research and attract more citations. This article assesses career citation impact changes 2001–2016 for domestic researchers (definition: first and last Scopus journal article in the same country) from the 12 nations with most Scopus documents. Careers are analyzed longitudinally, so that changes are not due to personnel evolution, such as researchers leaving or entering a country. The results show that long-term domestic researchers do not tend to improve their citation impact over time but tend to achieve their average citation impact by their first or second Scopus journal article. In some countries, this citation impact subsequently declines. These longer-term domestic researchers have higher citation impact than the national average in all countries, however, whereas scholars publishing only one journal article have substantially lower citation impact in all countries. The results are consistent with an efficiently functioning researcher selection system but cast slight doubt on the long-term citation impact potential of long-term domestic researchers. Research and funding policies may need to accommodate these patterns when citation impact is a relevant indicator.

Researchers may be tempted to attract attention through poetic titles for their publications, but would this be mistaken in some fields? Although poetic titles are known to be common in medicine, it is not clear whether the practice is widespread elsewhere. This article investigates the prevalence of poetic expressions in journal article titles from 1996–2019 in 3.3 million articles from all 27 Scopus broad fields. Expressions were identified by manually checking all phrases with at least five words that occurred at least 25 times, finding 149 stock phrases, idioms, sayings, literary allusions, film names, and song titles or lyrics. The expressions found are most common in the social sciences and the humanities. They are also relatively common in medicine, but almost absent from engineering and the natural and formal sciences. The differences may reflect the less hierarchical and more varied nature of the social sciences and humanities, where interesting titles may attract an audience. In engineering, natural science, and formal science fields, authors should take extra care with poetic expressions in case their choice is judged inappropriate. This includes interdisciplinary research overlapping these areas. Conversely, reviewers of interdisciplinary research involving the social sciences should be more tolerant of poetic license.

The speed with which biomedical specialists were able to identify and characterize COVID-19 was partly due to prior research with other coronaviruses. Early epidemiological comparisons with Severe Acute Respiratory Syndrome (SARS) and Middle East Respiratory Syndrome (MERS), also made it easier to predict COVID-19’s likely spread and lethality. This article assesses whether academic interest in prior coronavirus research has translated into interest in the primary source material, using Mendeley reader counts for early academic impact evidence. The results confirm that SARS and MERS research in 2008–2017 experienced anomalously high increases in Mendeley readers in April–May 2020. Nevertheless, studies learning COVID-19 lessons from SARS and MERS or using them as a benchmark for COVID-19 have generated much more academic interest than primary studies of SARS or MERS. Thus, research that interprets prior relevant research for new diseases when they are discovered seems to be particularly important to help researchers to understand its implications in the new context.

Ongoing problems attracting women into many Science, Technology, Engineering and Mathematics (STEM) subjects have many potential explanations. This article investigates whether the possible undercitation of women associates with lower proportions of, or increases in, women in a subject. It uses six million articles published in 1996–2012 across up to 331 fields in six mainly English-speaking countries: Australia, Canada, Ireland, New Zealand, the United Kingdom and the United States. The proportion of female first- and last-authored articles in each year was calculated and 4,968 regressions were run to detect first-author gender advantages in field normalized article citations. The proportion of female first authors in each field correlated highly between countries and the female first-author citation advantages derived from the regressions correlated moderately to strongly between countries, so both are relatively field specific. There was a weak tendency in the United States and New Zealand for female citation advantages to be stronger in fields with fewer women, after excluding small fields, but there was no other association evidence. There was no evidence of female citation advantages or disadvantages to be a cause or effect of changes in the proportions of women in a field for any country. Inappropriate uses of career-level citations are a likelier source of gender inequities.

Current attempts to address the shortfall of female researchers in science, technology, engineering, and mathematics (STEM) have not yet succeeded, despite other academic subjects having female majorities. This article investigates the extent to which gender disparities are subject-wide or nation-specific by a first-author gender comparison of 30 million articles from all 27 Scopus broad fields within the 31 countries with the most Scopus-indexed articles 2014–2018. The results show overall and geocultural patterns as well as individual national differences. Almost half of the subjects were always more male (seven; e.g., Mathematics) or always more female (six; e.g., Immunology & Microbiology) than the national average. A strong overall trend (Spearman correlation 0.546) is for countries with a higher proportion of female first-authored research to also have larger differences in gender disparities between fields (correlation 0.314 for gender ratios). This confirms the international gender equality paradox previously found for degree subject choices: Increased gender equality overall associates with moderately greater gender differentiation between subjects. This is consistent with previous United States-based claims that gender differences in academic careers are partly due to (socially constrained) gender differences in personal preferences. Radical solutions may therefore be needed for some STEM subjects to overcome gender disparities.

The COVID-19 pandemic requires a fast response from researchers to help address biological, medical, and public health issues to minimize its impact. In this rapidly evolving context, scholars, professionals, and the public may need to identify important new studies quickly. In response, this paper assesses the coverage of scholarly databases and impact indicators during March 21, 2020 to April 18, 2020. The rapidly increasing volume of research is particularly accessible through Dimensions, and less through Scopus, the Web of Science, and PubMed. Google Scholar’s results included many false matches. A few COVID-19 papers from the 21,395 in Dimensions were already highly cited, with substantial news and social media attention. For this topic, in contrast to previous studies, there seems to be a high degree of convergence between articles shared in the social web and citation counts, at least in the short term. In particular, articles that are extensively tweeted on the day first indexed are likely to be highly read and relatively highly cited 3 weeks later. Researchers needing wide scope literature searches (rather than health-focused PubMed or medRxiv searches) should start with Dimensions (or Google Scholar) and can use tweet and Mendeley reader counts as indicators of likely importance.

Within academia, mature researchers tend to be more senior, but do they also tend to write higher impact articles? This article assesses long-term publishing (16+ years) United States (U.S.) researchers, contrasting them with shorter-term publishing researchers (1, 6, or 10 years). A long-term U.S. researcher is operationalized as having a first Scopus-indexed journal article in exactly 2001 and one in 2016–2019, with U.S. main affiliations in their first and last articles. Researchers publishing in large teams (11+ authors) were excluded. The average field and year normalized citation impact of long- and shorter-term U.S. researchers’ journal articles decreases over time relative to the national average, with especially large falls for the last articles published, which may be at least partly due to a decline in self-citations. In many cases researchers start by publishing above U.S. average citation impact research and end by publishing below U.S. average citation impact research. Thus, research managers should not assume that senior researchers will usually write the highest impact papers.

A research doctorate normally culminates in publishing a dissertation reporting a substantial body of novel work. In the absence of a suitable citation index, this article explores the relative merits of alternative methods for the large-scale assessment of dissertation impact, using 150,740 UK doctoral dissertations from 2009–2018. Systematic methods for this were designed for Google Books, Scopus, Microsoft Academic, and Mendeley. Fewer than 1 in 8 UK doctoral dissertations had at least one Scopus (12%), Microsoft Academic (11%), or Google Books citation (9%), or at least one Mendeley reader (5%). These percentages varied substantially by subject area and publication year. Google Books citations were more common in the Arts and Humanities (18%), whereas Scopus and Microsoft Academic citations were more numerous in Engineering (24%). In the Social Sciences, Google Books (13%) and Scopus (12%) citations were important and in Medical Sciences, Scopus and Microsoft Academic citations to dissertations were rare (6%). Few dissertations had Mendeley readers (from 3% in Science to 8% in the Social Sciences) and further analysis suggests that Google Scholar finds more citations, but does not report information about all dissertations within a repository and is not a practical tool for large-scale impact assessment.

Although explicitly labeled research questions seem to be central to some fields, others do not need them. This may confuse authors, editors, readers, and reviewers of multidisciplinary research. This article assesses the extent to which research questions are explicitly mentioned in 17 out of 22 areas of scholarship from 2000 to 2018 by searching over a million full-text open access journal articles. Research questions were almost never explicitly mentioned (under 2%) by articles in engineering and physical, life, and medical sciences, and were the exception (always under 20%) for the broad fields in which they were least rare: computing, philosophy, theology, and social sciences. Nevertheless, research questions were increasingly mentioned explicitly in all fields investigated, despite a rate of 1.8% overall (1.1% after correcting for irrelevant matches). Other terminology for an article’s purpose may be more widely used instead, including aims, objectives, goals, hypotheses, and purposes, although no terminology occurs in a majority of articles in any broad field tested. Authors, editors, readers, and reviewers should therefore be aware that the use of explicitly labeled research questions or other explicit research purpose terminology is nonstandard in most or all broad fields, although it is becoming less rare.

Initiatives addressing the lack of women in many academic fields, and the general lack of senior women, need to be informed about the causes of any gender differences that may affect career progression, including citation impact. Previous research about gender differences in journal article citation impact has found the direction of any difference to vary by country and field, but has usually avoided discussions of the magnitude and wider significance of any differences and has not been systematic in terms of fields and/or time. This study investigates differences in citation impact between male and female first-authored research for 27 broad fields and six large English-speaking countries (Australia, Canada, Ireland, New Zealand, the UK, and the USA) from 1996 to 2014. The results show an overall female first author citation advantage, although in most broad fields it is reversed in all countries for some years. International differences include Medicine having a female first author citation advantage for all years in Australia, but a male citation advantage for most years in Canada. There was no general trend for the gender difference to increase or decrease over time. The average effect size is small, however, and unlikely to have a substantial influence on overall gender differences in researcher careers.