Latmétricas: Special issue on developments of S&T indicators in Latin America

In recent years, Latin America has deepened its reflections on scientific research assessment. Different agendas have emerged from the debates between governments and academics at the national and regional levels. Researchers from academic institutions have shared their perspectives on various initiatives, such as FOLEC¹ (Latin American Forum for Scientific Assessment from CLACSO) and RICYT² (the Latin American Network of Indicators in Science and Technology). Other institutions also have advanced proposals for new directions on research assessment, such as LA Referencia³ (a network of Open Access repositories), Latindex⁴ (a catalogue of journals), and the regional bibliographic databases of SciELO⁵ and Redalyc⁶, as well as other local institutions such as governments and universities. Of course, those initiatives have been related to the Global North’s initiatives such as DORA, the Leiden Manifesto, the Metric Tide, INorms, and other proposals linked to responsible metrics.

In general, these initiatives capture the struggles between the criteria of accountability proposed by national governments and the social realities experienced by scientific researchers in their everyday activities to acquire funds, make scientific discoveries or support the development of their respective countries. These struggles also reflect the long-standing discussions about the organization of scholarly communication systems that are composed mainly of local journals with a diamond open access model—far away from commercial dynamics, the need to diversify the affiliations of researchers, most of them affiliated to universities, and the strategies to avoid the pressures of the evaluation system associated with publication in journals with high APC costs for Latin America.

This situation has drawn the attention of researchers to scientific evaluation, leading them to propose new evaluation methods and indicators that are in accordance and appropriate to their social realities. There have been two relevant programs. The first is the scientometric and bibliometric community assembled around the IDICT Congress (Instituto de Información Científica y Tecnológica) celebrated in Cuba from 2002 to 2018. Its last version was a Symposium on Quantitative and Qualitative Studies of Science and Technology held in Mexico in 2019. The other program is Latmetrics⁷, dedicated to discussing altmetric instruments in 2018 and 2019 in Brazil. Both experiences showed advances on assessment issues and promoted the emergence of new programs, such as geohistoriometric tools (Flores-Vargas, Vitar-Sandoval et al., 2018)⁸, changes in the instruments employed by regional journals to deliver their performance, and new ways to be accountable for different groups, organizations and collectives.

In 2021, the academic committees of both conferences agreed to merge Latmetrics and the scientometric symposium into a single regional conference: Latmétricas⁹. This conference brought together more than 500 practitioners and researchers from every corner of Latin America and the Caribbean. There were sessions on new metrics, measuring open access effects on productivity and impact, the relationship between gender and scientific assessment, STI policies and assessment by regional governments, geohistoriometric indicators to evaluate the historical trajectories of scientific production, and assessment of infrastructure capacities. The conference was also important to the editorial teams of scientific journals, as the Latin American Association of Scientific Editors (ALAEC)¹⁰ was launched there, under the auspices of the editors’ associations from Colombia¹¹, Brazil¹², and Uruguay¹³, respectively.

Besides the vibrant enthusiasm of debates between classical scientometric analysis and new analytical possibilities, it was possible to witness significant advances in platforms and proposals to retrieve regional information from different databases and produce indicators based on this information: Dialnet metrics¹⁴, OLIVA¹⁵, ImpactU¹⁶, and RICyT indicators¹⁷. These platforms are good news for the region, as they enable the region to become more independent of the rigid schemes of global platforms, which do not include all of the production relevant to the region, restricting themselves mainly to publications from commercial journals. One good example of these conceptual, methodological, and technological developments is the advances in recognizing research products resulting from research in the field of arts (different from the experimental and empirical fields of research), also called research-creation, a novel concept for scientometricians, who need to develop metrics for those types of research results. One of the panels was dedicated to this critical matter. This panel discussed ways to assess products from the field of arts research¹⁸.

Another workshop was focused on building new visions of quantitative science studies. The panelists talked about the usefulness of classic indicators and the need for new frameworks to understand science in its dynamic relationship with its environment, responsible metrics, and the accountability of science. Finally, one of the closing panelists, Juan Pablo Alperin, drew attention to the essential changes that have been taking place in Latin America in the last few years. The region has evolved from a small group of practitioners interested in scientometrics developments to a larger and stronger collective with many forward-looking proposals.

This special issue presents a small selection of contributions from Latmétricas 2021. It includes the following articles:

• Mapping the use of Google Scholar in bibliometric or scientometric studies: A systematic and bibliometric review by Fabiana Andrade Pereira and Rogerio Mugnaini.

• The management of scientific and technological infrastructures: The case of the Mexican National Laboratories by Leonardo Munguía, Juan Escalante, and Eduardo Robles Belmont.

• The transformation of medical research in Mexico. A structural analysis of thematical domains, institutional affiliations, authors’ cohorts, and possible correlations by Matías Milia, Claudia Gonzalez-Brambila, Ángel Lee, and José Ponce Sánchez.

• Indicators of research circulation: Localization and internationalization under scrutiny. An exploratory case study of the Cuyo Manual in Argentina by Víctor Algañaraz, Flavia Prado and, María Pía Rossomando.

Mugnaini and Andrade present a map of the evolution of quantitative science studies using Google Scholar. They show the global landscape highlighting the role played by some researchers in Latin America. Google Scholar is one of the platforms to follow the citation counting of different kinds of academic documents and the diversity of sources citing them, expanding the possibilities of assessment and research on science.

Munguía et al. present some tools to assess performance and leadership in Mexican National Laboratories. Algarañaz et al. show how the Cuyo Manual can help evaluate the relationships between the university and its environment. Both papers show possible paths to address research assessment by considering the particularities of local and national contexts. One of the most significant criticisms of scientometric models is the lack of diversity of indicators and the partial viewpoint that can result from such a narrow analysis. The organizational-level analysis proposed by these papers provides some new horizons to the debate. Note that Algarañaz et al. have worked on the Cuyo Manual to standardize these indicators of localization, in the wake of a series of RICyT Manuals of regionally contextualized indicators, such as the Manual of Valencia¹⁹ of indicators of knowledge exchange.

Finally, Milia et al. offer a model to explore the evolution of a scientific community in a national context. As opposed to the strategies of scientific rankings (which normalize their indicators according to citation counts and impact measures), they evaluate the field of medical studies according to the capacities developed to introduce communities and researchers to those communities. Those significant developments call attention to regional advances that can give better information to decision makers.

This issue also includes an interview with Argentinian sociologist Fernanda Beigel, professor at the National University of Cuyo, Argentina, who shares her expertise and views about how Latin America, where open science has traditionally been practiced by academics, is dealing with global changes on policies towards more openness.

We hope that this special issue helps to introduce some relevant developments in quantitative science studies from Latin America and the Caribbean. Research and policy agendas are shifting, and this issue presents some of the new directions followed in Latin America.