For a long time, citation counts have been used to measure scientific impact or quality. Do such measures align with researchers’ assessments of the quality of their works? In this study, we address this issue by decomposing the research quality concept into constituent parts and analysing their correspondence with citation measures. The focus is on individual publications, their citation counts and how the publications are rated by the authors themselves along quality dimensions. Overall, the study shows a statistically significant relationship for all dimensions analysed: solidity, novelty/originality, scientific importance and societal impact. The highest correlation is found for scientific importance. However, it is not very strong, but we find distinct gradients when publications are grouped by quality scores. This means that the higher the researchers rate their works, the more they are cited. The results suggest that citation metrics have low reliability as indicators at the level of individual articles, while at aggregated levels, the validity is higher, at least according to how authors perceive quality.

Peer Review

https://publons.com/publon/10.1162/qss_a_00241

This content is only available as a PDF.

Author notes

Handling Editor: Ludo Waltman

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. For a full description of the license, please visit https://creativecommons.org/licenses/by/4.0/legalcode.

Article PDF first page preview

Article PDF first page preview