Artificial intelligence (AI) is widely regarded as one of the most promising technologies for advancing science, fostering innovation, and solving global challenges. Recent years have seen a push for teamwork between experts from different fields and AI specialists, but the outcomes of these collaborations have yet to be studied. We focus on approximately 15,000 papers at the intersection of AI and COVID-19—arguably one of the major challenges of recent decades—and show that interdisciplinary collaborations between medical professionals and AI specialists have largely resulted in publications with low visibility and impact. Our findings suggest that impactful research depends less on the overall interdisciplinary of author teams and more on the diversity of knowledge they actually harness in their research. We conclude that team composition significantly influences the successful integration of new computational technologies into science and that obstacles still exist to effective interdisciplinary collaborations in the realm of AI.