Implementation of an Open Science Instruction Program for Undergraduates

In the Spring of 2018, the Library and the Department of Biology at the University of British Columbia's (UBC's) Okanagan Campus in Kelowna, Canada, embarked upon an undergraduate Open Science (OS) instructional program—a venture unique not only in Canada, but also in North America, as far as we are aware. This instructional program has since evolved into one wing of a larger Fostering OS initiative spanning all of UBC.

The initiative and growth of the undergraduate instructional program was made possible through the efforts of a number of champions at UBC who succeeded in achieving recognition of OS instruction as a strategic institutional priority. The program has seen planning, aided by relationships between the Library and the Department of Biology, for a multi-module program to be integrated into key courses across all four years of the undergraduate Biology degree. Starting in the fall of 2018, approximately 60 students in two third-year biology classes tested a pilot module to evaluate interest and means of implementing content that supported, without altering, existing curricula and class assignments. After this initial success and the subsequent hiring of a dedicated OS Librarian to support this work in the Spring of 2019, 328 students participated in a voluntary pilot module called Open Science 101 (OS 101) in two first-year Biology classes in Fall 2019.

OS 101 defines OS as a movement to make products of all stages of the research lifecycle (e.g., scientific research, data, and publications) more accessible and provide transparency, openness, integrity, and reproducibility to scientific studies. The module provides an overview of the practical challenges of conducting reproducible research, the societal impact of irreproducible research, and philosophical and ethical issues surrounding OS. Assessment of the module revealed that a major attitudinal shift had occurred among participating students: They were convinced of the importance of a shift toward OS. We will look at the background and context surrounding our OS instruction; the planning, delivery, and evaluation of OS 101; and future plans for the undergraduate program.

For readers to understand the novelty of the UBC Okanagan program fully, we must situate this program within the Canadian context. Although Canada has lagged behind Europe in its promotion of OS, between 2015 and 2020 the Canadian government introduced measures to promote Open Access [1], articulated a strategic plan to foster OS practices among federal government researchers and recipients of federal government funding [2], and participated in a UNESCO recommendation on the development of an inclusive OS policy framework [3]. These developments affected university researchers funded by Canada's TriAgency, three government bodies that support scholarly research in Canada: the National Science and Engineering Council of Canada (NSERC), the Canadian Institutes of Health Research (CIHR, which had implemented an open access policy at an early date), and the Social Sciences and Humanities Research Council of Canada (SSHRC).

These important efforts, preceded by similar initiatives in the United States and Europe, emphasize infrastructure, funding, and other supports targeted largely to the public sector and to practicing researchers or graduate students. But OS requires a culture of change that reaches beyond these domains, one that prepares future citizens and future researchers. It is with this understanding that we undertook our instructional program.

A recent environmental scan confirmed that no other Canadian universities are delivering undergraduate instruction in OS [4]. Open scholarship instruction in Canadian post-secondary institutions appears to concentrate on three areas:

• Open Educational Resources (OER) as learning tools for students;

• Open Access databases;

• Research Data Management (primarily for graduate students).

Although we found no evidence that any other universities were considering an undergraduate OS program on the scale of UBC's, we know anecdotally that many individual instructors at a variety of universities are integrating aspects of OS into their labs and classrooms. At the very least, this suggests an appetite for a shift in how we teach the scientists of the future.

This landscape, while perhaps more integrated at the institutional policy level, appears very much mirrored in the European Union. While there is little evidence of the degree of undergraduate integration we are attempting at UBC Okanagan, Horizon Europe, the 9th EU framework program for research and innovation, specifically names OS as one of its three pillars [5]. In accordance with the new program, many European universities are beginning to put comprehensive OS policies in place. One example of this is the Delft University of Technology (TU Delft) in the Netherlands. The TU Delft Strategic Plan OS: 2020–2024 outlines plans to implement coordinated OS training modules for instructors, research staff, and students [6]. However, education for students at the undergraduate level is not mentioned.

Given the nascent nature of the OS movement, scant literature on best practices in library OS information literacy instruction exists. Extant publications on this topic (e.g., [7]) tend to focus on information literacy instruction of researchers pursuing graduate studies—that is, EU Level 2 or higher. Some exceptions include Ayris and Ignat [8], who suggested involving undergraduates (EU Level 1 students) in Citizen Science projects; Button [9], who established an undergraduate Psychology consortium in Britain to introduce rigorous research practices to writers of undergraduate theses, and Jekel et al. [10], who incorporated replication projects into Honors thesis requirements.

Efforts to promote best practices in OS at UBC began in earnest in 2015, under the direction of an assistant professor in the Biology Department of UBC Okanagan. Over the first two years efforts were largely focused on (i) identifying and reaching out to the administrative units that would be key to establishing OS as a core tenet of UBC's research agenda, and (ii) building a grassroots network of OS advocates. Key partners in this early work included the Vice-Provost and Association Vice President Academic of UBC Vancouver; UBC Okanagan's Associate Vice-Principal and Associate Vice-President, Research and Innovation; and UBC Okanagan's Chief Librarian and Associate Provost, Learning Services. As well, the movement was supported by UBC's Advanced Research Computing group (ARC), who recognized at the outset how OS best practices aligned with their core mandate of facilitating research through highperformance computing infrastructure and support.

Despite support from upper administrative positions and ARC, one challenge remained: All of this work was happening “off the sides of desks”, and so progress was at the mercy of already full work schedules. Nevertheless, between 2017 and 2019, a new Southern Medical Program Librarian, in conjunction with ARC and the aforementioned Biology professor, succeeded in launching Canada's first instance of OS Framework Institutions at UBC^①.

In 2019, the UBC Okanagan OS advocates secured $100K (CAD) in strategic funding for OS over two years. Their proposal included short- and long-term objectives within both the teaching and research realms, the former being described below under Sections 4 and 5. And the funds supported the hiring of a part-time OS Librarian who could focus on building the OS instructional program, and enable team members to broaden the reach of instructional seminars and workshops.

Why target undergraduates? Johannes Vogel, director general of the Museum für Naturkunde and Professor of Biodiversity and Public Science at Humboldt University in Berlin, recognizes the power that young people have to mobilize movements and suggests that they can help to open up science. In his article “Scientists need to learn from the young”, Johannes Vogel states,

Students—our future scientists, professors, practitioners, and policy makers—have long been catalysts of social and political change. As such, the transformation of academic culture we desire requires cultivating in our undergraduates an appreciation of OS values and familiarity with transparent and reproducible practices.

Introducing OS values and practices as early as possible has numerous benefits, both for the prospective researchers and for the entire undergraduate population. First, by participating in OS training in the first year, students are better equipped to critically evaluate the materials and methods they are exposed to in subsequent years. Our qualitative experience is that students feel empowered by the early OS training, and therefore show greater interest and engagement when presented, for example, with case studies or primary research articles. We are hopeful that this applies more broadly, outside the classroom, where students need to make informed decisions as active members of society.

Second, values and practices that are introduced early can be more fully developed and reinforced throughout the undergraduate curriculum, and this in turn increases the likelihood that the practices become habits. This is especially beneficial to students who aim to pursue science careers. Crucially, we have seen how these habits can propagate up through research labs to principal investigators, and ultimately precipitate wholesale changes to research workflows.

Lastly, by their final year, students will be well equipped to apply OS best practices to research endeavors such as honors research projects and capstone projects. We therefore anticipate substantive improvements in the quality of undergraduate-led research, which is a focus of many undergraduate programs.

Our initiative at the undergraduate level is two-fold. The first is to build a series of learning objects that will allow faculty and lab managers to adapt current curricula and assignments to a model that integrates the fundamental principles and practices of OS in the instruction and evaluation of their existing teaching practices. The goal here is not to change what is being taught, but how undergraduate students are taught and evaluated. The second is to build a test bed of integration working closely with instructors in the Biology Department at UBC's Okanagan campus. This integration allows us to identify where discipline modularity in these objects is best situated and at what levels it makes the most sense to introduce various concepts related to OS. This in turn is allowing us to build better learning objects that can be adapted for different disciplines.

At the outset, our undergraduate program sought to give students a grounding in several categories of best practices that facilitate reproducibility, replicability, and inclusiveness in science. These include open access, transparency at all stages of the research cycle, data management, citizen engagement, and applications of technology to OS. The program is designed to scaffold from introductory courses in first and second year to more advanced courses in third year; it will culminate in a fourth-year replication project. We intend the module templates to be flexible enough to integrate into courses in a variety of disciplines.

The plan, and challenge, is to integrate this education smoothly into the existing course content in such a way that it becomes just another part of the experience of completing a science degree, and minimizes the burden to instructors.

Our OS library information literacy program for undergraduates aims to provide templates and resources to enable instructors to weave OS principles and practices into the existing curriculum in such a way that Open practices become as automatic to students as putting on a lab coat.

In creating a comprehensive undergraduate OS program that can be implemented in diverse disciplinary settings, we must consider that professors and instructors have varying levels of comfort with and knowledge about OS and its associated technologies and that OS practices look different in different subject contexts. Therefore, we plan to introduce some initial modules and later, assuming good results, open a dialog with faculty at large about our ideas for a comprehensive program. In the list below, italics indicate a module that has been implemented; as aspects of our work are released, they will be found on our website, https://openscience.ubc.ca/.

First Year

• OS 101: Principles of OS (Definition, Rationale, Values, Benefits, and Constraints)

• OS 102: Reproducibility through the Research Cycle; critical evaluation of scientific papers; basic data management

Second Year

• OS 201: Reproducible analysis and literate programming using tools such as RMarkdown

• OS 202: Modular Lab Components: Review of labs in stages (similar to the preregistration process)

• OS 203: Digitization of lab books and field notes, & intro to electronic lab books

Third Year

• OS 301: Use and Creation of Open Data Collections with regard for security and privacy constraints

• OS 302: Science communication, Citizen Science, or Participatory Research

• OS 303: Peer review

• OS 304: Advanced prevention of Questionable Research Practices

Fourth Year

• OS 401: Replication Project (optional, at the Department's discretion)

Since the Associate Professor in Biology had cultivated close relationships with his colleagues in Biology and had planted the seeds of change by speaking to them of the benefits of OS practices, this department seemed a natural setting in which to introduce our pilot project. OS 101 was to provide a general introduction to OS concepts and fits naturally into first year. Moreover, the lab manager for first-year Biology showed receptivity to the idea. Thus, the team built a module to be implemented in the Lab components of Biology 116 (for Biology majors) and Biology 122 (for majors in other subjects).

Our pilot was designed to give students an overview of the practical challenges of conducting reproducible research, the societal impact of irreproducible research, and philosophical and ethical issues surrounding OS.

Our intent was for students to be able to explain the following on completion of the unit:

• The nature of scientific knowledge

• The science-society relationship

• The replication crisis

• Core values of OS

• Benefits of OS

• Barriers to the realization of OS ideals

Both students and instructors face considerable demands on their time. To avoid creating extra work for instructors, we introduced OS 101 in the lab component of the course, rather than the lecture component, and made it a self-contained module with short unit quizzes that were marked automatically. Assessments at the start and end of OS 101 contained some open questions, but these were to be marked by the OS Librarian, not by the lab manager or teaching assistants.

To avoid burdening the students overly, the lab manager and the OS Librarian arranged for the students to receive extra participation marks for attempting the quizzes; students were not penalized if they did not participate in the module. We chose a medium in which the module could be completed online and at the student's own pace: we mounted it on Canvas, UBC's Learning Management System (see Figure 1).

We tried to provide examples that the students could relate to based on their own experience: A case in point is this example relating the desirability of transparency in practices to a student's wish to know how and why marks are assigned for an assignment:

In making the material accessible to first-year students, it was imperative to consider their limited familiarity with (a) research data collection, management, and sharing and (b) statistics. Note that first-year students, the target of this module, are not yet engaged in collecting or sharing their own data. As such, we did not refer specifically to initiatives such as FAIR data principles [13]. However, data are mentioned as being critical for reproducibility and replicability, and later modules will discuss FAIR data principles.

In a similar vein, we had to consider that most students would not have taken a university course in statistics at this point in their careers. Consequently, when mentioning questionable research practices, we did not refer to topics such as p-values or margins of error: We kept our explanations more general. For instance, we defined replication as “thorough repetition” and skirted the issue of statistical significance in our exploration of publication bias and questionable research practices.

Finally, we could not take for granted that the students coming into first year remembered learning about the nature of scientific knowledge and the scientific method, or had in fact received such instruction in high school. Students come from various educational backgrounds. Therefore, in the first unit, we laid a foundation for the introduction of OS by first examining how scientific knowledge is built.

At the end of the term, students had the option to answer open-ended questions through an exit survey mounted on Canvas. We posed the following questions:

• If you had to sum up your view on OS, what would you say?

• In your opinion, what is the most important benefit that OS can bring?

• Which aspects of the OS movement discussed this semester resonate the most with you?

As noted in our presentation at the OSC2020 Conference in Berlin in March, the students showed a keen interest in OS principles, and professed, by the end of the course, both positive attitudes toward OS practices and an appreciation of the ways in which OS can improve science and society. The top benefits of OS as cited by the students were scientific progress, transparency, and collaboration.

Seventy-two students responded to the survey, and many of them wrote inspiring and moving words, demonstrating that the module had indeed engendered in them an appreciation of OS as a positive movement for scientific research and for society as a whole [14, 15]. The following are two examples of the feedback [14, 15].

In the last year and half working on this project, we have been struck by the value of the combined support provided to this initiative by faculty, the Library, the Provost's office, our Advanced Research Computing Department, and the Strategic Initiatives and Planning office at UBC. While the Library has taken the lead on the promotion and development of these undergraduate materials, lacking this support from any one of these bodies would have made our efforts much more challenging.

Although we created detailed content for OS 101 and 102, our role in OS instruction is changing to one of providing templates, resources, and consulting advice to instructors and lab managers interested in introducing OS principles and practices into their classrooms and labs. As such, we've been introducing alternative examples that speak to course areas other than Biology and examples related to COVID-19. Producing templates for second-year modules lies just ahead on the horizon.

The ability to make discrete connections across the research life cycle and to critically identify sources of bias underpins OS principles and practices. Working with undergraduate students and liaising with faculty and lab managers, our OS Librarian introduces students to this landscape of scholarly communication in the context of OS.

In the summer and fall of 2021, we will be working with the Biology Lab Managers to tie OS 101 and 102 more closely to the curriculum and to introduce lab exercises that specifically link to the content. Lab teaching assistants will also receive an OS orientation. OS will make up a larger portion of a student's overall course mark.

At this point, whether the current funding of an OS Librarian position through the President's Excellence fund will be renewed remains unknown. The COVID-19 pandemic has affected our project both by introducing uncertainty into the university's budget forecast and by raising awareness of the boons that OS practices such as data sharing and open-source medical equipment designs can bring. Our qualitative results show that it is possible to foster change in the next generation of scientists with a relatively small investment of human resources, and given the policy changes occurring in the Canadian government and the international context, there seems little doubt that OS will continue to grow in importance.

S. Hanna ([email protected]) led the writing of the paper, with contributions and revisions by M. Vis-Dunbar ([email protected]) and J. Pither ([email protected]).

We are most grateful for the support that Eric Eich, Phil Barker, and Heather Berringer have provided to our OS efforts and to the Fostering OS at UBC initiative, in particular. In addition, we would like to express our sincere thanks to the Leibniz Research Alliance OS for inviting us to present at the OS Conference in Berlin in March 2020, opening the door to the publication of this paper.