“Dancing with Atoms”: A Tribute to Sheila Tinney

According to UNESCO’s Institute for Statistics, less than 30% of the world’s researchers are women [1]. In recent years, governments and funders have encouraged and supported active involvement of women in STEM (Science, Technology, Engineering, and Mathematics). As part of that strategy, we recognize and celebrate pioneering women with distinguished careers in STEM. One such woman was Irish mathematical physicist Sheila Tinney (1918–2010), the first Irish-born and -raised woman to receive a doctorate in the mathematical sciences [2]. She completed her 1941 PhD from the University of Edinburgh on the stability of crystal lattices [3] in just two years under the supervision of Nobel laureate (1954, Physics) Max Born. Sheila contributed across a wide range of scientific topics, including crystal lattices, wave mechanics, quantum electrodynamics, cosmic radiation, and meson theory [4–8]. Throughout her career, she collaborated with top scientists such as Erwin Schrödinger, Hideki Yukawa (both Nobel laureates), and Walter Heitler, at a time when STEM fields were almost exclusively male (Fig. 1).

STEAM refers to the close association of the Arts to STEM disciplines. Music was of great importance to Sheila, and she was a gifted and dedicated amateur musician, the interest having been passed on to her from her mother. She in turn passed this love of music on to her son Hugh Tinney, who went on to become an award-winning concert pianist and professor at the Royal Irish Academy of Music.

In 2018, Science Foundation Ireland awarded academic geriatrician (and trained musician) Roman Romero-Ortuno the President of Ireland Future Research Leaders Award. This award funded the study of physiological signals to help in the early detection of frailty in older adults [9]. This interdisciplinary research program, called “FRAILMatics,” recruited researchers from physics, mathematics, and medicine, and involved the setup of a new high-performance computer (HPC) cluster in Trinity College Dublin (TCD). As per TCD tradition, the new HPC system was named in honor of a distinguished scientist. On the suggestion of Louise Newman, biomedical engineer in The Irish Longitudinal Study on Ageing (TILDA), it was named “Tinney” in Sheila’s honor [10,11].

To mark these developments and celebrate the work of both Sheila and Hugh, we created a short dynamic visualization titled “Dancing with Atoms,” combining a representation of Sheila’s groundbreaking work on crystal lattice structures with her son’s piano music, with the main processing of the visualization performed on the Tinney HPC cluster. In the present work, we used the piece of music “Prelude in C major” (BWV 846i) from The Well-Tempered Clavier, Book 1, performed by Hugh [12]. Three frequency bands were isolated: low, mid, and high. The magnitude change for these frequency bands was normalized to produce Cartesian coordinates (x, y, and z), which were then used as inputs to an in-house-developed atomic crystal lattice simulation, allowing the atoms to move (“dance”) in 3D space in time with the music.

Audio processing was performed using MATLAB (R.2020b; TheMathWorks, Inc.). The audio source was an MP3 file with a sampling rate of 48kHz, bit rate of 192 kbps, and length of 2 minutes and 40 seconds. All processing was performed using in-built MATLAB functions. Firstly, both channels of audio (stereo) were combined into a single vector. A fast Fourier transform (fft.m) was then used to convert this vector into frequency space, and the real component of the data was isolated. Three new vectors were then produced, which isolated three frequency bands: low frequency (“sub-base” to “lower midrange”; 16 Hz–500 Hz), midrange (500 Hz–2 kHz), and high frequency (“higher midrange” to “brilliance”; 2 kHz–20 kHz). All frequencies outside of these ranges were set to zero for each new vector. An inverse fast Fourier transform (ifft.m) was then used to convert the three vectors back into time signals. A quadratic regression smoothing function (smoothdata.m) was then applied, with a window of 48000 (corresponding to 1 sec. for 48 kHz data). All three vectors were then decimated to 60 Hz and normalized to produce Cartesian coordinates (x, y, and z), which were exported as a CSV file (along with a time vector) for use in the crystal lattice simulation.

The rendering software used for this project was Trinity Visualisation Suite 2.0 (TVS2). This software was developed by Research IT, formerly known as the Trinity Centre for High Performance Computing (TCHPC; TCD, Dublin, Ireland), as part of the Institute for Information Technology and Advanced Computing (IITAC) project, with the goal of being able to visualize VASP files for the Trinity Computational Chemistry group. Over the years, additional functionality has been added to this software, making it well suited for custom applications such as the ones examined in this project.

In this software, the rendering of a VASP dataset is based on the image of a single lattice cell. The dataset provides information about some of the atoms in the original cuboid, and it is up to the software to determine which ones will be in the border of the same cell as it considers the neighboring lattices. The bonds are then created by setting a distance interval that is larger than the van der Waals radii but is small enough that any pair of atoms within that distance must be bonded by either a covalent or hydrogen bond.

The atoms that were rendered can be, and often are, located at the boundary between two cells, so technically they belong to both (the boundary can occur between up to eight cells, if they are in one of the top or bottom corners of the lattice). Because of this, it was not possible to use a discrete number of copies of the lattice in each direction when choosing the magnitude of the movement. Instead, the distance to the original cell was used, so the distortion or perturbation became smaller the further the atoms were from the initial cell. Each one of these moved independently, which meant that all the atoms in the original cell moved with the original perturbation, with no scale or timestep distortion (i.e. all moving the same distance proposed by the current timestep), whereas the movement of atoms outside the original cell were affected by both perturbation and distortion. That way, the further away that an atom was from the original cell, the smaller the perturbation and the larger the applied distortion.

Bonds were considered links between two atoms within a predetermined distance interval. Thus, the movement of a particular bond was determined by the movements of the two atoms that it linked. We acknowledge that this approach risks not preserving scientific accuracy, as both the perturbations and distortions applied to them could potentially allow a pair of atoms to move away from each other beyond the distance that would determine a chemical bond between them. However, for the sake of maintaining the crystal structure of the hypothetical material used for this visualization, bonds that were calculated from the unperturbed, undistorted crystal structure were retained throughout the animation, regardless of the distance between atoms.

All atoms in the original cell were perturbed by the amount determined by the CSV file extracted from the input audio data, which contained four columns (timesteps and x, y and z displacements).

Next, the propagation of the perturbation to atoms in other cells was calculated. In this case, the perturbation was reduced compared to the original cell because of distance from the original cell. Since, as previously mentioned, an atom can belong to more than one cell, two arrays were created, atoms-Distance and atomsWeight. In atomsDistance the distance of each atom that was rendered to the original cell was stored. In atomsWeight, the applied weight to each atom was stored, depending on their corresponding value in atomsDistance. These weights were used to scale the perturbation of a particular atom, allowing for a smooth transition between cells. A time lag was also implemented as a function of distance from the original cell. The atomsDistance array was reused to determine how many timesteps to step back as the atoms got further away from the initial cell.

The atomic crystal lattice animation was then composited with the original audio source and the FRAILMatics logo added using Adobe Premier Pro (R2022; Adobe Inc.). The final video was rendered in MPEG-4 format at 1562 × 932 (1.0) pixel resolution, 60 fps, progressive field order with 2-pass variable bit rate (VBR: target bit rate: 27 Mbps, maximum bit rate: 30 Mbps) and the audio: AAC 320 kbps, 48 kHz, stereo.

Figure 2 shows plots of the raw stereo audio data, as well as the derived high (x-axis), medium (y-axis), and low frequency (z-axis) data. Figure 3 provides a screenshot taken from the final visualization, which is available to watch via the QR code in Fig. 4 [13].

In this study, a short visualization titled “Dancing with Atoms” was produced in honor of Irish scientist Sheila Tinney, after whom a new HPC system was named at TCD. The visualization was inspired by Sheila Tinney’s work on crystal lattices and consisted of an atomic lattice structure that was perturbed in three dimensions using inputs derived from a musical piece by J.S. Bach performed by her son, Hugh Tinney. This creative signal processing project had the purpose of honoring Sheila Tinney’s work and linking it to the work of her son. Although the visualization may not be chemically accurate, it resonates deeply with FRAILMatics’ aim to promote collaborations by celebrating and promoting the work of women in STEAM.

There are several possible applications for this type of visualization, beyond the mostly artistic/visual purpose for the present work. One such application could be as a novel, visual hearing test, testing the frequency ranges that are audible for an individual. It is well known that, as we age, our ability to hear certain frequencies diminishes, and this is more pronounced for higher frequencies at older ages. For example, our hearing threshold for a 6kHz tone lowers in a fairly linear manner from the age of 50 to 100 years; however, our hearing threshold for a 0.5 kHz tone is generally fairly stable, with a slight reduction from the ages of 50 to 80 years, but can rapidly decline after the age of 80 years [14]. To use the visualization for this application, one would have to simply adjust the input x, y, z, and timestep data for the lattice visualization, potentially cycling through ever-increasing frequency tones, with the lattice vibrating accordingly. The advantage to such an approach would be that it could be precise, with short steps of small frequency changes, which could identify subtle changes in hearing thresholds that may go undetected in traditional auditory tests. Another advantage of this approach is that the visual cue provided by the lattice vibration can make the test more engaging and less frustrating for participants, particularly children and older individuals who may find traditional auditory tests difficult or uncomfortable, which could lead to better compliance and engagement. If the testing process were made more accessible and engaging, the visualization approach could also be used to raise awareness of hearing loss and encourage individuals to seek timely interventions and treatments.

In material science, crystal lattices are defined as repeating 3D arrangements of atoms or molecules, which can be perturbed in various ways, leading to the emergence of different vibrational modes. These modes can be used to identify and characterize the physical and chemical properties of materials. However, in the present work, we aimed to create a hypothetical material with a crystal lattice structure that would allow for a wide and different frequency bandwidth to be absorbed and oscillated along each Cartesian axis (x, y, and z). While this is not a realistic scenario, it served as an effective visual aid for our creative signal processing project. Additionally, such dynamic visualization of a live musical performance could be a useful tool for training musicians; for example, the required smoothness of the sound (as in Hugh Tinney’s performance of J.S. Bach) could be visualized through the movements of the lattice and perhaps provide an aid for training musicians to achieve the required sound by means of visual feedback. For example, one could program the lattice to “break” above a predefined sound intensity (e.g., pianissimo) and/or when correct tempo is not maintained.

The choice of J. S. Bach’s “Prelude in C major” (BWV 846i) from The Well-Tempered Clavier, Book 1, for this study was carefully considered. J. S. Bach is counted as one of the greatest composers of all time and has made significant contributions to the development of Western classical music. By using his music, we hoped to pay tribute to his legacy and highlight the intersection between science, art, and culture. Additionally, Hugh Tinney is a renowned pianist and has a deep connection to both his mother’s work and J.S. Bach’s music. Using his performance as the input for our visualization provided a personal and meaningful connection to the project.

In the creation of a synchronized audiovisual piece, rhythm plays a significant role in achieving a coherent and engaging result. In this work we visualized the frequency content of the music; however, other aspects of the music (such as rhythm) and their relationship to the visual elements were not fully considered. As a result, there is a slight dissociation between the music and the animation, and the synchronization between the two is not always apparent. In future work, other common signal processing methods could be utilized to produce more visually apparent image-music visualizations, such as chromagrams or tempograms [15,16]. Chromagrams are a type of spectrogram that displays the distribution and intensity of musical pitches over time. They are useful for identifying harmonic patterns and melodic characteristics in the music while being robust to changes in timbre and instrumentation [17]. Tempograms, on the other hand, measure the rate of the musical beat (in beats per minute [bpm]), and provide a feature matrix that indicates the prevalence of certain tempi at each moment in time [18]. Each method has its own strengths and weaknesses depending on the application, and the choice of method would depend on the rhythmic characteristics of the music that was of interest. For example, if the rhythm is a key aspect of the music that is of interest, tempograms may be the most fitting method to use; however, if the harmonic content is more important, chromagrams may be a better choice. Additionally, in this work we chose to use J.S. Bach’s “Prelude in C major” (BWV 846i) from The Well-Tempered Clavier, Book 1; however, audio sources that have a clearer rhythmic structure may result in a more apparent relationship between the music and the visual elements, potentially resulting in more effective and impactful visualizations. Understanding the impact of different audio sources and signal processing methods on the final visual output would be useful for developing more refined workflows and achieving more compelling visual results in future work.

There are several limitations to the current work. Firstly, an MP3 file was used as a basis for the audio processing; MP3 is a lossy encoding format, meaning that some compression of the audio was implemented. The use of a lossless format such as FLAC would have provided an uncompressed audio source. However, the MP3 data were encoded at 192 kbps, which would have preserved most of the audible sonic information of the original recording. Another limitation is that higher-end audio equipment (amplifier, speakers, etc.) would be required to reproduce the full frequency range (16–20 kHz) correctly. It is a simple process to reduce the ranges used to accommodate the specifications of a particular system as required.

In conclusion, the current study presented a method for extracting specified frequency ranges from audio data and using said data to simulate how an atomic crystal lattice structure might be perturbed in 3D space by such sound waves in a theoretical material. We dedicate this work to the life and work of Sheila Tinney, a truly inspiring and distinguished Irish scientist.

Calculations were performed on the Tinney cluster maintained by the Trinity Centre for High Performance Computing (Research IT). The cluster is funded from a grant from Science Foundation Ireland under Grant No. 18/FRL/6188.