In estimating the frequency spectrum of real-world time series data, we must violate the assumption of infinite-length, orthogonal components in the Fourier basis. While it is widely known that care must be taken with discretely sampled data to avoid aliasing of high frequencies, less attention is given to the influence of low frequencies with period below the sampling time window. Here, we derive an analytic expression for the side-lobe attenuation of signal components in the frequency domain representation. This expression allows us to detail the influence of individual frequency components throughout the spectrum. The first consequence is that the presence of low-frequency components introduces a 1/f component across the power spectrum, with a scaling exponent of . This scaling artifact could be composed of diffuse low-frequency components, which can render it difficult to detect a priori. Further, treatment of the signal with standard digital signal processing techniques cannot easily remove this scaling component. While several theoretical models have been introduced to explain the ubiquitous 1/f scaling component in neuroscientific data, we conjecture here that some experimental observations could be the result of such data analysis procedures.
The Fourier transform of a function has many applications in physics and engineering, and each is referred to as frequency domain and time-domain representation, respectively (Fourier, 1822). The power spectral density (PSD) is a powerful tool in the analysis of signals and distinguishing systems. However, the main assumption of a Fourier series decomposition, infinitely long data, is violated, since all experimental data are inherently of finite length.
We have investigated this low-frequency-induced artifact present in spectral analysis of finite-length signals. This study demonstrates that although the effect becomes dominant for test frequencies below (see equation 1.3), small artifacts are present throughout the frequency spectrum for all test frequencies. Finally, we demonstrate that these effects are robust to removal with standard filtering techniques (see the appendix). While theoretical models have been proposed to explain the ubiquitous scaling observed in neuroscientific data, for example as a result of neuronal shot noise (Milstein, Mormann, Fried, & Koch, 2009), we propose that the effect studied here is important for the analysis of neuroscientific data and that care should be taken with interpreting power spectral estimates.
In summary, this letter has three main points:
Fourier analysis is based on the assumption of infinite window length. This assumption is always necessarily violated in real-world data analysis.
Consequences of this violation include nonlinearities resulting from multiple frequencies in the data, spectral leakage, and scaling artifacts induced by low-frequency components in the data (below ).
The interplay of frequencies and phase is often neglected in the analysis of real-world data.
The letter is organized as follows. In section 2, the analytical forms of the continuous-time Fourier transform for infinite and finite windows are derived. From the finite-length form, we derive the analytical form of the spectrum and the analytical forms of its piecewise envelopes. We make the connection to spectral leakage in section 2.1.1, the observed low-frequency induced scaling artifact in section 2.3, and nonlinearities in the spectrum when dealing with multiple frequencies in the data in section 2.1.3. We perform the analyses following Goertzel (1958) and an alternative approach. In section 2.3, we present an analytical derivation of the 1/f scaling artifact and briefly address the interplay between frequency and phase, which will be the subject of future research. Section 3 presents concluding remarks and provides perspectives on how this spectral scaling artifact could be an important consideration for both theoreticians and experimentalists using spectral analysis. The appendix presents rigorous numerical evaluations of the robustness of this artifact to common signal processing techniques, such as windowing and prefiltering.
2 Analytical Results
2.1 Goertzel Algorithm
2.1.1 Single-frequency signal
2.1.2 Time Shift Theorem for a Single Frequency
2.1.3 Multiple Frequency Signal
2.2 Alternative Representation
2.2.1 Single Frequency
2.3 Analytical Derivation of the 1/f Scaling Artifact
As a direct consequence of equations 2.15b, 2.16b, and 2.16c for two frequencies greater than , where is smaller than , the signal phases (both absolute and relative) determine their discriminability in the resulting spectrum. This effect will be the subject of future study.
We have presented analytical derivations illustrating the nonlinear effects present in spectral analysis. In the appendix, we support our results with numerical simulations. These aspects are a direct consequence of violating the assumption of infinite-length sinusoidal components in Fourier analysis. These nonlinear effects result from couplings of the spectral leakage contributions from individual frequency components and can become profound in certain instances. We applied the theoretical results to the case of low-frequency components in a noise-free signal, where a specific low-frequency-induced spectral scaling artifact can occur.
To summarize, following are the main points explored in this work:
Consequences of analyzing finite-length signals in Fourier analysis
Nonlinearities resulting from multiple frequencies in the data. For real-world, finite-length data, the limit of is not realized, which causes nonlinear interactions between single-frequency components.
Spectral leakage. We analyzed the spectral leakage effects introduced by Harris (1978) in an analytical manner.
Scaling artifacts induced by sub-cutoff frequencies (below ). Scaling artifacts can be caused by special spectral leakage effects.
Interplay of frequencies and phase is often neglected in analysis of real-world data. Due to nonlinearities caused by finite-length windows, frequencies and phases are interconnected in a nonlinear way. Consequences of this are the subject of ongoing research.
Recognition of the possibility of this artifact and nonlinear interconnections between phase and frequency is important for neuroscientists, physicists, and engineers who use spectral analysis in their work. As illustrated in the appendix, this artifact is robust to removal through common linear filtering and detrending techniques, and thus may be an important practical consideration for analysis of experimental data. More robust filtering approaches and further consequences of nonlinearities in the signal processing technique, will be the subject of future work.
Appendix A: Numerical Robustness Tests
This work was supported by the Howard Hughes Medical Institute and the Crick-Jacobs Center for Theoretical and Computational Biology; the U.S. Office of Naval Research under grants N00014-10-1-0072 and N00014-12-1-0299; and NIH under grants R01-EB009282 and T32-EY20503-5. We thank Andrew Viterbi for helpful discussions.