Recent neuroscience experiments have brought inconsistent findings to light about the influence of neural activity in the alpha-frequency band (at ≈10 Hz) on the temporal dynamics of visual perception. Whereas strong alpha effects were found when perception was more based on endogenous factors, there were null-effects for alpha when perception relied more on objective physical parameters. In this Perspective, I open up a new view on neural alpha activity that resolves some important aspects of this controversy by interpreting alpha not as temporal processing of sensory inputs per se but above all as the observer's internal processing dynamics, their so called “perception sets.” “Perception sets” reflects internally stored knowledge for how to organize and build up perceptual processes. They result from previous sensory experiences, are under top–down control to support goal-directed behavior and root in pre-established neural networks that communicate through alpha frequency channels. I present three example cases from the recent neuroscience literature that show an influence of alpha-driven “perception sets” on the observer's visual-temporal resolution, object processing, and the processing of behaviorally relevant image content. Because alpha-driven “perception sets” can structure perception from its high-level aspects, like categories, down to its basic building blocks, like objects and time samples, they may have a fundamental impact on our conscious experience of the sensory world, including our perception of time itself.