We propose a real-time, online temporal action localization system that requires a small amount of annotated data. The main challenges we address are high intra-class variability and a large and diverse background class. We address these using a flexible frame descriptor, dynamic time warping, and a novel approach to database construction. Our solution receives egocentric RGB-D streams as input and makes predictions at regular temporal intervals. We validate our approach by localizing actions in a digital twin of an electrical substation, in which certain objects have been replaced by functional virtual replicas.

The usefulness of auricular muscle activation as a means of selection and activation of objects or locations is investigated. We find that in nearly half of those studied, use of this latent communication channel expands total expression channel capacity. A method is described that detects tension in auricular muscles, which is used to signify intent. This intent signal is then transmitted on the person's visual attention vector to select, activate, and control objects in an environment.