The distance between a pair of spike trains, quantifying the differences between them, can be measured using various metrics. Here we introduce a new class of spike train metrics, inspired by the Pompeiu-Hausdorff distance, and compare them with existing metrics. Some of our new metrics (the modulus-metric and the max-metric) have characteristics that are qualitatively different from those of classical metrics like the van Rossum distance or the Victor and Purpura distance. The modulus-metric and the max-metric are particularly suitable for measuring distances between spike trains where information is encoded in bursts, but the number and the timing of spikes inside a burst do not carry information. The modulus-metric does not depend on any parameters and can be computed using a fast algorithm whose time depends linearly on the number of spikes in the two spike trains. We also introduce localized versions of the new metrics, which could have the biologically relevant interpretation of measuring the differences between spike trains as they are perceived at a particular moment in time by a neuron receiving these spike trains.

The persistent modification of synaptic efficacy as a function of the relative timing of pre- and postsynaptic spikes is a phenomenon known as spike-timing-dependent plasticity (STDP). Here we show that the modulation of STDP by a global reward signal leads to reinforcement learning. We first derive analytically learning rules involving reward-modulated spike-timing-dependent synaptic and intrinsic plasticity, by applying a reinforcement learning algorithm to the stochastic spike response model of spiking neurons. These rules have several features common to plasticity mechanisms experimentally found in the brain. We then demonstrate in simulations of networks of integrate-and-fire neurons the efficacy of two simple learning rules involving modulated STDP. One rule is a direct extension of the standard STDP model (modulated STDP), and the other one involves an eligibility trace stored at each synapse that keeps a decaying memory of the relationships between the recent pairs of pre- and postsynaptic spike pairs (modulated STDP with eligibility trace). This latter rule permits learning even if the reward signal is delayed. The proposed rules are able to solve the XOR problem with both rate coded and temporally coded input and to learn a target output firing-rate pattern. These learning rules are biologically plausible, may be used for training generic artificial spiking neural networks, regardless of the neural model used, and suggest the experimental investigation in animals of the existence of reward-modulated STDP.