We can talk about learning optimisation in terms of three biological processes: evolution, development and learning. It has been argued that all three are necessary for intelligence to emerge. Together, they shape the brain through regressive and progressive plasticity.
In this paper, we explored the effects of structural plasticity on learning in spiking neural networks with spike-timing-dependent plasticity: first, we systematically analysed three synapse pruning approaches (random, weight-dependent and activity-dependent) and their effects on networks’ weights, spiking activity and performance on a clustering task. Then, we examined the use of a minimalistic evolutionary approach to develop growth rules for spiking neural networks with or without pruning.
We found that pruning combined with a simple weight homeostasis mechanism can be used to reduce spiking neural networks’ size without a performance loss; pruning of weak connections increases the learning rate. Evolution of developmental rules led to a rapid fitness increase of the rudimentary embryo networks; addition of pruning significantly improved the learning rate of the model, and synaptic homeostasis preserved stable spiking activity in the networks even during drastic growth.