A Brain-Inspired Learning Algorithm Enables Metaplasticity in Artificial and Spiking Neural Networks

Credit assignment in neural networks for correcting global output mistakes has been determined using many synaptic plasticity rules in natural neural networks. Short-term plasticity, Hebbian learning, and spike-timing-dependent plasticity (STDP) have been the primary focuses of previous attempts to bring biologically relevant plasticity principles into spiking and nonspiking ANNs. STDP goes beyond Hebbian learning by considering the temporal order of pre- and postsynaptic spikes to alter synapses. Synaptic plasticity rules in both circumstances are solely based on local neuronal activity rather than accurately representing global instructional messages. Neuromodulators like dopamine, noradrenaline, serotonin, and acetylcholine work at many synapses and come from widely scattered axons of specific neuromodulatory neurons to produce global modulation of synapses during reward-associated learning.

Methods of biological neuromodulation have inspired several plasticity algorithms in models of neural networks. There is a significant lag between the Hebbian modification and the reward, but the rule has inspired other forms of reinforcement learning. For instance, the three-factor rule for reinforcement learning uses pre- and postsynaptic neuronal activity as the first two factors and distal reward-dependent neuromodulator levels as the third factor. Eligibility trace models store a record of prior pre-and postsynaptic spikes that occurred simultaneously to facilitate delayed reward-dependent synaptic changes. Synaptic amplitude and polarity have been determined in computational neuroscience models at the neuromodulator level, but these methods still need to be included in ANNs or SNNs. When it comes to supervised learning of image and speech recognition, the NACA algorithm has not only significantly reduced the problem of catastrophic forgetting during class-CL but has also improved recognition accuracy and reduced computing cost. Synaptic weight changes in the buried layer were mapped further, revealing that NACA’s distribution of weight changes avoided excessive synaptic potentiation or depression, therefore protecting a high proportion of synapses with little tweaks. Our findings collectively present a novel brain-inspired algorithm for expectation-based global neuromodulation of synaptic plasticity, which enables neural network performance with high accuracy and low computing cost across a range of recognition and continuous learning tasks. 

To address the problem of catastrophic forgetting in ANN and SNN, researchers at the Institute of Automation of the Chinese Academy of Sciences presented a novel brain-inspired learning approach (NACA) based on neuronal modulation-dependent plasticity.

This technique is founded on a mathematical model of the neural modulation pathway in the form of anticipated matrix encoding, which in turn is based on the brain’s structure of the neural modulation pathway. Dopamine supervisory signals of varying strengths are created in response to the stimulus signal and influence the plasticity of nearby neurons and synapses.

Both ANNs and SNNs can be trained with the help of NACA’s endorsement of pure feed-forward flow learning techniques. It syncs up with the input signal and even forward propagates information before the incoming call has finished. Significant benefits in rapid convergence and reduction of catastrophic forgetting are demonstrated by NACA when combined with specific modification of the spike-timing-dependent plasticity. Furthermore, the research team expanded the neural modulation to the range of neuronal plasticity and tested NACA’s continuous learning ability in class continuous learning.

Researchers defined neuromodulator levels at subpopulations of synapses in the hidden and output layers during network training utilizing the NACA algorithm, considering input type and output error. The dependency of synaptic efficacy on the level of neuromodulators or calcium inspired the nonlinear modulation of LTP and LTD amplitude and polarity at each synapse in SNNs. Dopamine binding to synapses containing D1-like or D2-like receptors, for instance, may variably activate intracellular signaling cascades, resulting in the modification of activity-induced LTP or LTD.

We implemented neuromodulation-dependent synaptic plasticity into a learning algorithm called NACA for SNNs and ANNs. We found significant improvements in accuracy and a dramatic decrease in computing cost when applying the network to common image and voice recognition tasks. Five class-CL tasks of varied complexity had their catastrophic forgetting greatly reduced by the NACA technique. While other neuromodulation-inspired network learning algorithms, such as the global neuronal workspace theory in spiking neural networks (SNNs) and neuromodulation of dropout probability in artificial neural networks (ANNs), have been developed, NACA stands out due to three distinct qualities that may contribute to its success. The neuromodulator level at specific neurons and synapses in the hidden and output layers is tuned by expectations based on the input type and output error. Second, the neuromodulator level nonlinearly affects local synaptic plasticity, such as LTP or LTD. Third, the global BP of erroneous signals is irrelevant to network learning, which depends entirely on local plasticity.

The NACA algorithm drastically lowered the computing cost of all jobs compared to existing learning algorithms. Using NACA helped reduce the extreme forgetfulness that often occurs during continuous learning. Further mapping of synaptic weight changes at hidden layer synapses during class CL revealed that NACA resulted in normally distributed synaptic weight changes without excessive potentiation or depression and preserved many synapses with minimum modification during class-CL. NACA’s ability to reduce extreme amnesia may be based on how changes in synaptic weight are distributed.

Below are some restrictions placed on the NACA algorithm that is proposed:

• First, in deeper neural networks, the NACA algorithm shows some nonstability during neuromodulation of synaptic changes. In the initial few epochs, for instance, parallel neuromodulation at multilayer synapses contributes to a temporary decline in test accuracy.
• Second, in keeping with predictive coding, the NACA algorithm is not easily integrated with the traditional BP algorithm since its global neuromodulation occurs with or even ahead of the local spike propagation.
• Third, NACA introduces and investigates only excitatory LIF neurons and a single type of neuromodulator without examining the interplay of neuromodulations from several neuron types.

The NACA algorithm, which incorporates biologically plausible learning rules without resorting to global BP-like gradient descent computations, could drive network learning for SNNs and ANNs, to sum up. It demonstrates that high efficiency and low computing cost in machine learning can be achieved by employing brain-inspired methods. The NACA algorithm, if implemented in neuromorphic devices, could pave the way for online continuous learning systems that are both energy- and time-efficient. When seen through computational neuroscience, NACA’s success proves that the flexibility of the brain’s neural circuits for ongoing learning may stem from the metaplasticity-based diversity of local plasticity.

Check out the Paper and Reference Article. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 29k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..