Balancing excitation and inhibition of spike neuron using deep Q network (DQN)

• Main Authors: Tan, Szi Hui, Ishak, Mohamad Khairi, Packeer Mohamed, Mohamed Fauzi, Mohd Fadzil, Lokman, Ahmad Afif, Ahmarofi
• Format: Conference or Workshop Item
• Language: English
• Published: IOP Publishing 2021
• Subjects: QA76 Computer software; TK Electrical engineering. Electronics Nuclear engineering
• Online Access: http://umpir.ump.edu.my/id/eprint/31934/1/Balancing%20excitation%20and%20inhibition%20of%20spike%20neuron.pdf; http://umpir.ump.edu.my/id/eprint/31934/; https://doi.org/10.1088/1742-6596/1755/1/012004

Deep reinforcement learning which involved reinforcement learning with artificial neural networks allows an agent to take the best possible actions in a virtual environment to achieve goals. Spike neuron has a non-differentiable spike generation function that caused SNN training faced difficulty. In order to overcome the difficulty, Deep Q Network (DQN) is proposed to act as an agent to interact with a custom environment. A spike neuron is modelled by using NEST simulator. Rewards are given to the agent for every action taken. The model is trained and tested to validate the performance of the trained model in order to attain balance the firing rate of excitatory and inhibitory population of spike neuron. Training result showed the agent able to handle the environment. The trained model capable to balance the excitation and inhibition of the spike neuron as the actual output neuron rate is close to or same with the target neuron firing rate. The average percentage error of rate of difference between output and target neuron rate for 5 episodes achieved 0.80%.