A Low-Power Actor-Critic Framework Based on Memristive Spiking Neural Network
[摘要] Traditional deep reinforcement learning (DRL) algorithms consume much energy. Energy-efficient spiking neural networks (SNNs) are promising technologies to bulid a low-power reinforcement learning architecture. In this paper, an actor-critic framework based on memrisitive SNN is proposed. To convey and process information in SNN, spike encoding and decoding systems are created. Then, an improved learning algorithm based on spike-timing-dependent plasticity (STDP) learning rule is designed to combine actor-critic method with SNN. Moreover, this learning algorithm is also hardware-friendly. Besides, memristive synapse is designed to accelerate this learning algorithm. Finally, a continuous control problem is applied to illustrate the effectiveness of the proposed framework. The results show the proposed framework is prior to traditional methods.
[发布日期] [发布机构] College of Computer and Information Science, Southwest University, Chongqing; 400715, China^1;Brain-inspired Computing and Intelligent Control of Chongqing Key Lab, Chongqing; 400715, China^2;College of Electronic and Information Engineering, Southwest University, Chongqing; 400715, China^3
[效力级别] 材料科学 [学科分类]
[关键词] Actor-Critic methods;Continuous control;Encoding and decoding;Energy efficient;Learning rules;Process information;Spike timing dependent plasticities;Spiking neural networks [时效性]
