现有神经网络处理器已广泛应用于计算机视觉、自然语言处理等领域。然而，现有片上加速方案对控制领域的强化学习算法支持较少，而基于神经网络的强化学习是智能系统决策技术的核心。该文采用可重构阵列体系结构，通过片上配置、动作与奖励存储的系统设计方案，可实现多种神经网 络算法的灵活部署，并支持强化学习使用模式。基于 65 nm CMOS 工艺的逻辑综合结果显示，处理器主频为 200 MHz 时，计算模块面积仅需 0.32 mm2，计算功率约 15.46 mW。
Current artificial intelligent (AI) engines are usually designed for specific supervised learning algorithms, which have been widely used in computer vision and natural language processing domains etc. However, very few AI engines have been designed to support on-chip reinforcement learning algorithms, which is the foremost algorithm kernel for decision-making subsystem of many autonomous systems. In this work, a coarse-grained reconfigurable array like AI computing engine has been designed for the deployments of both supervised and reinforcement learning through on-chip configuration, action Random Access Memory (RAM) and reward RAM. Logic synthesis at the design frequency of 200 MHz based on 65 nm CMOS technology reveals the physical statistics of the proposed engine of 0.32 mm2 in silicon area, 15.46 mW in power consumption. The proposed on-chip AI engine facilitates the implementation of end-to-end perceptual and decision-making networks, which has great potentials in applications like autonomous driving, robotics and unmanned aerial vehicle.
梁明兰,王 峥,陈名松.基于可重构阵列架构的强化学习计算引擎 [J].集成技术,2018,7(6):19-30
LIANG Minglan, WANG Zheng, CHEN Mingsong. A Reconfigurable Computing Engine for Neural Network-Based Reinforcement Learning[J]. Journal of Integration Technology,2018,7(6):19-30