• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学 ›› 2021, Vol. 43 ›› Issue (02): 266-273.

• 高性能计算 • 上一篇    下一篇

基于RISC-V架构的强化学习容器化方法研究

徐子晨,崔傲,王玉皞,刘韬   

  1. (南昌大学信息工程学院,江西 南昌 330031)

  • 收稿日期:2020-05-03 修回日期:2020-07-06 接受日期:2021-02-25 出版日期:2021-02-25 发布日期:2021-02-23
  • 基金资助:
    国家自然科学基金(61702250);国家重点研发计划(2018YFB14043033);国家核高基(2018ZX01035-101);中科院计算机体系结构国家重点实验室开放课题(CARCHB202017)

A containerization method for reinforcement learning based on RISC-V architecture

XU Zi-chen,CUI Ao,WANG Yu-hao,LIU Tao   

  1. (School of Information Engineering,Nanchang University,Nanchang 330031,China )

  • Received:2020-05-03 Revised:2020-07-06 Accepted:2021-02-25 Online:2021-02-25 Published:2021-02-23

摘要: RISC-V作为近年来最热门的开源指令集架构,被广泛应用于各个特定领域的微处理器,特别是机器学习领域的模块化定制。但是,现有的RISC-V应用需要将传统软件或模型在RISC-V指令集上重新编译或优化,故如何能快速地在RISC-V体系结构上部署、运行和测试机器学习框架是一个亟待解决的技术问题。使用虚拟化技术可以解决跨平台的模型部署和运行问题。但是,传统的虚拟化技术,例如虚拟机,对原生系统性能要求高,资源占用多,运行响应慢,往往不适用于RISC-V架构的应用场景。讨论在资源受限的RISC-V架构上的强化学习虚拟化问题。首先,通过采用容器化技术减少上层软件构建虚拟化代价,去除冗余中间件,定制命名空间隔离特定进程,有效提升学习任务资源利用率,实现模型训练快速执行;其次,利用RISC-V指令集的特征进一步优化上层神经网络模型,
提高强化学习效率;最后,实现整体优化和容器化方法系统原型,并通过多种基准测试集完成系统原型性能评估。容器化技术和传统RISC-V架构下交叉编译深度神经网络模型的方法相比,仅付出相对较小的额外性能代价,能快速实现更多、更复杂的深度学习软件框架的部署及运行;与Hypervisor虚拟机方法相比,基于RISC-V的模型具有近似的部署时间,并大量减少了性能损失。初步实验结果表明,容器化及其上的优化方法是实现基于RISC-V架构的软件和学习模型快速部署的一种有效方法。


关键词:

Abstract: As the hottest open-source instruction set architecture in recent years, RISC-V is widely used in a variety of domain-specific microprocessors, especially for modular customization in the field of machine learning. However, existing RISC-V applications require recompilation or optimization of legacy software or models on the RISC-V instruction set. Therefore, how to rapidly deploy, run, and test machine learning frameworks on RISC-V architectures is a pressing technology challenges. The use of virtualization technology can solve the problem of deploying and running models across platforms. However, traditional virtualization techniques, such as virtual machines, are often not applicable to RISC-V architecture scenarios due to their high performance requirements for native systems, high resource footprint, and slow operational response. Discussion of reinforcement learning virtualization on resource-constrained RISC-V architectures. Firstly, by adopting containerization technology, reducing the cost of virtualization for upper-level software builds, removing redundant middleware, and customizing namespaces to isolate specific processes, we effectively improve the resource utilization for learning tasks and achieve the rapid execution of model training. Secondly, the features of the RISC-V instruction set are used to further optimize the upper neural network model and optimize the reinforcement learning efficiency. Finally, a system prototype of the overall optimization and containerization method is implement- ed and the performance evaluation of the prototype is completed by testing multiple benchmark test sets. Containerization techniques enable the rapid deployment and operation of more complex and deep learning software frameworks at a relatively small additional performance cost, compared to traditional methods of cross-compiling deep neural network models in RISC-V architectures. RISC-V based models have approximate deployment time and reduce substantial performance losses compared to the hypervisor VM method. Preliminary experimental results demonstrate that containerization and the optimization method on it are an effective way to achieve the rapid deployment of software and learning models based on RISC-V architecture.


Key words: virtualization, neural network, RISC-V

中图分类号: