数据中心网络四层负载均衡技术综述

摘要/Abstract

摘要： 针对数据中心难以适应流量增长进行横向扩展并保证连接一致性的问题，阐述了四层负载均衡技术在应对高并发访问和提高资源利用率方面的重要作用，梳理了国内外四层负载均衡模块的设计与算法，总结了负载均衡器以不同方式进行部署分别存在的优缺点，同时分析了网络可编程转发技术在四层负载均衡领域中的应用与最新进展。最后，对网络新形势下负载均衡技术的发展进行了进一步展望并讨论了未来的研究方向。

关键词: 数据中心, 负载均衡, 连接一致性, 可编程转发技术

Abstract: Aiming at the problem of data centers difficulty in adapting to traffic growth for horizontal expansion and ensuring connection consistency, this paper explains the important role of layer-4 load balancing technology in coping with high concurrent access and improving resource utilization, and sorts out four-layer load balancing modules domestic and international. The advantages and disadvantages of load balancers deployed in different ways are summarized, and the application and latest development of network programmable forwarding technology in the field of layer-4 load balancing are analyzed. Finally, The development of load balancing technology under the new network situation is further prospected and future research directions are discussed.

Key words: data center, load balancing, connection consistency, programmable forwarding technology

李力, 汪硕, 黄韬, 刘韵洁, . 数据中心网络四层负载均衡技术综述[J]. 计算机工程与科学, 2022, 44(01): 48-59.

LI Li , WANG Shuo, HUANG Tao, LIU Yun-jie, . Review on four-layer load balancing technology in data center network[J]. Computer Engineering & Science, 2022, 44(01): 48-59.

参考文献［56］

［1］	Cisco. Cisco global cloud index:Forecast and methodology,2015-2020［EB/OL］.［2016-06-01］.http://www.audentia-gestion.fr/cisco/white-paper-c11-738085.pdf.
［2］	Andreyev A. Introducing data center fabric,the next-generation Facebook data center network［EB/OL］.［2014-11-14］.https://engineering.fb.com/production-engineering/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/.
［3］	Singh A,Ong J,Agarwal A,et al.Jupiter rising:A decade of clos topologies and centralized control in Google’s datacenter network［J］.ACM SIGCOMM Computer Communication Review,2015,45(4):183-197.
［4］	Patel P,Bansal D,Yuan L,et al.Ananta:Cloud scale load balancing［J］.ACM SIGCOMM Computer Communication Review,2013,43(4):207-218.
［5］	MacVittie L.Message-based load balancing［R］.Seattle:F5 Networks, 2010.
［6］	Array Networks Inc.Global server load balancing［EB/OL］.［2017-03-01］.https://arraynetworks.com/ufiles/resources/WP-APV-Global-Server-Load-Balancing.pdf.
［7］	Kunamneni V. Dynamic load balancing for the cloud［J］.
	International Journal of Computer Science and Electrical Engineering,2012,2315:33-37.
［8］	Zhang Wen-song,Wu Ting-ting,Jin Shi-yao,et al.Design and implementation of a virtual internet server［J］.Journal of Software,2000,11(1):122-125.(in Chinese)
［9］	Alibaba.LVS FULLNAT outline design ［EB/OL］.［2013-07-02］.https://github.com/alibaba/LVS/blob/master/docs/LVS_FULLNAT.pdf.(in Chinese)
［10］	IQIYI. DPVS［EB/OL］.［2021-01-25］.https://github.com/iqiyi/dpvs/blob/master/README.md.
［11］	JingDong.High-performance four-layer load balancer SKYLB based on DPDK［EB/OL］.［2018-03-29］.http://www.um l.org.cn/zjjs/201803293.asp.(in Chinese)
［12］	MeiTuan.MGW——High-performance four-layer load balancing ［EB/OL］.［2017-01-05］.https://tech.meituan.com/2017/01/05/mgw.html.(in Chinese)
［13］	Pongrácz G, Molnár L, Kis Z L. Removing roadblocks from SDN: OpenFlow software switch performance on Intel DPDK［C］∥Proc of the IEEE 2nd European Workshop on Software Defined Networks, 2013:62-67.
［14］	Gandhi R,Liu H H,Hu Y C,et al.Duet:Cloud scale load balancing with hardware and software［J］.ACM SIGCOMM Computer Communication Review,2014,44(4):27-38.
［15］	Gandhi R,Hu Y C,Koh C K,et al.Rubik:Unlocking the power of locality and end-point flexibility in cloud scale load balancing［C］∥Proc of the 15th USENIX Annual Technical Conference,2015:473-485.
［16］	Eisenbud D E,Yi C,Contavalli C,et al.Maglev:A fast and reliable software network load balancer［C］∥Proc of the 13th USENIX Symposium on Networked Systems Design and Implementation,2016:523-535.
［17］	GitHub.GLB director［EB/OL］. ［2018-07-27］.https://github.com/github/glb-director/blob/master/README.md.
［18］	Miao R,Zeng H Y,Kim C,et al.SilkRoad:Making stateful layer-4 load balancing fast and cheap using switching ASICs［C］∥Proc of the Conference of the ACM Special Interest Group on Data Communication,2017:15-28.
［19］	Olteanu V, Agache A,Voinescu A,et al.Stateless datacenter load-balancing with Beamer［C］∥Proc of the 15th USENIX Symposium on Networked Systems Design and Implementation,2018:125-139.
［20］	Araújo J T,Saino L,Buytenhek L,et al.Balancing on the edge:Transport affinity without network state［C］∥Proc of the 15th USENIX Symposium on Networked Systems Design and Implementation,2018:111-124.
［21］	Barbette T, Tang C, Yao H, et al. A high-speed load- balancer design with guaranteed per-connection-consistency［C］∥Proc of the 17th USENIX Symposium on Networked Systems Design and Implementation,2020:667-683.
［22］	Reese W.Nginx:The high-performance web server and reverse proxy［J］.Linux Journal,2008(173):62-67.
［23］	Prasetijo A B,Widianto E D,Hidayatullah E T.Performance comparisons of web server load balancing algorithms on HAProxy and Heartbeat［C］∥Proc of IEEE 3rd International Conference on Information Technology,Computer,and Electrical Engineering,2016:393-396.
［24］	Liu Yu-yan, Shen Ming-yu.Research on the LVS load balance in Web services［J］.Journal of Hefei University of Technology(Natural Science),2007,30(12):1592-1595.(in Chinese)
［25］	Li Jie.Research on dynamic load balancing algorithm based on LVS cluster［D］.Nanchang:Nanchang Hangkong University,2018.(in Chinese)
［26］	Li Kun.Server load balancing algorithm based on dynamic feedback ［J］.Electronic Science and Technolog,2015,28(9):45-49.(in Chinese)
［27］	Zhang Yu-fang,Wei Qin-lei,Zhang Ying.Load balancing algorithm based on load weights ［J］.Application Research of Computers,2012,29(12):4711-4713.(in Chinese)
［28］	Zhu Li, Shen Wei-ming,Li Rui,et al.Genetic algorithm-based dynamic load balancing for server cluster in network GIS［J］.Geomatics and Information Science of Wuhan University,2011,36(6):721-725.(in Chinese)
［29］	Wang Peng,Huang Hong-qiong.An improved load balanc- ing algorithm based on neural network feedback mechanism［J］.Modern Computer (Professional Edition),2018(12):3-9.(in Chinese)
［30］	Karger D,Lehman E,Leighton T,et al.Consistent hashing and random trees:Distributed caching protocols for relieving hot spots on the world wide web［C］∥Proc of the 29th Annual ACM Symposium on Theory of Computing,1997:654-663.
［31］	DeCandia G, Hastorun D, Jampani M, et al. Dynamo: Amazon’s highly available key-value store［J］. ACM SIGOPS Operating Systems Review, 2007, 41(6): 205-220.
［32］	Ba Zi-yan,Wu Jun,Ma Yan.The optimization for consistent hash based on virtual node ［J］.Software,2014,35(12):26-29.(in Chinese)
［33］	Srinivasan L,Varma V.Adaptive load-balancing for consistent hashing in heterogeneous clusters［C］∥Proc of the 15th IEEE/ACM International Symposium on Cluster,Cloud and Grid Computing,2015:1135-1138.
［34］	Perkins C.IP encapsulation within IP:RFC 2003［S］.1996-10.
［35］	Farinacci D,Li T,Hanks S,et al.Generic routing encapsulation (GRE):RFC 2784［S］.2000-03.
［36］	Bosshart P, Daly D, Gibb G, et al. P4: Programming protocol-independent packet processors［J］.ACM SIGCOMM Computer Communication Review,2014,44(3):87-95.
［37］	Hunt P, Konar M,Junqueira F P,et al.ZooKeeper:Wait-free coordination for internet-scale systems［C］∥Proc of the USENIX Annual Technical Conference,2010:1-14.
［38］	Barbette T,Soldani C,Mathy L.Fast userspace packet processing［C］∥Proc of ACM/IEEE Symposium on Architectures for Networking and Communications Systems,2015:5-16.
［39］	Behavioral model(bmv2)［EB/OL］.［2020-10-10］.https://github.com/p4lang/behavioral-model/blob/main/README.md.
［40］	Shen Geng-biao,Li Qing,Jiang Yong,et al.Research on load balancing in data center networks［J］.Journal of Software,2020,31(7):2221-2244.(in Chinese)
［41］	Wen Shu-bo.Research and implementation of load balancing in data center［D］.Beijing:Beijing University of Posts and Telecommunications,2020.(in Chinese)
［42］	Jiang Wen-ti.Server load balancing and DDoS attack detection technology based on SDN［D］.Nanjing:Nanjing University of Posts and Telecommunications,2019.(in Chinese)
［43］	Tang Yue-ting.A research of a technology to mitigate DDoS attack based on SDN［D］.Guiyang:Guizhou University,2018.(in Chinese)
［44］	Aghdai A,Wang I C,Xu Y,et al.In-network congestion- aware load balancing at transport layer［C］∥Proc of IEEE Conference on Network Function Virtualization and Software Defined Networks.2019:1-6.
［45］	Zhang Jiao,Wen Shu-bo,Pan Tian,et al.A four-layer load balancer and balancing algorithm with server status perceivable:CN201910122857.3［P］.2019-07-02.(in Chinese)
［46］	Pit-Claudel B,Desmouceaux Y,Pfister P,et al.Stateless load-aware load balancing in P4［C］∥Proc of IEEE 26th International Conference on Network Protocols,2018:418-423.
［47］	Desmouceaux Y,Pfister P,Tollet J,et al.6LB:Scalable and application-aware load balancing with segment routing［J］.IEEE/ACM Transactions on Networking,2018,26(2):819-834.
［48］	Aghdai A,Chu C Y,Xu Y,et al.Spotlight:Scalable transport layer load balancing for data center networks［J］.arXiv:1806.08455V3,2019.
［49］	Herbert T,Yong L,Zia O.Generic UDP encapsulation［EB/OL］.［2016-01-01］.https://www.ietf.org/proceedings/91/slides/slides-91-nvo3-1.pdf.
［50］	Liu Chang, Tao Yun-xiang,Rong Yin.Study on hardware offload based cloud network infrastructure ［J］.Telecom Engineering Technics and Standardization,2020,33(8):76-81.(in Chinese)
［51］	Le Y,Chang H,Mukherjee S,et al.UNO:Uniflying host and smart NIC offload for flexible packet processing［C］∥Proc of the 2017 Symposium on Cloud Computing,2017:506-519.
［52］	Grant S,Yelam A,Bland M,et al.SmartNIC performance isolation with FairNIC:Programmable networking for the cloud［C］∥Proc of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications,Technologies,Architectures,and Protocols for Computer Communication,2020:681-693.
［53］	Sabin G,Rashti M.Security offload using the SmartNIC, a programmable 10 Gbps ethernet NIC［C］∥Proc of 2015 National Aerospace and Electronics Conference,2015:273-276.
［54］	Yan Y,Beldachi A F,Nejabati R,et al.P4-enabled smart NIC:Enabling sliceable and service-driven optical data centers［J］.Journal of Lightwave Technology,2020,38(9):2688-2694.
［55］	Dukic V,Jyothi S A,Karla B,et al.Is advance knowledge of flow sizes a plausible assumption?［C］∥Proc of the 16th USENIX Symposium on Networked Systems Design and Implementation,2019:565-580.
［56］	Yan Wen-qing. The effect of web load balancing in anti distributed denial of service attacks［J］.Cyberspace Security,2016(7):73-76.(in Chinese)
	附中文参考文献：
［8］	章文嵩,吴婷婷,金士尧,等.一个虚拟Internet服务器的设计与实现［J］.软件学报,2000,11(1):122-125.
［9］	阿里巴巴.LVS FULLNAT概要设计［EB/OL］.［2013-07-02］.https://github.com/alibaba/LVS/blob/master/docs/LVS_FULLNAT.pdf.
［11］	京东.基于DPDK技术的高性能四层负载均衡器SKYLB［EB/OL］.［2018-03-29］.http://www.um l.org.cn/zjjs/201803293.asp.
［12］	美团.MGW——美团点评高性能四层负载均衡［EB/OL］.［2017-01-05］.https://tech.meituan.com/2017/01/05/mgw.html.
［24］	刘玉艳,沈明玉.LVS负载均衡技术在网络服务中的应用［J］.合肥工业大学学报(自然科学版),2007,30(12):1592-1595.
［25］	李杰.基于LVS集群的动态负载均衡算法研究［D］.南昌:南昌航空大学,2018.
［26］	李坤.基于动态反馈机制的服务器负载均衡算法研究［J］.电子科技,2015,28(9):45-49.
［27］	张玉芳,魏钦磊,赵膺.基于负载权值的负载均衡算法［J］.计算机应用研究,2012,29(12):4711-4713.
［28］	朱莉,沈未名,李锐,等.利用遗传算法的网络GIS集群服务器动态负载均衡算法［J］.武汉大学学报(信息科学版),2011,36(6):721-725.
［29］	王鹏,黄洪琼.基于神经网络反馈机制的改进型负载均衡算法［J］.现代计算机(专业版),2018(12):3-9.
［32］	巴子言,吴军,马严.基于虚节点的一致性哈希算法的优化［J］.软件,2014,35(12):26-29.
［40］	沈耿彪,李清,江勇,等.数据中心网络负载均衡问题研究［J］.软件学报,2020,31(7):2221-2244.
［41］	文殊博.数据中心负载均衡的研究与实现［D］.北京:北京邮电大学,2020.
［42］	姜文醍.基于SDN的服务器负载均衡及DDoS攻击检测技术［D］.南京:南京邮电大学,2019.
［43］	唐月婷.一种基于SDN的缓和DDoS攻击技术研究［D］.贵阳:贵州大学,2018.
［45］	张娇,文殊博,潘恬,等.一种服务器状态可感知的四层负载均衡器及均衡方法:CN201910122857.3［P］.2019-07-02.
［50］	刘畅,陶云祥,戎胤.智能网卡应用于云网络加速方案研究［J］.电信工程技术与标准化,2020,33(8):76-81.
［56］	严文卿.Web负载均衡在抗DDos攻击中的作用［J］.网络空间安全,2016(7):73-76.

[1]	陈冬林, 马一帆, 邹安琪, . 基于可再生能源消纳的数据中心负荷调度研究综述[J]. 计算机工程与科学, 2023, 45(05): 790-801.
[2]	刘忠沛, 吕高锋, 王继昌, 杨翔瑞. 专用数据处理器综述[J]. 计算机工程与科学, 2023, 45(02): 215-227.
[3]	李文佳, 史岚, 季航旭, 罗意彭. 面向Flink的负载均衡任务调度算法的研究与实现[J]. 计算机工程与科学, 2022, 44(07): 1141-1151.
[4]	曹继军. 面向HPC和DC的可重构光互连网络体系结构综述[J]. 计算机工程与科学, 2022, 44(06): 951-963.
[5]	徐浩桐, 黄山, 孙国璋, 贺菲莉, 段晓东, . 面向云环境的Flink负载均衡策略[J]. 计算机工程与科学, 2022, 44(05): 779-787.
[6]	段晨, 彭伟, 王宝生. 数据中心网络路由研究进展[J]. 计算机工程与科学, 2022, 44(04): 631-644.
[7]	罗晓霞, 王佳, 罗香玉, 李嘉楠 . 一种基于GN算法的动态图划分方法[J]. 计算机工程与科学, 2022, 44(02): 306-311.
[8]	毛媛媛, 石恩雅, 蒋从锋, 仇烨亮, 贾刚勇, 万健, 闫龙川. 基于遗传算法的数据中心能效仿真[J]. 计算机工程与科学, 2021, 43(08): 1341-1352.
[9]	黄山, 房六一, 徐浩桐, 段晓东, . 面向容器环境的Flink的任务调度优化研究[J]. 计算机工程与科学, 2021, 43(07): 1173-1184.
[10]	陈俊彦, 李玥, 梁楚欣, 雷晓春. SDN多控制器部署及流量均衡研究[J]. 计算机工程与科学, 2021, 43(05): 830-835.
[11]	路煜, 张路, 侯小凤, 郑文立, 李超. 数据中心功耗削峰电池的可用性分析[J]. 计算机工程与科学, 2021, 43(01): 9-16.
[12]	韩彪, 王韬, 王宝生. 无线数据中心网络：进展、挑战和展望[J]. 计算机工程与科学, 2021, 43(01): 70-81.
[13]	丁峻宏, 苗新强, 李根国. 面向异构超算的结构分析高效并行计算方法[J]. 计算机工程与科学, 2020, 42(12): 2133-2140.
[14]	王丽, 郭振华, 曹芳, 高开, 赵雅倩, 赵坤. 面向模型并行训练的模型拆分策略自动生成方法[J]. 计算机工程与科学, 2020, 42(09): 1529-1537.
[15]	张开琦, 刘晓燕, 王信, 吉春山, 严馨. 基于动态权重的一致性哈希微服务负载均衡优化[J]. 计算机工程与科学, 2020, 42(08): 1339-1344.