Abstract: Currently, intelligent computing centers primarily employ RDMA (remote direct memory access)protocol to achieve ultra-high-performance communication within clusters, where each pair of processes needs to establish a queue pair (QP) based on the reliable connection (RC) type. In the context of AI  large model scenarios in next-generation large-scale intelligent computing centers, distributed collective communication operations such as All-to-All and All Reduce will trigger fully connected communication between processes. The number of QPs that need to be maintained under the RC-based mechanism will exceed one million, posing significant challenges to the limited memory and performance of RDMA network interface cards (NICs). To address this issue, an RDMA QP communication mechanism named ERD (efficient reliable datagram) is proposed. On one hand, it replaces traditional RC with RD (reliable datagram) to enhance the scalability of QPs on NICs; on the other hand, it designs an RD-based reliable reception mechanism that incorporates packet loss handling and rapid ordered processing in the network stack, ensuring network reliability while improving transmission performance. Through experiments and NS3 simulation tests, ERD can reduce the number of QPs by 99.96% and enhance transmission performance by over 15% during network congestion.

Key words: intelligent computing center network, AI large-model communication, RDMA, QP communication