• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science ›› 2026, Vol. 48 ›› Issue (1): 1-10.

• High Performance Computing • Previous Articles     Next Articles

An optimization method for transmitter equalization based on neural network

SHEN Huiyi,LI Jinwen,CAO Jijun,LAI Mingche   

  1. (College of Computer Science and Technology,National University of Defense Technology,Changsha 410073,China)
  • Received:2024-06-17 Revised:2024-09-25 Online:2026-01-25 Published:2026-01-25

Abstract: With the ever-increasing demand for data transmission bandwidth in data centers and high performance computer systems, the data transmission rates of high-speed interconnection networks are increasing, while the signal transmission links are becoming increasingly complex. This places higher requirements on the equalization technology for high-speed serial communication SerDes (Serializer/Deserializer) signals. Currently, adaptive equalization can be achieved at the receiver end, but adaptive feed forward equalization (FFE) at the transmitter end remains challenging and requires manual configuration. To address this issue, this paper proposes a multi-objective optimization method for the transmitter-side FFE coefficients based on neural networks. Firstly, simulation data is collected, and a neural network is utilized to model the relationship between the FFE tap coefficients and eye height/width. Subsequently, a multi-objective optimization algorithm is applied to solve the trained neural network model, enabling the rapid determination of optimal FFE circuit tap coefficients. Compared to the traditional single-objective optimization method for FFE coefficients based on bit-by-bit simulation, the proposed method achieves a maximum improvement of approximately 25% in eye diagram area, significantly reduces time overhead, and enhances optimization efficiency.


Key words: transmitter, feed forward equalization(FFE), tap coefficient, eye diagram, neural network, multi-objective optimization algorithm