Energy consumption is a major challenge for current high-performance computing (HPC) system design. The technique of heterogeneous computing with accelerators connected to host processor can improve the system energy efficiency, and has been widely applied in HPC. However, heterogeneous systems have lower Linpack efficiency in comparison to homogeneous systems with the equivalent scale. Aiming at this problem, from the perspective of structural design, based on the design parameters and performance data of real computing systems, this paper analyzes the main factors limiting the Linpack efficiency of heterogeneous HPC systems and their demands of structural design. Besides, the analysis is verified with a Linpack performance model tailored for heterogeneous systems. This study results are of guiding significance to Linpack performance optimization and efficient architecting on future heterogeneous systems.