The internal network system of the high-performance computing center is divided by function into the monitoring management system, computing network and shared memory network. For security and stability of the system, different functional networks should be separated physically. The monitoring management network has a lower load and does not require a high performance. Generally the computing network and shared memory network has the same physical network. Generally they have a higher load and require higher performance.

Generally the monitoring management network uses 1000M Ethernet. For the large-scale high-performance computing center, 1000M switching and 10000M uplink solution is recommended. The switching module of the Sugon TC3600/4600 blade system is aggregated to the 10000M core switch via 10000M uplink module as the Ethernet switch at the access layer (including 1000M switch connected to the rack node). Compared with the traditional blade direct access scheme, the 10000M uplink Ethernet scheme features simple wiring and high reliability. Compared with the traditional blade 1000M switch access Ethernet, it can guarantee high performance Ethernet connection.

The computing network is mainly used for data exchange of parallel application. The MPI-based parallel high-performance computing programs will has a large number of communication data in multi-node parallel operation. The performance of the computing network determines the computing performance, parallel acceleration and scalability of the parallel program. If the communication data of the parallel computing program mainly includes small data packets and the data exchange frequency is high, these parallel programs will be sensitive to the delay performance of the computing network. Lower computing network delay leads to better parallel performance of the program. If there are more communication program large data packets of the parallel computing, it will be very sensitive to the bandwidth performance of the computing network. Higher computing network bandwidth leads to better parallel performance. In practice, most parallel applications depend on the bandwidth and delay performance of the computing network much. The low-delay and high-bandwidth computing network is necessary for large-scale parallel computing.

On the other hand, the computing network is generally also used as the shared memory network. Now all large-scale high-performance computing centers are designed as the distributed parallel memory architecture and have a large-scale cluster or the application program requires high disk I/O performance, so they require high performance, low delay and high bandwidth of the memory network of the parallel memory system.

Therefore, the industry-leading high-bandwidth and low-delay InfiniBand high-speed network is recommended for the computing and shared memory network, which can remarkably improve parallel efficiency of the parallel programs and performance of shared memory system.

Figure VSAP parallel efficiency test of different computing networks