For optimal AI performance, Dawning AI servers should use high-bandwidth, low-latency NICs with RDMA/RoCEv2 enabled, mapped per GPU via PCIe switches, and integrated into a lossless scale-out fabric.

Network Fabric Overview

Dawning AI servers rely on multiple network fabrics to support AI workloads:

• Front-End Fabric: Connects applications, storage, and GPU cluster management. Typically requires 1–2 NICs per server .
• Internal-AI Fabric: Integrates GPUs, CPUs, NICs, and storage via PCIe switches for efficient peer-to-peer communication .
• Scale-Up Fabric: Handles intra-server GPU-to-GPU communication using NVLink, Infinity Fabric, or Ethernet .
• Scale-Out Fabric: Connects multiple AI servers for distributed workloads. Requires one NIC per GPU with matching PCIe bandwidth (e.g., PCIe Gen5 x16 or 400Gbps) to ensure non-blocking, low-latency transfers .

NIC Configuration Guidelines

1. RDMA and RoCEv2:
   • Enable RoCEv2 on Broadcom or compatible NICs to offload transport tasks from the CPU and provide direct memory access between GPUs and network .
   • Ensure NICs support UDP/IP routable RoCEv2 for cluster-wide communication.
2. PCIe Mapping:
   • Each GPU should have a dedicated NIC connected through a PCIe switch to avoid bottlenecks .
   • Verify PCIe bandwidth matches NIC throughput (e.g., 400Gbps NICs require PCIe Gen5 x16 lanes).
3. Server BIOS Settings:
   • For AMD-based servers (e.g., Supermicro AS-8125GS-TNMR2), disable IOMMU and ACS if required by the workload .
   • Dell XE9680 servers may require careful SR-IOV configuration; consult vendor documentation to avoid boot faults .
4. GPU-NIC Topology:
   • Use tools like rocm-smi for AMD GPUs to verify hardware topology and NIC mapping .
   • Ensure GPUs and NICs are correctly paired for optimal RDMA performance.

Network Design Considerations

• Lossless Ethernet: Configure Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) on switches to prevent packet loss during high-throughput AI workloads .
• Spine-Leaf Architecture: Recommended for scale-out fabrics to reduce latency and provide sufficient bandwidth for multi-node training .
• Quality of Service (QoS): Assign DSCP values (e.g., CS3) for RoCEv2 traffic and configure class-maps and policy-maps on switches to maintain low-latency, high-priority traffic .

Monitoring and Validation

• Use NIC and GPU management tools to monitor throughput, latency, and RDMA performance.
• Validate network paths and congestion management to ensure non-blocking communication across the AI cluster . By following these guidelines, Dawning AI servers can achieve high-throughput, low-latency networking, essential for distributed AI training and inference workloads.