Choosing Ethernet for a training cluster requires a conversation about synchronized traffic, congestion and predictable job completion. Port speed is only one part of that conversation.
What the deployment actually establishes
NVIDIA’s October 2024 Colossus report names SN5600 switches and BlueField-3 SuperNICs in xAI’s Memphis training cluster. For the cited Grok workload, NVIDIA reported 95% data throughput. These are vendor-reported results for that system; the release does not provide a complete reproducibility protocol. NVIDIA’s deployment report
The useful purchasing lesson is to evaluate the fabric as a coordinated system. Avoid lifting a number from that report and placing it next to a different switch model.
Build an evaluation around the jobs
Start with the collective patterns your models use, the size of each job and the number of concurrent jobs. Include a large training job, a smaller job sharing the same fabric and a workload that creates bursty traffic. Record the software versions and the placement of workers for each run.
Compare outcomes the application team can interpret:
- Median and tail training-step time
- Collective completion time under contention
- Completed work during the reserved test window
- Behavior when a link is degraded or removed
- Time required to locate and correct a network fault
These are proposed evaluation measures, not published Colossus results. They provide a way to discover whether a proposed system fits the buyer’s own operating conditions.
Keep the platform boundary visible
A Spectrum-X discussion should identify the switch, endpoint, software and supported interconnects together. The cited Colossus configuration does not establish that every BlueField-3 variant has identical behavior. It also does not establish that a later switch is a drop-in replacement.
For example, the case names SN5600. A product page for SN5610 may link to this article as broader Spectrum-X context, but should say that the reported deployment used a different model. No exact ordering suffix is established by the case.
Ask for evidence that travels with the proposal
Require a topology drawing, port map, supported-component list and test record. Make the acceptance workload representative enough that both supplier and buyer can rerun it. If performance changes after a firmware update or an expansion, compare against that baseline.
Colossus is a useful reference because it identifies a real application and a real fabric. A credible selection process carries that specificity forward while keeping the buyer’s performance target separate from somebody else’s published result.
Sources checked: 10 October 2026. This article discusses third-party evidence or an explicitly labeled engineering scenario; it does not establish a reseller’s delivery history, current stock or exact-SKU deployment.
Convert the case into an evaluation worksheet
| Question for your project | Record before the trial | Evidence to retain |
|---|---|---|
| Does the fabric meet the job target? | Workload, worker placement, dataset, software and agreed completion-time target | Repeated job and collective timings under the same conditions |
| Do shared jobs interfere? | Baseline run and concurrent workloads with the same configuration | Median and tail times, congestion counters and completed work |
| Can a fault be contained? | A controlled link-loss scenario and agreed recovery target | Recovery timeline, affected jobs and operator actions |
| Are the physical links reproducible? | Exact switch/adapter/module/cable PNs, firmware, port map and installation procedure | Recognition, negotiated modes, channel checks and error counters |
This worksheet is SwitchInfra’s proposed engineering method. Its targets and results must come from the buyer’s project; they are not measurements published for Colossus. Set acceptance thresholds with the application owner before testing rather than reusing NVIDIA’s reported percentage as a purchase guarantee.
Use the AI Ethernet planning guide to size the port budget, then the LinkX complete-link guide to select exact components. The 100G / 400G and fiber comparison contains separate procurement examples; it is not the Colossus equipment list. Send the known topology and open questions for a scoped review.
Technical illustrations
