Accelerating Scientific Computing Applications with NVIDIA SHARP In-Network Computing Technology

NVIDIA Mellanox Scalable Hierarchical Aggregation and Reduction Protocol (SHARP) technology improves upon the performance of MPI and machine learning collective operation by offloading collective operations from the CPU or GPU to the network and eliminating the need to send data multiple times between endpoints. This innovative approach decreases the amount of data traversing the network as aggregation nodes are reached, and dramatically reduces collective operations time. Implementing collective offloads communication algorithms supporting streaming for machine learning in the network also has additional benefits, such as freeing up valuable CPU resources for computation rather than using them to process communication. We'll present an in-depth overview of the SHARP architecture and how to leverage its open API to accelerate AI and HPC applications