Why is NVIDIA HGX H100 an in-demand GPU for AI?

NVIDIA announced the HGX H100 in March 2022 with the first processors available to customers in October 2022. It quickly became the reliable solution for demanding compute tasks like High-Performance Computing (HPC) and Artificial Intelligence (AI) due to its flexibility and scalability.

Many GPU Cloud providers limit access to HGX H100 with long-term contracts, but Voltage Park offers the flexibility of both On-Demand GPU and Long Term Reserve.

The feedback we get on HGX H100 from the industry’s most-prominent AI companies is straight and to the point: “they’re reliable; they just work.”

What is NVIDIA HGX?

NVIDIA HGX H100 is a GPU used on an NVIDIA HGX server, which begs the question: what is HGX?

HGX stands for “Hyperscale Graphics eXtension,” which is a modular hardware design specification created by NVIDIA that defines standards for hardware interfaces and interconnects. It is a scalable, flexible platform that allows Original Equipment Manufacturers (OEMs) to customize their designs of NVIDIA 8x GPU systems. Because of its flexibility, it is useful for modular deployment in settings like data centers, where compute requirements may increase.

HGX is an evolution of their systems brand, DGX (Deep GPU Xceleration), which had largely fixed configurations.

The NVIDIA HGX H100 uses the NVIDIA Hopper GPU architecture, named in honor of computer scientist and mathematician Grace Hopper. NVIDIA Hopper architecture was built specifically to handle complex compute tasks that require large-scale parallel processing, such as training large AI models, AI inference, data analytics, and prediction and forecasting. One feature of the NVIDIA Hopper GPU architecture are its fourth-generation tensor cores.

How Does the NVIDIA HGX H100 Accelerate AI?

The NVIDIA HGX H100 was designed and optimized with the high performance computing (HPC) requirements of large-scale parallel processing in mind, making it an industry standard for HPC workloads like AI training and inference.

High Memory Bandwidth. Complex compute tasks, such as large-scale simulations, benefit from high memory bandwidth to support working with large datasets. HGX H100 has HBM3 ("High Bandwidth Memory 3"), a 3D stacking architecture memory technology that achieves extremely high data transfer speeds. Each HGX H100 has HBM3 memory with a capacity of up to 80 GB and approximately 2 TB/s memory bandwidth.

High Speed, Low Latency. The NVIDIA Quantum-2 InfiniBand platform features software-defined networking, In-Network Computing, performance isolation, advanced acceleration engines, remote direct-memory access (RDMA), and speeds up to 400Gb/s.

Enhanced Interconnect for Scalability. Multi-GPU communication is critical for scaling workloads. NVIDIA NVLink Network Interconnection is a technology that maximizes throughput by enabling high-speed, direct connection between multiple NVIDIA GPUs within a server. This allows for multi-node configurations connecting up to 256 GPUs across multiple compute nodes.

Support for Difference Precisions. NVIDIA HGX H100 supports different precisions, including FP64, TF32, FP16, and INT8. Precision support is important for developing and training neural networks. Single-precision training can be computationally expensive. The option to leverage mixed-precision training can speed up computation speed, lowering memory use without sacrificing accuracy.

Secure Multi-Instance GPU (MIG). MIG technology provides a high level of security and isolation between different workloads running on multiple isolated instances on a single physical GPU. This creates resource efficiency and enables scalability, since smaller workloads can run in parallel on a single GPU rather than requiring additional physical GPUs for each task or workload.

Outro: Harness the power of HGX H100 for your AI compute

NVIDIA HGX H100 remains the industry workhorse for HPC and AI tasks. Are you ready to accelerate the development of your AI platform and capabilities?

At Voltage Park, we make it easy to access a GPU Cloud powered by HGX H100 with both long term reserve and on-demand GPU. We focus on democratizing access to powerful, cost-efficient compute tools.

Sign up for On-Demand GPU within minutes or speak with one of our Sales Engineers about long-term reserve.