NVIDIA A40 vs H100 for Local AI

Short answer: choose NVIDIA A40 when a measured workload fits 48GB and a lower-power Ampere PCIe server card, vGPU/display capability or supported two-card NVLink path meets the requirement. Choose H100 only when 80GB or 94GB, Hopper software, MIG or a qualified NVLink/NVSwitch platform solves a measured constraint. “H100” must be resolved to PCIe 80GB, SXM 80GB or NVL 94GB before comparing price or fit.

A40 vs H100 specifications

Hardware	Architecture / capability	Memory / bandwidth	Power	Deployment clues
NVIDIA A40	Ampere / 8.6	48GB GDDR6 ECC / 696GB/s	300W	Passive dual-slot PCIe; CPU 8-pin power; vGPU/configurable display; no MIG; two-way NVLink
H100 PCIe	Hopper / 9.0	80GB HBM2e / 2TB/s	350W maximum	Passive dual-slot PCIe accelerator; MIG
H100 SXM	Hopper / 9.0	80GB HBM3 / 3.35TB/s	Up to 700W	SXM module for compatible HGX/DGX; MIG and NVLink/NVSwitch platform
H100 NVL	Hopper / 9.0	94GB HBM3 / 3.9TB/s	350–400W	Dual-slot air-cooled PCIe data-center product; MIG and paired-NVL configuration

The products sit in different generations and roles. A40 combines a 48GB Ampere server/virtual-workstation profile with display, media and vGPU features. H100 is a Hopper compute accelerator with larger HBM capacity, MIG and variant-specific scale-up fabric. Validate the exact driver, CUDA, framework, kernels, virtualization licensing and host qualification instead of assuming a product-family feature is included.

Exact Qwen3.5 capacity boundaries

Margins below subtract decimal file totals from nominal decimal memory. They exclude KV cache, hybrid-attention state, vision processing, allocator behavior, runtime workspaces and temporary buffers. A positive remainder is not proof of runtime fit.

Artifact	Files	A40 48GB	H100 80GB	H100 NVL 94GB
Qwen3.5-35B-A3B Q8_0 + F16 projector	37.802GB	10.198GB	42.198GB	56.198GB
Qwen3.5-35B-A3B official BF16	71.904GB	Does not fit files	8.096GB — bounded	22.096GB
Qwen3.5-122B-A10B Q4_K_M + projector	77.446GB	Does not fit files	2.554GB — unsafe	16.554GB
Qwen3.5-122B-A10B official GPTQ-Int4	78.864GB	Does not fit files	1.136GB — unsafe	15.136GB
Qwen3.5-122B FP8 / Q8 + projector / BF16	127.163 / 130.781 / 250.173GB	Does not fit files	Does not fit files	Does not fit files

H100 80GB changes the listed 35B BF16 file-fit result but leaves only 1.136GB or 2.554GB for the two listed 122B 4-bit paths—unsafe before runtime state. H100 NVL 94GB raises those margins to 15.136GB and 16.554GB. None of these single devices holds the listed 122B FP8, Q8-plus-projector or BF16 files. Capacity does not establish throughput.

Cooling, power and form factor are purchase gates

• A40 and H100 PCIe are server products. A40 and ordinary H100 PCIe depend on documented directed chassis airflow; physical fit in a tower is insufficient.
• A40 power: NVIDIA specifies an 8-pin CPU connector. Do not substitute a superficially similar PCIe GPU cable.
• H100 SXM is not PCIe. A bare module requires the matching baseboard, cooling, power, firmware and management stack.
• H100 NVL is a separate 94GB product. Confirm qualified chassis, power mode and paired-NVL details rather than treating every “H100” listing as equivalent.
• MIG changes visible memory. H100 can be partitioned; record the exact profile and memory exposed to the process. A40 does not support MIG.

NVLink does not create one transparent memory pool

A40 supports a two-way low-profile NVLink bridge, while H100 interconnect depends on the exact PCIe, SXM or NVL platform. In every case, the runtime must explicitly partition or parallelize the workload. Two 48GB A40 cards do not automatically behave like one 96GB device, and aggregate H100 capacity does not remove per-device placement, duplicated-allocation or headroom constraints.

Which should you buy?

• Choose A40 when the complete measured workload fits 48GB and a qualified passive PCIe host, Ampere support, vGPU/display behavior or supported two-card design is the better economic fit.
• Choose H100 PCIe 80GB when crossing 48GB or using MIG solves a requirement and the passive PCIe host is already qualified.
• Choose H100 SXM only as part of a compatible HGX/DGX system whose topology, cooling, power, software and delivered cost are justified.
• Choose H100 NVL 94GB when the additional 14GB over ordinary H100 materially changes the measured workload and the exact qualified PCIe/NVL system is available.
• Benchmark complete systems when speed or efficiency decides: hold artifact, runtime, precision, context, concurrency, placement and timing definition constant.

Used-market checklist

• Verify the exact label and part number: A40, H100 PCIe 80GB, H100 SXM 80GB or H100 NVL 94GB.
• Request current device identity, visible memory, health/error evidence, configured power and any MIG state.
• Document the complete host, directed airflow, A40 CPU-power cable, H100 power delivery, PCIe lanes or SXM platform.
• Confirm driver/runtime support, compiled kernels and any vGPU, AI Enterprise or virtualization licensing.
• Compare delivered-system cost, warranty and return terms through the screened-offers directory and used-hardware checklist.

Continue: read the A40 guide, H100 variant guide, 48GB vs 80GB capacity guide, VRAM calculator, or save private criteria and follow RSS.

Sources and limits

• NVIDIA A40 datasheet
• NVIDIA A40 product page
• NVIDIA A40 product brief and NVLink requirements
• NVIDIA H100 product page
• NVIDIA H100 GPU datasheet
• NVIDIA H100 PCIe board specification
• NVIDIA MIG supported-GPU table
• Qwen3.5-35B file provenance and Qwen3.5-122B file provenance

Reviewed October 25, 2026. Specifications are sourced from NVIDIA and may vary by SKU, revision and configured power mode. HighVRAM has not independently benchmarked an A40–H100 pair. Used configurations, licensing, availability and prices change.

Newer consumer GPU or Hopper accelerator? Compare RTX 5090 with H100 80GB/94GB across exact Qwen3.5 boundaries and complete-system requirements.