GPUs for fine-tuning
As of — QLoRA on an 8B model: from $0.12 per GPU-hour on the NVIDIA RTX A4000 (GPU.ai), 54 cards qualify; LoRA on an 8B model, 16-bit base: from $0.16 per GPU-hour on the NVIDIA V100 32GB (Geodd), 40 cards qualify; Full fine-tuning of an 8B model on one GPU: from $1.71 per GPU-hour on the AMD Instinct MI300X (TensorWave), 10 cards qualify. Rates are each card’s cheapest live on-demand rate card across the providers we track, read hourly, every one linked to the page it was read from.
Full fine-tuning with the Adam optimizer in mixed precision keeps 16 bytes per parameter: 16-bit weights and gradients (2 + 2 bytes) and 32-bit master weights, momentum and variance (4 + 4 + 4) - the accounting of the ZeRO paper. LoRA freezes the base weights and trains small adapters, so its memory is close to the weights alone; QLoRA stores those frozen weights in 4-bit, half a byte per parameter. Activations come on top in every case.
QLoRA on an 8B model
8 billion parameters × 0.5 bytes = 4 GB of frozen weights, plus adapters and activations: a card with 16 GB or more.
| Card, memory | Cheapest on-demand | Provider, read | Median rate card | $ per GB-hour | Deploy |
|---|---|---|---|---|---|
| NVIDIA RTX A4000 · 16 GB | $0.12 from |  GPU.aicaptured | $0.17 9 providers | $0.007 | Deploy ↗ |
| NVIDIA GeForce RTX 5060 Ti · 16 GB | $0.14 |  Edgevanacaptured | — 1 provider | $0.009 | Deploy ↗ |
| NVIDIA GeForce RTX 4060 Ti · 16 GB | $0.15 | Edgevana captured | — 1 provider | $0.010 | Deploy ↗ |
| NVIDIA T4 · 16 GB | $0.15 | Edgevana captured | $0.53 13 providers | $0.010 | Deploy ↗ |
| NVIDIA V100 32GB · 32 GB | $0.16 |  Geoddcaptured | $0.76 8 providers | $0.005 | Deploy ↗ |
| NVIDIA GeForce RTX 3090 · 24 GB | $0.17 | Edgevana captured | $0.50 8 providers | $0.007 | Deploy ↗ |
| NVIDIA A2 · 16 GB | $0.20 | Infomaniak captured | $0.23 2 providers | $0.012 | Deploy ↗ |
| NVIDIA RTX 4000 Ada · 20 GB | $0.20 | Edgevana captured | $0.28 5 providers | $0.010 | Deploy ↗ |
| NVIDIA V100 · 16 GB | $0.21 |  Verdacaptured | $2.52 10 providers | $0.013 | Deploy ↗ |
| NVIDIA GeForce RTX 5070 Ti · 16 GB | $0.23 | Edgevana captured | — 1 provider | $0.014 | Deploy ↗ |
44 more cards qualify, dearer: NVIDIA RTX 2000 Ada $0.24, NVIDIA GeForce RTX 4080 $0.24, NVIDIA TITAN RTX $0.24, NVIDIA Quadro RTX 6000 $0.27, NVIDIA RTX A5000 $0.27, NVIDIA L4 $0.27, NVIDIA GeForce RTX 5080 $0.30, NVIDIA A30 $0.32, NVIDIA A10 $0.32, NVIDIA RTX A6000 $0.35, NVIDIA GeForce RTX 4080 SUPER $0.37, NVIDIA RTX 4090 $0.39, NVIDIA RTX PRO 4500 Blackwell $0.39, NVIDIA RTX 4000 SFF Ada $0.44, NVIDIA RTX 5000 Ada Generation $0.45, NVIDIA GeForce RTX 3090 Ti $0.46, NVIDIA L40 $0.53, NVIDIA RTX 5090 $0.54, NVIDIA RTX 4500 Ada Generation $0.54, NVIDIA A40 $0.59, NVIDIA GeForce RTX 4070 Ti SUPER $0.62, NVIDIA RTX 6000 Ada $0.65, NVIDIA RTX PRO 6000 Blackwell $0.65, NVIDIA RTX PRO 5000 Blackwell $0.71, NVIDIA L40S $0.72, NVIDIA A100 40GB $0.76, NVIDIA A100 80GB $0.89, NVIDIA A10G $1.00, AMD Instinct MI300X $1.71, NVIDIA H100 80GB (form factor not stated) $1.71, NVIDIA H100 SXM $1.79, NVIDIA RTX PRO 6000 Blackwell Max-Q $1.80, NVIDIA H100 PCIe $1.83, NVIDIA H200 $2.09, AMD Instinct MI325X $2.25, NVIDIA GH200 $2.29, NVIDIA H100 NVL $2.92, AMD Instinct MI355X $2.95, NVIDIA H200 NVL $3.29, NVIDIA B200 $3.69, AMD Instinct MI350X $4.00, NVIDIA B300 $4.89, NVIDIA GB200 NVL72 $6.94, NVIDIA GB300 NVL72 $10.74.
LoRA on an 8B model, 16-bit base
8 billion parameters × 2 bytes = 16 GB of frozen weights, plus adapters and activations: a card with 24 GB or more.
| Card, memory | Cheapest on-demand | Provider, read | Median rate card | $ per GB-hour | Deploy |
|---|---|---|---|---|---|
| NVIDIA V100 32GB · 32 GB | $0.16 | Geodd captured | $0.76 8 providers | $0.005 | Deploy ↗ |
| NVIDIA GeForce RTX 3090 · 24 GB | $0.17 | Edgevana captured | $0.50 8 providers | $0.007 | Deploy ↗ |
| NVIDIA TITAN RTX · 24 GB | $0.24 | Edgevana captured | — 1 provider | $0.010 | Deploy ↗ |
| NVIDIA Quadro RTX 6000 · 24 GB | $0.27 GPU only |  centroncaptured | $0.51 4 providers | $0.011 | Deploy ↗ |
| NVIDIA RTX A5000 · 24 GB | $0.27 from | GPU.ai captured | $0.57 8 providers | $0.011 | Deploy ↗ |
| NVIDIA L4 · 24 GB | $0.27 GPU only |  Hikubecaptured | $0.75 22 providers | $0.011 | Deploy ↗ |
| NVIDIA A30 · 24 GB | $0.32 |  SynpixCloudcaptured | $0.41 7 providers | $0.013 | Deploy ↗ |
| NVIDIA A10 · 24 GB | $0.32 GPU only |  Intelion Cloudcaptured | $1.22 9 providers | $0.013 | Deploy ↗ |
| NVIDIA RTX A6000 · 48 GB | $0.35 |  Thunder Computecaptured | $0.55 17 providers | $0.007 | Deploy ↗ |
| NVIDIA RTX 4090 · 24 GB | $0.39 |  CloudRiftcaptured | $0.48 15 providers | $0.016 | Deploy ↗ |
30 more cards qualify, dearer: NVIDIA RTX PRO 4500 Blackwell $0.39, NVIDIA RTX 5000 Ada Generation $0.45, NVIDIA GeForce RTX 3090 Ti $0.46, NVIDIA L40 $0.53, NVIDIA RTX 5090 $0.54, NVIDIA RTX 4500 Ada Generation $0.54, NVIDIA A40 $0.59, NVIDIA RTX 6000 Ada $0.65, NVIDIA RTX PRO 6000 Blackwell $0.65, NVIDIA RTX PRO 5000 Blackwell $0.71, NVIDIA L40S $0.72, NVIDIA A100 40GB $0.76, NVIDIA A100 80GB $0.89, NVIDIA A10G $1.00, AMD Instinct MI300X $1.71, NVIDIA H100 80GB (form factor not stated) $1.71, NVIDIA H100 SXM $1.79, NVIDIA RTX PRO 6000 Blackwell Max-Q $1.80, NVIDIA H100 PCIe $1.83, NVIDIA H200 $2.09, AMD Instinct MI325X $2.25, NVIDIA GH200 $2.29, NVIDIA H100 NVL $2.92, AMD Instinct MI355X $2.95, NVIDIA H200 NVL $3.29, NVIDIA B200 $3.69, AMD Instinct MI350X $4.00, NVIDIA B300 $4.89, NVIDIA GB200 NVL72 $6.94, NVIDIA GB300 NVL72 $10.74.
Full fine-tuning of an 8B model on one GPU
8 billion parameters × 16 bytes = 128 GB: a card with 141 GB or more, or the states split over several GPUs, which is what ZeRO does.
| Card, memory | Cheapest on-demand | Provider, read | Median rate card | $ per GB-hour | Deploy |
|---|---|---|---|---|---|
| AMD Instinct MI300X · 192 GB | $1.71 from |  TensorWavecaptured | $2.81 8 providers | $0.009 | Deploy ↗ |
| NVIDIA H200 · 141 GB | $2.09 from |  Beamcaptured | $4.54 45 providers | $0.015 | Deploy ↗ |
| AMD Instinct MI325X · 256 GB | $2.25 |  Bentauscaptured | $2.25 3 providers | $0.009 | Deploy ↗ |
| AMD Instinct MI355X · 288 GB | $2.95 from | TensorWave captured | $5.99 3 providers | $0.010 | Deploy ↗ |
| NVIDIA H200 NVL · 141 GB | $3.29 from | GPU.ai captured | $3.50 6 providers | $0.023 | Deploy ↗ |
| NVIDIA B200 · 192 GB | $3.69 |  Deep Infracaptured | $7.20 32 providers | $0.019 | Deploy ↗ |
| AMD Instinct MI350X · 288 GB | $4.00 | CloudRift captured | $5.08 2 providers | $0.014 | Deploy ↗ |
| NVIDIA B300 · 288 GB | $4.89 | Deep Infra captured | $8.99 20 providers | $0.017 | Deploy ↗ |
| NVIDIA GB200 NVL72 · 186 GB | $6.94 from |  Bitdeer AIcaptured | $13.25 6 providers | $0.037 | Deploy ↗ |
| NVIDIA GB300 NVL72 · 288 GB | $10.74 | Verda captured | $18.00 3 providers | $0.037 | Deploy ↗ |
How these figures are read
A card qualifies by its memory alone; speed, interconnect and software support are not ranked here. The rate is the card’s cheapest live on-demand rate card - each provider once, at its cheapest on-demand rate, community capacity and marketplace floors out, read within 48 hours - and the median is taken over the same rate cards, as on every card page. Memory figures are floors from the arithmetic above, not measurements: context length, batch size and the serving stack change them. Methodology.
Sources
- Rajbhandari et al., ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (2019)
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (2021)
- Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (2023)
Other workloads: GPUs for LLM inference · GPUs for training · GPUs for image generation · Cheapest GPU memory per hour.
Data updated: