Choosing a cloud GPU for LLM fine-tuning

Start with the memory budget, not the FLOPs

Fine-tuning a language model is a memory problem before it is a compute problem. Weights, gradients and optimiser state all have to fit, and the optimiser state is usually the largest of the three. Before you compare hourly rates, work out whether your target model fits on a single 80 GB device or whether you need tensor parallelism across several.

LoRA changes the economics

Parameter-efficient methods such as LoRA and QLoRA keep the base weights frozen and train a small adapter, which removes most of the optimiser state. That single change often moves a job from a multi-GPU H100 node down to a single A100 or L40S instance.

Comparing providers fairly

Hourly rates are only comparable once normalised to a single GPU and to the same interruptibility tier. Marketplace capacity is cheaper but can be reclaimed; dedicated cloud costs more and behaves predictably. Match that trade-off to how long your job runs and whether it checkpoints regularly.

Watch storage and egress

Datasets and checkpoints are billed separately from GPU hours, and egress charges accumulate quickly when you pull artefacts out. Budget for them explicitly rather than discovering them on the invoice.