How to choose a cloud GPU for LLM fine-tuning

Start with the memory budget, not the FLOPs

Fine-tuning a language model is a memory problem before it is a compute problem. Weights, gradients and optimiser state all have to fit, and the optimiser state is usually the largest of the three. Before you compare hourly rates, work out whether your target model fits on a single 80 GB device or whether you need tensor parallelism across several.

LoRA changes the economics

Parameter-efficient methods such as LoRA and QLoRA keep the base weights frozen and train a small adapter, which removes most of the optimiser state. That single change often moves a job from a multi-GPU H100 node down to a single A100 or L40S instance.

Comparing providers fairly

Hourly rates are only comparable once normalised to a single GPU and to the same interruptibility tier. Marketplace capacity is cheaper but can be reclaimed; dedicated cloud costs more and behaves predictably. Match that trade-off to how long your job runs and whether it checkpoints regularly.

Watch storage and egress

Datasets and checkpoints are billed separately from GPU hours, and egress charges accumulate quickly when you pull artefacts out. Budget for them explicitly rather than discovering them on the invoice.

Frequently asked questions

Do I need multiple GPUs to fine-tune a 7B model?

Usually not. With LoRA or QLoRA a 7B model fits comfortably on a single 48 GB or 80 GB device, and often on a 24 GB card with 4-bit quantisation.

Is marketplace capacity safe for long training jobs?

Only if your job checkpoints frequently. Interruptible capacity can be reclaimed, so treat it as suitable for work you can restart cheaply.

B2B bulk GPU enquiry

Reserving large amounts of GPU capacity? Tell us what you need and we will forward your request to the right provider team.