# Spot or on-demand GPUs: accounting for interruptions

Compare interruptible and ordinary GPU rental through recovery, repeated work, replacement availability and the cost of useful completed jobs.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/spot-vs-on-demand-gpu/
Language: en

Interruptible GPU capacity may look cheaper than ordinary rental. The outcome depends on work lost at interruption and how quickly replacement capacity becomes available. For equipment owners and GPU-pool funders, this concerns architecture and operating budgets as much as choosing a tariff.

## Understand the two modes

[AWS Spot Instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html) use spare capacity for flexible, interruptible work. [Google Spot VMs](https://cloud.google.com/compute/docs/instances/spot) have their own provisioning and interruption conditions. Check exact rules, GPU availability and pricing for the intended region and configuration. One cloud's conditions do not apply universally to every offer labelled spot.

On-demand rental still needs recovery from failures. What differs is the resource-provisioning model, not the existence of universal protection against application interruption.

## Choose a suitable workload

Background processing of independent documents differs from an interactive session with a waiting user. First determine whether work can be divided into completed units with results saved between them. For an illustrative archive, record each document's ID, processing status and result location. A restarted worker should recognize completed items instead of restarting the entire set.

This is an application requirement rather than a property conferred by the rental label. Preserve the expected completion deadline and identify which interruptions the workflow can tolerate before judging the apparent hourly saving.

## Rehearse interruption and recovery

Stop an agreed test workload within a controlled window and restore it on an available configuration. Measure obtaining the machine, loading the model and returning the application to readiness. Account separately for recomputation and data transfer.

A low hourly rate does not establish a low cost per completed batch if substantial work is repeated. Also preserve the case where no suitable replacement is available. For a deadline-sensitive job, this may matter more than average recovery time under successful provisioning.

## Compare a mixed scheme

A persistent base resource with extra interruptible capacity is one architecture to test for suitable tasks. Show useful completions, waiting, restarts and total elapsed time in the pilot record. Maximum GPU utilization is insufficient: recomputing lost work also creates utilization.

Use a bounded experiment budget to discover where flexibility justifies additional complexity without promising savings or returns. Distinguish resource interruption, application error and rejected invalid input in the batch log. They require different repeat actions; merging them into one error rate makes costs and recovery choices harder to explain.

## Assessment notes

Owners can propose resources with defined availability windows. Describe them accurately: a privately owned machine is not automatically equivalent to cloud spot capacity, although its breaks also require planning.

Related reading: [Hosting batch inference: using GPUs on a schedule](/journal/batch-inference-hosting/); [Reserved GPUs and flexible capacity: what are you committing to?](/journal/reserved-vs-flexible-capacity/).