# RTX 5090 LLM Tokens per Second: Define the Benchmark

Interpret RTX 5090 LLM token rates by model, numerical format, request lengths, concurrency, first-token delay and successful output.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/rtx-5090-tokens-per-second/
Language: en

There is no single LLM tokens-per-second figure for the RTX 5090. Throughput belongs to a specific model, weight format and request stream. One session's generation rate and aggregate output from many sessions measure different things.

Before purchasing a card or offering it to a pool, reconstruct the conditions behind the result that interests you. Missing benchmark details are a reason to repeat the test, not a performance promise.

## Start with what the specification establishes

NVIDIA specifies Blackwell and 32GB GDDR7 for the RTX 5090. These facts identify the device. Peak arithmetic operations, gaming frames and generated tokens are not interchangeable metrics.

Record the actual product and host. Power, cooling and competing processes can change operating conditions even when the GPU model is the same. Preserve normal protections during the experiment.

## Identify the measured quantity

The vLLM serving benchmark exposes workload settings rather than defining one universal vLLM test.

| Metric | Detail needed |
| --- | --- |
| Single-session rate | Whether first-token waiting is included |
| Aggregate output | Number of active requests |
| Response duration | Input and output lengths |
| Success rate | Treatment of errors and cancellations |
| Recovery | Whether model loading is included |

Label output tokens separately from input tokens. A high combined value can be misleading if readers assume it describes the experience of one interactive user.

## Build a bounded test set

Use one compatible model and preserve its weights and revision. Prepare short and long inputs with known quality checks. Start with a single request, then add concurrency while holding comparable output limits fixed.

Record shared prompt prefixes and cache conditions. Repeated instructions in a warm cache can produce a different workload from unrelated documents. Both are valid scenarios when labelled clearly; combining them without explanation hides a material condition.

## Repeat an ordinary operating interval

After initial runs, observe a sustained agreed interval. Record memory, temperature, errors and successful outputs. Also measure readiness after restarting the service, especially if weights must be fetched again.

For a cost comparison, use the complete machine and useful completed work. Do not multiply a best session rate by a month and present the result as sales capacity or revenue. A defensible conclusion states what the described configuration achieved on the described stream.

A measured, useful configuration provides a stronger basis for a pilot discussion than an unqualified token-rate claim.

Continue with [RTX 4090 vs RTX 5090 for LLMs: Compare 24GB and 32GB](/journal/rtx-4090-vs-5090-llm/) and [vLLM bench serve: Read a Serving Benchmark Result](/journal/vllm-bench-serve-command/).

## Official references

- [www.nvidia.com — rtx 5090](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/)
- [vLLM — serve](https://docs.vllm.ai/en/stable/cli/bench/serve/)