In this article
Start with one reproducible model configuration on the existing 3090, then identify the request profile it can actually sustain.
On an RTX 3090 with its standard 24GB VRAM, choose a local LLM by exact artifact and bounded workload rather than parameter count alone. Load one known configuration, measure memory and verify useful output. Expand context and concurrency only after that baseline works.
A successful first test identifies a role for existing hardware. It does not establish that every model with a similar name or every advertised context window fits.
Record the starting machine
NVIDIA’s specification identifies the card’s 24GB capacity. Add driver version, CPU, RAM, storage and the applications already sharing the GPU. Actual free memory is more relevant to the test than assuming the entire nominal capacity is available.
For previously used hardware, include ordinary cooling and a condition check. Do not begin model selection by overclocking, rebuilding the owner’s whole workstation or buying components before finding a limitation.
Choose one specific artifact
Qwen3-8B is an example of an official model card to inspect, not a guarantee of every context length on a 3090. Preserve the revision, numerical format, source and loading method.
| Record | Why it matters |
|---|---|
| Model identifier | Distinguishes the artifact |
| Revision and format | Makes the setup reproducible |
| Engine version | Defines execution support |
| Input and output limits | Defines the workload |
| Licence and intended use | Requires separate review |
Downloading weights does not establish suitability for a particular commercial use. Converted files also need a clear relationship to the original model.
Begin with quality checks
Prepare known-text questions, required-field output and an instruction requiring no additional prose. Define success before running the model. One attractive free-form answer is not an adequate quality measure.
Use one session with moderate limits. Record first-token delay, completion time, errors and memory. If a reasoning mode is involved, include its settings because the generation profile can differ from a short direct answer.
Expand one factor at a time
Increase input length with the output limit fixed, then test longer output separately. Add simultaneous requests afterwards. Repeat the original control case following changes; if it deteriorates too, investigate the environment and background work.
Save the working configuration and three control requests, then repeat after a restart. Reliance on an undocumented old session means preparation remains incomplete. The next experiment can then begin from evidence rather than choosing a model again from the GPU name.
Continue with Calculate Weight Memory for an 8B Model and RTX 3090 vs RTX 4090 for LLMs: Compare the Same Workload.