In this article
An 8B weight estimate is not the full memory budget; choose a tested configuration rather than a universal model list.
For a local LLM on 16GB VRAM, choose an exact artifact and bounded workload first. An illustrative eight billion parameters at 16 bits already implies around sixteen billion bytes of weights. Extra runtime structures and unit conventions mean 8B does not simply equal a complete 16GB service.
A short list of tested configurations is more useful than a broad list of popular models described as guaranteed fits.
Start from specific model cards
Qwen3-8B and Qwen3-14B are different model sizes, not the same artifact with one switch changed. Use their official cards to understand architecture and execution requirements, then measure the chosen configuration.
Preserve the licence, revision and provenance of converted weights. Download availability and suitability for a proposed external workload are separate questions. Do not infer placement in every format from the model’s rounded name.
Define a small set of scenarios
| Scenario | Initial boundary |
|---|---|
| Short chat | One session and known questions |
| File fragment | Limited input and verifiable output |
| Longer document | Separate context-growth experiment |
| Multiple clients | After single-session validation |
Record model, format, engine, input and maximum output for each candidate. Different formats should appear as different configurations. This keeps the comparison tied to a user’s need rather than popularity.
Verify a useful result
Use questions with known answers and structured tasks with mandatory fields. Check unsupported additions as well as missing content. If the model supports a reasoning mode, include its setting in the protocol.
A timing difference may reflect generation behaviour rather than a hardware fault. Preserve failure examples: a fast configuration that does not solve the task is not a suitable service merely because it fits.
Establish the memory boundary
After loading, observe a short input and a longer input separately. Increase allowed output independently, then test concurrency. Record competing desktop or GPU processes because the full nominal 16GB is not necessarily free.
System RAM belongs in another field. If a supported setup offloads work to CPU, label it as a hybrid configuration and measure the complete result time. Do not add RAM and VRAM as one interchangeable capacity.
Record what each candidate actually passed: loading, short dialogue, typical document or several sessions. A smaller model with useful context may be preferable to a larger artifact restricted to an impractical input. Equipment assessment need not begin with purchasing a larger card.
Continue with A 14B Model on 16GB VRAM: Understand the Bit Estimate and Qwen3-8B VRAM: Thinking Mode and a First Local Test.