# LLM hosting architectures: five arrangements for different tasks

Compare five AI hosting arrangements by readiness, resource roles, data boundaries, failure handling and the unit of useful output.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/top-hosting-architectures-llm/
Language: en

An AI service can be hosted in several ways, moving costs and responsibilities between components. This helps GPU owners identify a role for their equipment and funders understand what an agreed budget creates. The following is an editorial architecture comparison, not a provider ranking or common benchmark.

## Comparison criteria

We consider readiness, resource composition, data boundaries, updates and the unit of output. Price cannot be ranked without region, hardware and request profile. [EC2 classification](https://docs.aws.amazon.com/ec2/latest/instancetypes/ec2-instance-type-specifications.html) distinguishes compute, storage, memory-optimized and accelerated resources. [Kubernetes GPU scheduling](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/) describes one device-management method, not a product-architecture decision.

System RAM, VRAM and vCPU remain different characteristics. No arrangement automatically replaces a shortage of one with the sum of the others.

## Five arrangements to consider

| Arrangement | When to test | Main pilot question |
| --- | --- | --- |
| CPU application and external model | Early task exploration | Complete answer and API conditions |
| One GPU server | A clear local setup | Combined resource load |
| Separate retrieval and GPU model | Distinct stage profiles | Networking and failure boundaries |
| Scheduled GPU batch | Work permits a timetable | Completing by the deadline |
| Startup on demand | Irregular requests | Cold start and readiness |

[Qdrant capacity planning](https://qdrant.tech/documentation/capacity-planning/) illustrates retrieval's independent data-and-index budget. [Cloud Run GPU](https://cloud.google.com/run/docs/configuring/services/gpu) is a particular managed capability whose conditions do not apply to all serverless offers.

## Start with the smallest testable chain

Choose one task and define a correct result. For a document assistant, use questions with known relevant fragments and answer requirements. Test preparation, retrieval and model separately, then the full cycle. Infrastructure scaling does not resolve a product problem if a useful answer is still absent.

Preserve versions and data profiles. Comparing different models or documents can be useful research, but is not a clean comparison of hosting arrangements.

## Test readiness and failure

A single server has fewer components, but maintenance may stop the entire chain. Separation adds interfaces and network dependencies. For batches, test restart after interruption. For on-demand startup, test the first post-idle request. For an external API, check request restrictions and unavailability handling.

One successful example does not demonstrate operational resilience. Every selected arrangement needs recovery conditions and responsible operators.

## Resource roles in the architecture

A GPU, high-RAM server and operating funds can support different parts of the task. Identify the resource's role before choosing attachment and budget.

Configurations are considered individually; ready-made implementations of all five arrangements are not announced. Prepare with [CPU and GPU hosting](/journal/cpu-vs-gpu-hosting/), [capacity planning](/journal/inference-capacity-planning/) and [a technical pilot](/journal/pilot-checklist/). Let measurement and owner conditions determine the choice.