# AI service network traffic: accounting for egress before GPU placement

Map weights, user inputs, storage, responses, backups and updates before comparing GPU placement and complete operating costs.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/cloud-egress-model-serving/
Language: en

Placement comparisons often begin and end with GPU price. A service also retrieves weights, receives files, accesses storage and sends results. Site owners and funders need to know which transfers are included and which create separate charges.

## Map traffic directions

Identify model sources, data stores, compute nodes, clients and backups. For each connection, record direction, expected volume and component location. [Google network pricing](https://cloud.google.com/vpc/network-pricing) separates transfer categories, while [EC2 On-Demand pricing](https://aws.amazon.com/ec2/pricing/on-demand/) includes conditions to check alongside compute.

Exact amounts depend on region, product and architecture. One pricing table does not establish that ingress or egress is free across every platform.

## Separate startup and ordinary requests

Weights may be retrieved rarely while users call continuously. Frequent cold starts repeat the initial network stage, creating a different budget from one stable replica. In a pilot, measure artifact retrieval, a day's test traffic and recovery after an agreed stop. Make a local copy's presence or absence explicit.

Do not infer all networking from generated tokens. Input documents, images and service calls can transfer substantial separate data.

## Check component placement

When retrieval, models and storage are separate, a request crosses several boundaries. Cross-region or external links can affect expense and delay simultaneously. Compare one-region and distributed arrangements on the same task, recording complete-response time as well as bytes.

A cheaper compute node elsewhere does not necessarily lower total service cost. Establish data requirements and acceptable locations before moving anything. A technically reachable route does not automatically satisfy organizational data-handling conditions.

## Include updates and backups

Weight updates, log exports and data restoration belong to the lifecycle but are easily missed by a brief conversation on a prepared server. Schedule ordinary operations and a failed-update scenario. Repeated retrieval of a large artifact must appear in pilot costs.

For an owned site, check included transfer, provider limits and sustained-transfer capability. Advertised connection speed does not remove other service conditions. After testing, reconcile measured transfers with platform billing details. Differences may reveal a missing component, backup or accounting boundary; refine the map rather than simply multiplying tokens by a hypothetical network rate.

## Compare complete offers

Use a compute–storage–transfer–maintenance table, identifying included services and the current conditions source for each row. Do not compare a bare card price with an offer that already covers other infrastructure.

Read about [connection availability](/journal/network-and-uptime/) and [hybrid architecture](/journal/cpu-gpu-hybrid-architecture/).