In this article
Peak internet speed does not establish inference readiness. Connection stability, the API path, recovery and an honestly stated operating schedule matter.
When assessing a site, megabits per second are an easy starting point. They are insufficient for an operating service. A fast but frequently interrupted connection may cause more problems than a predictable link with a lower maximum speed.
Test the network against its intended use: downloading models, ordinary API requests and recovery after interruptions are different tasks.
Separate file transfer from API operation
Initial weight downloads can be large, while a typical text request is much smaller. This does not imply every connection suits inference. Latency, connection stability, provider restrictions and concurrent request volume matter.
Check external connectivity and site restrictions. Do not expose administrative ports to the internet just to simplify a preliminary test: agree on the access design separately.
Describe availability unambiguously
Distinguish a powered-on machine, reachable network, running model and ready API. A server may answer diagnostics while the model is still loading. Conversely, the process may run while a network fault blocks external clients.
Record interruption causes and durations separately in reports. This makes needed improvements and responsibility easier to identify.
Test recovery
Include process restart, connection loss and ordinary reboot in the pilot plan. Measure the interval from restored power or networking to request readiness. Do not test failures against someone else’s working jobs without an agreed window.
Define an incident contact and expected response. If someone can reach the server only on weekdays, do not promise overnight physical intervention. Remote diagnosis cannot replace a person when hardware needs servicing.
Agree on planned interruptions
Maintenance, your own research workloads and site moves should appear in the schedule. Specify advance notice and how a node is drained. A predictable break is easier to handle than a machine disappearing unexpectedly.
For a new pool, measure actual site behavior before discussing availability commitments. Read the security guide for server access.
Related reading: GPU inference pilot: what to check before an operating agreement.