# SGLang in Docker: launch and verify your first model

Launch SGLang in Docker on a compatible NVIDIA environment, retain model files and verify readiness, model identity and a repeated clean start.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/sglang-docker-launch/
Language: en

A first SGLang Docker launch needs a compatible host, accelerator access inside the container and a supported model. Verify those prerequisites, then test readiness and one small request. Do not treat the image name or a running container as evidence that the full serving path works.

## Match the image to the host

Use the current installation documentation for the hardware and driver requirements. Record GPU, operating system, driver and chosen image. A mutable tag is convenient for an example, but a reproducible pilot should preserve the actual tested image version or digest.

Confirm Docker can provide the intended GPU access before loading a large model. A host-level device check does not by itself establish visibility inside the container. Keep the first failure log to distinguish preparation, discovery and model-loading problems.

## Run a bounded local example

The source example for a compatible NVIDIA setup is:

```bash
docker run --rm --gpus all --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -v sglang-hf-cache:/root/.cache/huggingface \
  lmsysorg/sglang:latest \
  sglang serve Qwen/Qwen3-0.6B \
  --host 0.0.0.0 --port 30000
```

It illustrates the command structure; it does not install the driver. The container listens on its reachable interface while the host publication is restricted to loopback. Use a supported model and verify the current requirements before executing it in your environment.

## Wait for real readiness

Initial preparation can include downloading, loading, compilation and warm-up. A short automatic restart loop may repeatedly interrupt normal work. Inspect progress before deciding that the process is stuck.

After readiness, request the supported model list and send a short control request through the API documented for that version. Save the advertised identifier, status and response structure. If the request fails, separate model naming, message formatting and memory errors from network reachability.

## Check persistence and a new launch

The named volume preserves downloaded model files for reuse. It does not automatically save every result or every manually installed dependency. Identify which paths belong to the volume and which remain in the disposable container.

Measure a cold start separately from a launch with cached files. Then stop normally and repeat from a new session with the recorded configuration. Confirm the control answer and save the actual image identity. This establishes a useful pilot baseline before changing memory fractions, concurrency or engines.

## Turn the check into a practical next step

Save the image, model, command, readiness log and control answer. A container being created does not confirm a usable model-serving environment.

For the documented details, consult the [primary documentation](https://docs.sglang.io/docs/get-started/install); [configuration reference](https://docs.sglang.io/docs/get-started/quickstart). Continue with [sglang vs vllm serving](/journal/sglang-vs-vllm-serving/) and [sglang memory fraction](/journal/sglang-memory-fraction/).