# Migrate legacy Triton TensorRT-LLM engine builds to the LLM API

Start a Triton TensorRT-LLM migration by identifying the deployment path rather than automatically repeating trtllm-build.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/tensorrt-llm-triton-llmapi-migration/
Language: en

Start a Triton TensorRT-LLM migration by identifying the deployment path rather than automatically repeating `trtllm-build`. The current guide uses the LLM API/PyTorch path with `model.yaml` and a direct model source. Older engine directories belong to a legacy workflow. Migrate input, sampling, and readiness contracts separately from loading. A shared backend name does not establish that old files and arguments remain valid.

## Record the original architecture

Preserve repository layout, versions, and commands. The [current guide](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/getting_started/llm.html) distinguishes the LLM API from the earlier checkpoint-conversion and engine-building sequence. Retain legacy artifacts until the replacement is validated, without presenting those steps as mandatory for every current deployment. Identify weight loading, tokenizer and generation settings, and the client's endpoint. Otherwise two distinct mechanisms can become one invalid hybrid instruction. The migration record should describe what actually runs rather than infer architecture from a directory name.

## Translate configuration by meaning

The [backend README](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/tensorrtllm_backend/README.html) maps `model.yaml` keys to LLM API arguments. Match responsibilities, not merely similar names. A legacy scheduler setting or engine limit may not map directly to the same file. Preserve a supporting source and effective-behavior check for every translated value. Do not promise support for every Hugging Face model: consult the selected release's support list and the particular checkpoint architecture, even when a short guide demonstrates direct loading from a model identifier.

## Build a migration table

Create rows for weight source, tokenizer/template, sampling, parallelism, readiness, and streaming. Record old method, new method, and acceptance fixture. This is a review template, not a universal ready-made configuration. Mark unknown mappings for investigation rather than carrying across an arbitrary default. List serialized TensorRT plans separately: their portability constraints should not be assigned to the PyTorch LLM API simply because both workflows share the TensorRT-LLM project name. Preserve the boundary in titles and operational instructions too.

## Validate the client contract

Compare ordinary and streaming requests, prompt formatting, and termination. A raw completion endpoint may not apply a chat template automatically, so moving from a chat API needs an explicit check. Preserve tokenized input or a rendered-prompt fixture where supported. Verify output limits and stopping rules, not merely a plausible answer. An endpoint-schema change requires an intentional application transformation rather than silent loss of parameters previously affecting generation or response fields. Keep differences visible in the migration table.

## Accept the replacement and retain rollback

Run the new configuration separately, validate functional fixtures, and exercise representative load without invented acceleration factors. Keep versions, source revisions, model.yaml, and readiness logs. Preserve reproducibility of the previous path until the replacement decision, without implying indefinite support. State what migrated and which features remain unverified. The result should be a coherent LLM API deployment, not old engine-building instructions mixed with new settings. This separates implementation changes from client behavior without falsely claiming that every modern TensorRT-LLM deployment requires precompiling a portable engine for every model.

## Related reading

[Reproducible inference stacks: preserving a GPU pilot’s result](/journal/software-stack-reproducibility/); [Updating an LLM on a GPU: testing a new version with a small rollout](/journal/model-update-canary/).