# LiteLLM guardrails: choosing where to inspect requests and responses

Place LiteLLM guardrails at the right request or response boundary, then test false positives, dependency failures and streaming behavior.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/litellm-guardrails-boundary/
Language: en

LiteLLM guardrails inspect requests or responses around a model call. Decide where a check must run: before transmission, after generation, during streaming or only for observation. These boundaries solve different problems. A response filter cannot undo sensitive input already sent to an external endpoint.

## Select the relevant boundary

| Placement | What to establish |
| --- | --- |
| Before a model call | Whether the check affects transmitted input |
| After a response | How an already generated result is inspected |
| During streaming | How incomplete content is treated |
| Logging only | Observation without equivalent blocking |

Verify the modes supported by the chosen integration. A setting in a file is not proof it runs at the intended point. Keep a controlled trace showing safe decision metadata. Do not retain the sensitive text the rule is intended to protect simply to demonstrate that the rule executed.

## Define a narrow policy

Begin with a synthetic identifier that must not leave the application. Specify whether the action is blocking, masking or recording. LiteLLM's content-filter documentation describes checks and actions; a word list or regular expression does not establish detection of every sensitive format.

Separate policy ownership from implementation ownership. The team should know what the rule intends to catch and what remains unresolved. Keep a versioned sample and expected decision so changes can be checked objectively. A successful demonstration string is one observation, not comprehensive protection.

## Test ordinary and awkward inputs

Use neutral data containing a known match, allowed request, near match and differently formatted value. Inspect the actual upstream input where the authorized test permits it. For output checks, distinguish generated content from the final text shown to the user.

Streaming needs a decision about content already displayed before a later match. A complete-response test does not validate this behavior. Test interruption and partial output with controlled fixtures rather than repeatedly submitting private production documents. Record false positives as well as successful detections.

## Decide failure behavior

Document what happens if the guardrail service times out or becomes unavailable. A blocking policy should not silently become observational because a dependency failed. Conversely, an observational pilot should not be described as enforcing a boundary it never tested.

Measure added latency and operational dependencies alongside model quality. These belong to the application pipeline rather than GPU-memory requirements. The proposed setup should describe an observed boundary, without claiming that one guardrail configuration guarantees confidentiality in every environment.

Official references: [docs.litellm.ai — quick start](https://docs.litellm.ai/docs/proxy/guardrails/quick_start), [docs.litellm.ai — litellm content filter](https://docs.litellm.ai/docs/proxy/guardrails/litellm_content_filter).

Related guides: [DLP at an AI gateway: reducing sensitive data exposure](/journal/ai-gateway-dlp/), [Errors inside an OpenRouter stream: why HTTP 200 is insufficient](/journal/openrouter-stream-errors/).