# ServiceAccount token expired: audiences and projected token rotation

Diagnose serviceAccount token expired: audiences and projected token rotation. Evidence to collect, a worked operational example and recovery checks.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/kubernetes-projected-serviceaccount-token/
Language: en

A projected ServiceAccount token has an audience and expiration. The kubelet refreshes its file, but the client must use the rotated value rather than cache the first token forever. Investigate expiration and audience separately from ordinary LLM API keys.

## Identify workload identity

Record serviceAccountName, projected volume, token path and intended receiver. Do not include the bearer token itself. Establish whether the application needs Kubernetes API access at all; automatic credential provision should not become undefined extra privilege. External receivers require real trust and validation rules. A ServiceAccount existing in a namespace does not oblige arbitrary external services to accept its token.


[Official reference 1](https://kubernetes.io/docs/concepts/storage/projected-volumes/).

## Inspect audience and lifetime

Audience identifies the intended recipient, which must validate the match rather than accept every correctly signed token. ExpirationSeconds requests a lifetime subject to API-server limits. The kubelet refreshes projected tokens according to documented policy, while the application may retain an older copy. Preserve settings and safe timing evidence without complete payloads. Verify behavior for the installed version and administrative configuration rather than inferring it from one YAML example.


[Official reference 2](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/).

## Examine a stale cache

Suppose an application reads the token file only at startup. Later, the kubelet writes a new value while the client keeps sending the old cached bearer. Restart temporarily restores access, then the problem returns. Investigate reload behavior rather than creating a nonexpiring legacy Secret. Check subPath separately: projected volumes mounted that way do not receive normal updates, potentially leaving stale data before the client even reads it.

| Boundary | Verification |
| --- | --- |
| Issuance | Correct ServiceAccount and audience |
| Projection | Refreshed accessible file |
| Client | Rereading rather than permanent caching |
| Receiver | Audience and time validation |

## Correct token consumption

Use the client's supported mechanism for reading rotated files and refreshing authentication state. Inspect mounts and unintended subPath usage. For a client without reload, design an appropriate lifecycle while acknowledging its limitations. Do not broaden RBAC or remove expiration merely to suppress a 401; that changes another access boundary. For an audience error, align the intended receiver rather than reusing the token for an unrelated purpose.

## Test across rotation

Observe at least one real refresh cycle in a test environment and perform control access before and afterward. Preserve timing, projection settings and results without token values. Success includes continued client operation after file replacement and expected rejection of an unsuitable audience. The artifact distinguishes expired cache from incorrect trust configuration. Repeat after library upgrades: a new authentication implementation can cache differently even when the manifest and file path remain unchanged. Keep the test focused on identity lifecycle rather than granting additional permissions to make it pass.

Related operational reading: [hardware security](/journal/hardware-security/); [kubernetes gpu scheduling](/journal/kubernetes-gpu-scheduling/).