TECHNOLOGY

Resource stuck Terminating: diagnosing a Kubernetes finalizer

In this article
THE QUICK TAKE

A resource stuck Terminating with a finalizer usually awaits required cleanup. Identify its controller and dependencies first. Removing the key manually can delete the API object while leaving real resources and broken dependencies, so it is not a universal repair.

A resource stuck Terminating with a finalizer usually awaits required cleanup. Identify its controller and dependencies first. Removing the key manually can delete the API object while leaving real resources and broken dependencies, so it is not a universal repair.

Preserve deletion metadata

Capture deletionTimestamp, finalizers, ownerReferences, UID and events. Compare owner and dependent states. A finalizer is a key watched by a controller, not an executable command stored in the object. Determine which component must perform cleanup and remove it. Terminating can persist because a controller is unavailable, a dependency failed or a resource remains legitimately in use. Each cause requires different action and evidence of completion.

Official reference 1.

Follow ownership and dependencies

Foreground and background deletion use different ordering for related objects. Owner references explain cascading relationships but do not replace the purpose of a particular finalizer. Volume protection may intentionally retain an object while a Pod uses it. Removing that key bypasses protection rather than restoring the controller. Check namespaces and identities too: a new object with an old name may not own an earlier dependency because its UID differs.

Official reference 2.

Examine controller failure

In an illustrative case, a custom resource waits on example.org/cleanup. Its controller is temporarily down and an external task still exists. Restoring the controller allows verified cleanup and removal of the key. This demonstrates a mechanism rather than describing an installed operator. If restoration is impossible, plan how to fulfill the controller’s responsibilities manually, checking external state before changing deletion metadata.

CheckEvidence retained
Who assigned the key?Documentation and controller ownership
What requires cleanup?Actual dependency inventory
Why is cleanup blocked?Events and controller logs
When is cleanup complete?Verifiable final state

Correct the waiting condition

Restore controller access to the API and dependency, or resolve the object legitimately blocking cleanup. Avoid bulk removal of finalizers across a namespace. If completed manual cleanup justifies removing one key, record the rationale and exact target. Kubernetes restricts finalizer changes after deletion starts; adding a new key or cancelling deletionTimestamp through ordinary editing is not available. Account for recreating a deleted object rather than assuming it can be revived from Terminating.

Check for residual resources

After the API object disappears, inspect related records, external tasks and remaining dependencies. Completed API deletion alone does not prove cleanup in every external system. The incident artifact should connect the UID, finalizer, cleanup owner and confirmed result. Preserve the observed cause for later incidents rather than retaining only a command that removes a key. This restores a controlled lifecycle instead of accumulating hidden resources behind apparently successful deletion.

Related operational reading: cloud gpu resource cleanup audit; gpu exit plan.

← All articles