# Init container downloads an artifact again: making preparation idempotent

Diagnose init container downloads an artifact again: making preparation idempotent. Evidence to collect, a worked operational example and recovery checks.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/kubernetes-init-artifact-download/
Language: en

Artifact preparation in an init container must tolerate another attempt. A file with the expected name does not prove that downloading completed or that its contents are valid. Define revision, integrity and completion checks before allowing the main process to start.

## Define a complete artifact

Record an immutable reference, expected files and integrity criteria from a trusted source. Give the directory explicit ownership and permissions. Avoid a simple existence flag when downloading creates the file before completion. For multiple files, define a marker representing completion of the entire set. Bind it to a specific revision so that an older successful attempt cannot authorize startup with a different configuration.


[Official reference 1](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/).

## Account for init retries

Kubernetes can rerun an init container, so its code must be idempotent. Files in a shared emptyDir may already exist after an earlier attempt. Distinguish a container retry within one Pod from a new Pod with a new temporary volume. An init container is not automatically a persistent cache across recreation. Record the mount and responsibility for partial downloads. Successful init completion establishes preparation, not readiness of the application itself.


[Official reference 2](https://docs.docker.com/build/building/best-practices/).

## Design a bounded download flow

In an example, revision A downloads into A.partial. After file verification, the workflow publishes a completed directory or marker for A and permits the main container to start. On retry, the downloader checks revision and integrity rather than appending to an unknown residue. Atomic movement requires suitable filesystem behavior and correctly chosen paths. Remote object storage must not automatically be assigned the same semantics as a local rename operation.

| State | Intended action |
| --- | --- |
| Artifact absent | Prepare the required revision |
| Partial data exists | Validate or clean the interrupted attempt |
| Verified A exists | Permit explicitly intended reuse |
| Revision B required | Reject A's completion marker |

## Bound the network dependency

Define download timeout, retry policy and credential delivery without embedding secrets in artifacts. Test DNS, egress and TLS trust inside the init container. Kubelet image pulling and process-level downloading use different network scopes. Do not add Pod activeDeadlineSeconds without understanding that the deadline can matter after init succeeds. Follow Kubernetes guidance for the workload type rather than introducing unlimited waiting.

## Test interruption and recovery

Interrupt a test attempt halfway through downloading, retry it and verify that the main container cannot see an unverified set. Change the revision and confirm that the old marker is invalidated. The procedure artifact should record the reference, checksum policy, partial state and publication criterion. It enables recovery without interactive repairs. Reproducible preparation does not guarantee identical LLM output; it establishes the verified file set passed to the next startup stage. Preserve that limited claim when interpreting a successful retry.

Related operational reading: [software stack reproducibility](/journal/software-stack-reproducibility/); [autoscaling gpu cold start](/journal/autoscaling-gpu-cold-start/).