Linux  ·  high  ·  Containers & Kubernetes

ImagePullBackOff, ErrImagePull and registry rate limits

The image cannot be fetched. The message after the status names which of four quite different reasons it is.

What you see

Pods stay pending with ImagePullBackOff, or docker pull fails. Describing the pod gives the real message.

What is actually wrong

A rate limit on anonymous pulls, credentials missing for a private registry, a tag that does not exist, or a TLS interception proxy the node does not trust.

Codes and articles

ImagePullBackOffErrImagePulltoomanyrequestsmanifest unknownunauthorized: authentication requiredx509 certificate signed by unknown authority

Fixes (3)

Stop pulling anonymously
Root shell30 minuteslow riskreversible

A rate limit message.

  1. Confirm the message.

    Shell
    kubectl describe pod mypod | tail -20
  2. Anonymous pulls from Docker Hub are limited per IP address, so an entire office or cluster behind one NAT address shares one small quota.

    This is why the problem appears suddenly on a cluster that has worked for months — the limit is per source address, not per machine, and one busy pipeline consumes it for everyone.

  3. Authenticate, which raises the limit substantially even on a free account.

    Shell
    kubectl create secret docker-registry dockerhub \  --docker-server=https://index.docker.io/v1/ \  --docker-username=USER --docker-password=TOKEN
  4. Attach it to the service account so every pod uses it without per-pod configuration.

    Shell
    kubectl patch serviceaccount default -p '{"imagePullSecrets":[{"name":"dockerhub"}]}'
  5. Better, run a pull-through cache or mirror so images are fetched once for the whole cluster.

    Shell
    kubectl get nodes -o wide
  6. Set imagePullPolicy to IfNotPresent for images with fixed tags, so a restart does not pull again.

    Shell
    kubectl get deploy myapp -o jsonpath='{.spec.template.spec.containers[*].imagePullPolicy}'
Confirm it workedPods start and the pull events show no rate limiting.
Shell
kubectl get events --sort-by=.lastTimestamp | tail -20
If you need to undo itRemove the secret and the service account patch.
Supply credentials the node can actually use
Root shell30 minutesmedium riskreversible

unauthorized, or authentication required.

  1. Check the secret exists in the same namespace as the pod. An imagePullSecret in a different namespace is invisible.

    Shell
    kubectl get secrets -n mynskubectl get pod mypod -n myns -o jsonpath='{.spec.imagePullSecrets}'

    Secrets are namespaced and image pull secrets are not shared. This is the most common cause of credentials that are definitely correct and definitely not working.

  2. Check the secret's contents decode to the right registry host.

    Shell
    kubectl get secret regcred -n myns -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d
  3. The registry server must match exactly what the image reference uses, including whether it has a port.

  4. Test the credentials directly from a node.

    Shell
    sudo crictl pull registry.example.com/myapp:1.2.3docker login registry.example.com
  5. Rotate the token if it has expired — most registry tokens are short-lived by design.

Confirm it workedThe image pulls and the pod reaches Running.
Shell
kubectl get pod mypod -n myns -w
If you need to undo itDelete and recreate the secret with the previous credentials.
Resolve a missing tag or a TLS trust problem
Root shell30 minutesmedium riskreversible

manifest unknown, or an x509 certificate error.

  1. For manifest unknown, check the tag exists and matches the node's architecture.

    Shell
    docker manifest inspect myimage:tag | head -30
  2. A tag that exists for amd64 but not arm64 fails exactly like a missing tag on a mixed cluster.

    Shell
    kubectl get nodes -o custom-columns=NAME:.metadata.name,ARCH:.status.nodeInfo.architecture

    Mixed-architecture clusters — a few Raspberry Pi or Graviton nodes alongside x86 — produce this intermittently, on whichever nodes the scheduler happened to choose.

  3. For a certificate error, the node does not trust the registry's issuer, usually because of a TLS inspection proxy.

    Shell
    openssl s_client -connect registry.example.com:443 -servername registry.example.com </dev/null 2>/dev/null | openssl x509 -noout -issuer -subject
  4. Install the issuing certificate on every node, in the system trust store, and restart the container runtime.

    Shell
    sudo cp corp-root.crt /usr/local/share/ca-certificates/sudo update-ca-certificatessudo systemctl restart containerd
  5. Do not add the registry to insecure-registries as a workaround — it removes certificate verification for that host entirely.

    Worth stating plainly because it is the first suggestion in most search results. It turns a trust problem into a permanently unverified connection carrying the images your cluster runs.

Confirm it workedThe pull succeeds over TLS with verification intact.
Shell
sudo crictl pull registry.example.com/myapp:1.2.3
If you need to undo itRemove the certificate from the trust store and re-run update-ca-certificates.

Where this stops. This write-up was written and checked by hand. It says what each step changes, how to confirm it worked and how to reverse it, and anything destructive is flagged before you reach it. If it does not match what your machine is doing, search the Support Centre for the exact code or message — and when something needs a person, get in touch.