Pages

Tuesday, April 7, 2026

Handling Multi-Architecture Images in a Kubernetes Cluster Where Nodes Run Different CPU Architectures

09-multi-architecture-images-in-kubernetes

By Vicente Arteaga Gomez

MisLinux · Last updated: April 7, 2026

This article is part of my MisLinux series on building and operating a small Kubernetes cluster on Hetzner. It reflects my own hands-on experience and opinions, and I am not affiliated with, sponsored by, or endorsed by Hetzner, Docker, or any tool mentioned here.

When I started building my Kubernetes cluster on Hetzner, I chose the ARM64-based cax instances because they offered a better price-to-performance ratio than comparable AMD64 options. That seemed straightforward at the time. A few months later, it became complicated when I needed to rebuild container images from my AMD64 development laptop and push them to a private registry used by an all-ARM64 cluster.

This post is about how I handle multi-architecture container images in that environment, including a workflow for pushing a single-architecture update without destroying the other architecture in the registry.

Why architecture matters for container images

A container image is not architecture-neutral. When you build an image on an AMD64 machine with docker build, you produce an AMD64 image. If a Kubernetes node running ARM64 tries to pull that image, the pull fails or the container crashes at startup.

For a homogeneous cluster, this is not a concern. Every node is the same architecture, every image is built for that architecture, and nothing collides. But the moment you mix architectures, or build from a machine that does not match your cluster, you have a problem.

The solution is a multi-architecture manifest. Instead of one image pointing to one set of layers, you create an OCI image index (often called a "fat manifest" or multi-arch manifest) that points to two or more architecture-specific images under the same tag. When a node pulls that tag, the registry serves the right variant for the node's architecture automatically.

How Docker buildx creates multi-arch images

Docker buildx is the standard tool for building multi-architecture images. With a properly configured builder, you can target multiple platforms in a single build command:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --tag registry.example.com/myimage:latest \
  --push \
  .

This produces an image index at registry.example.com/myimage:latest that contains two descriptors: one for linux/amd64 and one for linux/arm64. Any node pulling that tag gets the right image for its architecture.

There are two ways buildx produces the non-native architecture:

  • QEMU emulation: slow, but works on any machine. The non-native architecture is emulated in software.
  • Native cross-compilation: fast, but requires a builder node of the target architecture.

For production-quality builds, QEMU emulation is often too slow to be practical for large images. The right long-term answer is a native builder for each target architecture.

The current setup: AMD64 laptop, ARM64 cluster

In my environment today:

  • The Kubernetes cluster runs exclusively on Hetzner cax ARM64 nodes.
  • I build images from an AMD64 development machine.
  • There is no ARM64 build agent inside the cluster yet.

Building linux/arm64 images from an AMD64 machine via QEMU is feasible for small images but impractically slow for images that compile Rust code or run a full test suite during the build. For those, I need a native ARM64 build.

Until a native ARM64 builder is set up, the pragmatic solution is to build each architecture separately and merge the results.

Building for a single architecture and preserving the other

The scenario I face regularly is this: I have made a change that needs to be deployed now. The linux/arm64 image is already in the registry and working correctly. I want to build a new linux/amd64 image from my laptop and push it without removing the existing linux/arm64 descriptor from the registry.

If I do a naive docker buildx build --platform linux/amd64 --push, the tag is replaced with a single-arch manifest pointing only to AMD64. The ARM64 image is gone from that tag. Any ARM64 node that pulls the tag will fail.

The fix is to use docker buildx imagetools create to compose a new multi-arch index from the individual descriptors. Here is the workflow:

Step 1: Build and push the AMD64 image under a staging tag.

docker buildx build \
  --platform linux/amd64 \
  --tag registry.example.com/myimage:amd64-staging \
  --push \
  .

Step 2: Inspect the current multi-arch manifest to confirm the ARM64 descriptor is still there.

docker buildx imagetools inspect registry.example.com/myimage:latest

This shows the image index with each architecture's digest. If the ARM64 descriptor is present, proceed.

Step 3: Merge the new AMD64 staging image with the existing ARM64 image into the target tag.

docker buildx imagetools create \
  --tag registry.example.com/myimage:latest \
  registry.example.com/myimage:amd64-staging \
  registry.example.com/myimage:latest@sha256:<arm64-digest>

The imagetools create command reads the listed sources and produces a new image index containing all their descriptors. Referencing the ARM64 image by its digest guarantees you get exactly the version you inspected, not whatever might be at the tag at the moment the command runs.

Step 4: Verify the result.

docker buildx imagetools inspect registry.example.com/myimage:latest

Confirm both linux/amd64 and linux/arm64 descriptors are present. Then delete the staging tag to keep the registry clean.

Why the registry itself can cause problems

One thing I learned the hard way is that not every registry handles multi-arch manifests cleanly. In my environment, I run a private registry backed by the standard Docker registry:2 image. Under aggressive garbage collection, OCI image indexes can be partially invalidated even when the blobs they reference still exist.

The symptom is a tag that appears in the registry but returns a manifest parsing error or a missing-layer error when you try to pull it. The root cause is usually that GC removed a blob referenced by the manifest, or that a tag was rewritten in a format the registry did not store cleanly.

A few things that help:

  • Run garbage collection during low-traffic periods.
  • Validate multi-arch tags immediately after a push by running docker manifest inspect from a separate machine.
  • Keep a backup of the digest for each architecture before doing any manifest manipulation. Digests are content-addressed and immutable; you can always reconstruct a tag from them.

The long-term goal: native ARM64 builds in CI

Building ARM64 images from QEMU on an AMD64 laptop is a workaround, not a permanent solution. The right answer is a native ARM64 build node.

The Hetzner cax instances I already use in the cluster are ARM64. A cax21 instance costs roughly a few euros per month and can function as a docker buildx builder node using the remote or kubernetes buildx driver. Once that is in place, a CI pipeline can build both architectures natively and push a proper multi-arch manifest in a single step, without any manual digest tracking.

Until then, the staging-and-merge workflow described above is the practical path for keeping both architectures available under the same tag.

Summary

  • Container images are architecture-specific; a multi-arch manifest lets one tag serve both ARM64 and AMD64 nodes.
  • docker buildx build --platform linux/amd64,linux/arm64 builds both at once, but requires either QEMU or a native builder for the non-native architecture.
  • When updating only one architecture, use docker buildx imagetools create to merge the new descriptor with the preserved descriptor from the other architecture.
  • Always reference the other architecture by digest, not by tag, during the merge.
  • Validate the result immediately after pushing; registry GC and format mismatches can leave a tag broken without obvious errors.
  • The permanent solution is a native builder for each target architecture.