Multi-CPU Architecture builds with Docker — Understanding Docker manifest lists

Docker brought a massive change and practical movement in the virtualization world. And it couldn’t just be supported towards one operating system or one CPU architecture.

So, throughout the lifecycle of docker development, the docker platform expanded to multiple architectures like in 2013, docker spanned across x86_64 architecture, in 2014 docker spanned to ARM architectures etc.

So, this brought the fantastic capability to build, ship and run containerized workloads on almost any underlying architecture.

So, what was the problem?

The problem was that for every individual architecture, the docker image was supposed to be built corresponding to that architecture and run only on that architecture.

For example,

Let’s say, I want to create and run a golang docker container on one host backed by amd64 (x86–64, linux hosts) and on another host backed by ARM architecture.

So, this involved the following tasks:

  • For the one backed by amd64 — I would have to find and download the docker image of golang specifically built and made for amd64. And then, run it.
docker run amd64/golang
  • For the one backed by ARM — I would have to manually find and download the docker image of golang specifically build and made for ARM. And then, run it.
docker run arm64v8/golang
Linux runs over amd64 architecture. Screenshot from my laptop

So, the problem was that there wasn’t any capability of creating one single unified parent docker image which could be downloaded on any operating system (or over any architecture) and just be run, and docker behind the scenes will automatically make sure that it runs the image made for the corresponding arch. or operating system.

So, basically, that’s what we wanted,

Same command/image name producing desired result over all the architectures.

docker run super-golang#### docker, behind the scenes, automatically pulls and runs the golang image built for arm64 or amd64 depending on the host arch.

So, what was the exact solution from docker’s perspective?

Docker expects the developer to build and push docker images corresponding to each architecture.

Extending the above example,

docker expects the image of arm64v8/golang and amd64/golang to be individually built and pushed to docker hub (or any other registry :P)

Now, what we expect from docker is that when we run “docker run super-golang” then, docker determines the arch. of the host, downloads the single rightful image for that architecture (arm64v8/golang for ARM , amd64/golang for AMD) and runs it.

So, how do we make docker do this? Enter “Docker manifest lists”

So, here we create a manifest list with the name “super-golang” and it contains a list of platform/architecture segregated list of entries consisting to all the different images for different architectures.

But what is a docker manifest, in the first place?

So, every docker image is represented by a manifest. A manifest is a JSON file containing all the information about that docker image like information about it’s sub-layers, the size allocation, the platform on which individual layer is capable of being run, etc.

For dealing with manifests, enable experimental mode in docker on your OS by adding “experimental”: “enabled” in your ~/.docker/config.json file

Here is a pic of the docker manifest of nginx image on my laptop.

Every json object in the manifest denotes details about the subsequent layer under nginx image.Look at the “architecture” key under every json object.

Back to the manifest lists

Hence, through the power of “manifest” docker can know, without the developer telling it, that which image corresponds to be run on which architecture, by inspecting the “architecture” in that image’s manifest.

So, when we create that manifest list and push it to docker hub, it pushes all those “child” images to docker hub, along with the manifest named “super-golang”.

And when finally, a user or client having a Linux computer (having underlying architecture of amd64) runs the command “docker run super-golang”, docker performs the following steps behind the scenes:

  • It searches dockerhub for “super-golang” and finds that it’s a manifest list object.
  • It downloads the manifest list corresponding to “super-golang”.
  • It then looks at all the entries of that manifest list and determines which is the rightful entry (child image) to run on the user’s computer on the basis of their computer’s underlying architecture (that is, the one corresponding to amd64 for that user)
  • And finally, it pulls rightfully determined child image made for amd64 architecture, on the user’s computer and runs it.

Time for Hands-on!

Let’s take the same example of two golang docker images (amd64v8 and arm64) to be pushed under a single manifest-file named “super-golang”

  • So, say my laptop has two dockerfiles for building each of those docker images.
ls> Dockerfile.amd64
Dockerfile.arm64v8

Step 1). Build those dockerfiles to produce the respective docker images.

### Creates the "arm64v8/golang" image by building Dockerfile.arm64v8
docker build -t arm64v8/golang -f Dockerfile.arm64v8 .### Creates the "amd64/golang" image by building Dockerfile.amd64
docker build -t amd64/golang -f Dockerfile.amd64 .

Step 2). Now, write the manifest list with the name “super-golang” consisting of details about the child images with their platform details

Step 3). Install manifest-tool from this .

Step 4). Run the following command to push the manifest file. (I created my manifest.yml file here -> /home/yash1300/docker_manifest/manifest.yml)

manifest-tool push --from-spec /home/yash1300/docker_manifest/manifest.yml

OR you can do the same thing without dealing with this “manifest-tool”. But with that you would have to do the entire manifest-file thing with commands.

Here are the steps,

Step 2). Push the individual images to docker hub

docker push arm64v8/golang
docker push amd64/golang

Step 3). Create the manifest lists.

docker manifest create super-golang:latest \
-amend arm64v8/golang \
-amend amd64/golang

Step 4). Push the entire manifest file to docker hub.

docker manifest push super-golang:latest

And that’s it! You are ready with your multi-architecture supported and ready-to-use docker image.

Just run

docker run super-golang

and Enjoy!

PS: I found this image depicting a sample flow of multi-arch. build with docker providing a capability to create and run an image on windows or linux depending on client’s OS.

End note

Thanks a lot of reaching till here :D

I hope you understood this article and found it informative. Don’t worry if something went over your head through this article, this topic is pretty complicated but again, it is gorgeous in it’s own essence, so read it again, you’ll surely would understand it further :)

If you liked it, do give it some claps :)

Feel free to connect with me on my handles

LinkedIn

GitHub

Until next time,

Adios!

Site Reliability Engineer @ Red Hat | ex-Grofers | Contributing to {Cloud, Kubernetes}-native OSS