Container Images

The jobs described in this section all work together to handle the full gating process for continuously deployed container images. They can be used to build or test images which rely on other images using the full power of Zuul’s speculative execution.

There are a few key concepts to keep in mind:

A buildset is a group of jobs all running on the same change.

A buildset registry is a container image registry which is used to store speculatively built images for the use of jobs in a single buildset. It holds the differences between the current state of the world and the future state if the change in question (and all of its dependent changes) were to merge. It must be started by one of the jobs in a buildset, and it ceases to exist once that job is complete.

An intermediate registry is a long-running registry that is used to store images created for unmerged changes for use by other unmerged changes. It is not publicly accessible and is intended only to be used by Zuul in order to transfer artifacts from one buildset to another. OpenDev maintains such a registry.

With these concepts in mind, the jobs described below implement the following workflow for a single change:

Buildset registry image transfer

The intermediate registry is always running and the buildset registry is started by a job running on a change. The “Image Build” and “Deployment Test” jobs are example jobs which might be running on a change. Essentially, these are image producer or consumer jobs respectively.

There are two ways to use the jobs described below:

A Repository with Producers and Consumers

The first is in a repository where images are both produced and consumed. In this case, we can expect that there will be at least one image build job, and at least one job which uses that image (for example, by performing a test deployment of the image). In this case we need to construct a job graph with dependencies as follows:

The opendev-buildset-registry job will run first and automatically start a buildset registry populated with images built from any changes which appear ahead of the current change. It will then return its connection information to Zuul and pause and continue running until the completion of the build and test jobs.

The build-image job should inherit from opendev-build-docker-image, which will ensure that it is automatically configured to use the buildset registry.

The test-image job is something that you will create yourself. There is no standard way to test or deploy an image, that depends on your application. However, there is one thing you will need to do in your job to take advantage of the buildset registry. In a pre-run playbook, use the use-buildset-registry role:

- hosts: all
  roles:
    - use-buildset-registry

That will configure the docker daemon on the host to use the buildset registry so that it will use the newly built version of any required images.

A Repository with Only Producers

The second way to use these jobs is in a repository where an image is merely built, but not deployed. In this case, there are no consumers of the buildset registry other than the image build job, and so the registry can be run on the job itself. In this case, you may omit the opendev-buildset-registry job and run only the opendev-build-docker-image job.

Publishing an Image

So far we’ve covered the image building process. This system also provides two more jobs that are used in publishing images to Docker Hub.

The opendev-upload-docker-image job does everything the opendev-build-docker-image job does, but it also uploads the built image to Docker Hub using an automatically-generated and temporary tag. The “build” job is designed to be used in the check pipeline, while the “upload” job is designed to take its place in the gate pipeline. By front-loading the upload to Docker Hub, we reduce the chance that a credential or network error will prevent us from publishing an image after a change lands.

The opendev-promote-docker-image jobs is designed to be used in the promote pipeline and simply re-tags the image on Docker Hub after the change lands.

Keeping in mind that everything described above in Buildset registry image transfer applies to the opendev-upload-docker-image job, the following illustrates the additional tasks performed by the “upload” and “promote” jobs:

Jobs

opendev-buildset-registry

Starts a buildset registry which interacts with the intermediate CI registry to share speculative container images between projects.

Configure any jobs which require the use of a buildset registry to depend on this job using the “dependencies” job attribute.

This job will pause after starting the registry so that it is available to any jobs which depend on it. Once all such jobs are complete, this job will finish.

opendev-buildset-registry-consumer

Pull from the intermediate registry

This is a parent for jobs which use container images and expect a buildset registry to be running. It pulls images from the intermediate registry into it.

opendev-build-docker-image-base

This is a parent for an image build job which expects a buildset registry to be running and pulls images from the intermediate registry into it. It mostly exists so that the intermediate registry secret need not be supplied to the image build playbook.

opendev-build-docker-image

Starts a buildset registry (if one has not already been started, e.g., by invoking opendev-buildset-registry and specifying it as a dependency) and builds one or more docker images.

Analog of build-docker-image job, but with a buildset registry.

This job will pause after starting the registry so that it is available to any jobs which depend on it. Once all such jobs are complete, this job will finish.

This is one of a collection of jobs which are designed to work together to build, upload, and promote docker images in a gating context:

• opendev-build-docker-image: Build the images.

• opendev-upload-docker-image: Build and stage the images on dockerhub.

• opendev-promote-docker-image: Promote previously uploaded images.

The opendev-build-docker-image job is designed to be used in a check pipeline and simply builds the images to verify that the build functions.

The opendev-upload-docker-image job builds and uploads the images to Docker Hub, but only with a single tag corresponding to the change ID. This job is designed in a gate pipeline so that the build produced by the gate is staged and can later be promoted to production if the change is successful.

The opendev-promote-docker-image job is designed to be used in a promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by opendev-build-docker-image.docker_images.tags. It also removes the change ID tag from the repository in Docker Hub, and removes any similar change ID tags more than 24 hours old. This keeps the repository tidy in the case that gated changes fail to merge after uploading their staged images.

They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Job Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The project directory. Serves as the base for opendev-build-docker-image.docker_images.context.

docker_images
Type: list

A list of images to build. Each item in the list should have:

docker_images[].context

The docker build context; this should be a directory underneath opendev-build-docker-image.zuul_work_dir.

docker_images[].repository

The name of the target repository in dockerhub for the image. Supply this even if the image is not going to be uploaded (it will be tagged with this in the local registry).

docker_images[].path

Optional: the directory that should be passed to docker build. Useful for building images with a Dockerfile in the context directory but a source repository elsewhere.

docker_images[].build_args
Type: list

Optional: a list of values to pass to the docker --build-arg parameter.

docker_images[].target

Optional: the target for a multi-stage build.

docker_images[].tags
Default: ['latest']
Type: list

A list of tags to be added to the image when promoted.

opendev-upload-docker-image

Starts a buildset registry and builds and uploads one or more docker images to docker.io.

Analog of upload-docker-image job, but with a buildset registry.

This is one of a collection of jobs which are designed to work together to build, upload, and promote docker images in a gating context:

• opendev-build-docker-image: Build the images.

• opendev-upload-docker-image: Build and stage the images on dockerhub.

• opendev-promote-docker-image: Promote previously uploaded images.

The opendev-build-docker-image job is designed to be used in a check pipeline and simply builds the images to verify that the build functions.

The opendev-upload-docker-image job builds and uploads the images to Docker Hub, but only with a single tag corresponding to the change ID. This job is designed in a gate pipeline so that the build produced by the gate is staged and can later be promoted to production if the change is successful.

The opendev-promote-docker-image job is designed to be used in a promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by opendev-build-docker-image.docker_images.tags. It also removes the change ID tag from the repository in Docker Hub, and removes any similar change ID tags more than 24 hours old. This keeps the repository tidy in the case that gated changes fail to merge after uploading their staged images.

They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Job Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The project directory. Serves as the base for opendev-build-docker-image.docker_images.context.

docker_images
Type: list

A list of images to build. Each item in the list should have:

docker_images[].context

The docker build context; this should be a directory underneath opendev-build-docker-image.zuul_work_dir.

docker_images[].repository

The name of the target repository in dockerhub for the image. Supply this even if the image is not going to be uploaded (it will be tagged with this in the local registry).

docker_images[].path

Optional: the directory that should be passed to docker build. Useful for building images with a Dockerfile in the context directory but a source repository elsewhere.

docker_images[].build_args
Type: list

Optional: a list of values to pass to the docker --build-arg parameter.

docker_images[].target

Optional: the target for a multi-stage build.

docker_images[].tags
Default: ['latest']
Type: list

A list of tags to be added to the image when promoted.

docker_credentials
Type: dict

This is expected to be a Zuul Secret with these keys:

docker_credentials{}.username

The Docker Hub username.

docker_credentials{}.password

The Docker Hub password.

docker_credentials{}.repository

Optional; if supplied this is a regular expression which restricts to what repositories the image may be uploaded. The following example allows projects to upload images to repositories within an organization based on their own names:

repository: "^myorgname/{{ zuul.project.short_name }}.*"

opendev-promote-docker-image

Retag a previously-uploaded docker image.

Analog of promote-docker-image job.

This is one of a collection of jobs which are designed to work together to build, upload, and promote docker images in a gating context:

• opendev-build-docker-image: Build the images.

• opendev-upload-docker-image: Build and stage the images on dockerhub.

• opendev-promote-docker-image: Promote previously uploaded images.

The opendev-build-docker-image job is designed to be used in a check pipeline and simply builds the images to verify that the build functions.

The opendev-upload-docker-image job builds and uploads the images to Docker Hub, but only with a single tag corresponding to the change ID. This job is designed in a gate pipeline so that the build produced by the gate is staged and can later be promoted to production if the change is successful.

The opendev-promote-docker-image job is designed to be used in a promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by opendev-build-docker-image.docker_images.tags. It also removes the change ID tag from the repository in Docker Hub, and removes any similar change ID tags more than 24 hours old. This keeps the repository tidy in the case that gated changes fail to merge after uploading their staged images.

They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Job Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The project directory. Serves as the base for opendev-build-docker-image.docker_images.context.

docker_images
Type: list

A list of images to build. Each item in the list should have:

docker_images[].context

The docker build context; this should be a directory underneath opendev-build-docker-image.zuul_work_dir.

docker_images[].repository

The name of the target repository in dockerhub for the image. Supply this even if the image is not going to be uploaded (it will be tagged with this in the local registry).

docker_images[].path

Optional: the directory that should be passed to docker build. Useful for building images with a Dockerfile in the context directory but a source repository elsewhere.

docker_images[].build_args
Type: list

Optional: a list of values to pass to the docker --build-arg parameter.

docker_images[].target

Optional: the target for a multi-stage build.

docker_images[].tags
Default: ['latest']
Type: list

A list of tags to be added to the image when promoted.

docker_credentials
Type: dict

This is expected to be a Zuul Secret with these keys:

docker_credentials{}.username

The Docker Hub username.

docker_credentials{}.password

The Docker Hub password.

docker_credentials{}.repository

Optional; if supplied this is a regular expression which restricts to what repositories the image may be uploaded. The following example allows projects to upload images to repositories within an organization based on their own names:

repository: "^myorgname/{{ zuul.project.short_name }}.*"

opendev-build-container-image-base

This is a parent for an image build job which expects a buildset registry to be running and pulls images from the intermediate registry into it. It mostly exists so that the intermediate registry secret need not be supplied to the image build playbook.

opendev-build-container-image

Starts a buildset registry (if one has not already been started, e.g., by invoking opendev-buildset-registry and specifying it as a dependency) and builds one or more docker images.

Analog of build-docker-image job, but with a buildset registry.

This job will pause after starting the registry so that it is available to any jobs which depend on it. Once all such jobs are complete, this job will finish.

This is one of a collection of jobs which are designed to work together to build, upload, and promote container images in a gating context:

• opendev-build-container-image: Build the images.

• opendev-upload-container-image: Build and stage the images in a registry.

• opendev-promote-container-image: Promote previously uploaded images.

The opendev-build-container-image job is designed to be used in a check pipeline and simply builds the images to verify that the build functions.

The opendev-upload-container-image job builds and uploads the images to a registry, but only with a single tag corresponding to the change ID. This job is designed in a gate pipeline so that the build produced by the gate is staged and can later be promoted to production if the change is successful.

The opendev-promote-container-image job is designed to be used in a promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by opendev-build-container-image.container_images.tags. It also removes the change ID tag from the repository in the registry. If any changes fail to merge, this cleanup will not run and those tags will need to be deleted manually.

They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Job Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The project directory. Serves as the base for opendev-build-container-image.container_images.context.

container_filename

The default container filename name to use. Serves as the base for opendev-build-container-image.container_images.container_filename. This allows a global overriding of the container filename name, for example when building all images from different folders with similarily named containerfiles.

If omitted, the default depends on the container command used. Typically, this is Dockerfile for docker and Containerfile (with a fallback on Dockerfile) for podman.

container_command
Default: podman

The command to use when building the image (E.g., docker).

container_images
Type: list

A list of images to build. Each item in the list should have:

container_images[].context

The build context; this should be a directory underneath opendev-build-container-image.zuul_work_dir.

container_images[].container_filename

The filename of the container file, present in the context folder, used for building the image. Provide this if you are using a non-standard filename for a specific image.

container_images[].registry

The name of the target registry (E.g., quay.io). Used by the upload and promote roles.

container_images[].repository

The name of the target repository in the registry for the image. Supply this even if the image is not going to be uploaded (it will be tagged with this in the local registry). This should include the registry name. E.g., quay.io/example/image.

container_images[].path

Optional: the directory that should be passed to the build command. Useful for building images with a container file in the context directory but a source repository elsewhere.

container_images[].build_args
Type: list

Optional: a list of values to pass to the --build-arg parameter.

container_images[].target

Optional: the target for a multi-stage build.

container_images[].tags
Default: ['latest']
Type: list

A list of tags to be added to the image when promoted.

container_images[].siblings
Default: []
Type: list

A list of sibling projects to be copied into {{zuul_work_dir}}/.zuul-siblings. This can be useful to collect multiple projects to be installed within the same Docker context. A -build-arg called ZUUL_SIBLINGS will be added with each sibling project. Note that projects here must be listed in required-projects.

container_build_extra_env
Type: dict

A dictionary of key value pairs to add to the container build environment. This may be useful to enable buildkit with docker builds for example.

opendev-upload-container-image

Starts a buildset registry and builds and uploads one or more container images to a registry.

Analog of upload-container-image job, but with a buildset registry.

This is one of a collection of jobs which are designed to work together to build, upload, and promote container images in a gating context:

• opendev-build-container-image: Build the images.

• opendev-upload-container-image: Build and stage the images in a registry.

• opendev-promote-container-image: Promote previously uploaded images.

The opendev-build-container-image job is designed to be used in a check pipeline and simply builds the images to verify that the build functions.

The opendev-upload-container-image job builds and uploads the images to a registry, but only with a single tag corresponding to the change ID. This job is designed in a gate pipeline so that the build produced by the gate is staged and can later be promoted to production if the change is successful.

The opendev-promote-container-image job is designed to be used in a promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by opendev-build-container-image.container_images.tags. It also removes the change ID tag from the repository in the registry. If any changes fail to merge, this cleanup will not run and those tags will need to be deleted manually.

They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Job Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The project directory. Serves as the base for opendev-build-container-image.container_images.context.

container_filename

The default container filename name to use. Serves as the base for opendev-build-container-image.container_images.container_filename. This allows a global overriding of the container filename name, for example when building all images from different folders with similarily named containerfiles.

If omitted, the default depends on the container command used. Typically, this is Dockerfile for docker and Containerfile (with a fallback on Dockerfile) for podman.

container_command
Default: podman

The command to use when building the image (E.g., docker).

container_images
Type: list

A list of images to build. Each item in the list should have:

container_images[].context

The build context; this should be a directory underneath opendev-build-container-image.zuul_work_dir.

container_images[].container_filename

The filename of the container file, present in the context folder, used for building the image. Provide this if you are using a non-standard filename for a specific image.

container_images[].registry

The name of the target registry (E.g., quay.io). Used by the upload and promote roles.

container_images[].repository

The name of the target repository in the registry for the image. Supply this even if the image is not going to be uploaded (it will be tagged with this in the local registry). This should include the registry name. E.g., quay.io/example/image.

container_images[].path

Optional: the directory that should be passed to the build command. Useful for building images with a container file in the context directory but a source repository elsewhere.

container_images[].build_args
Type: list

Optional: a list of values to pass to the --build-arg parameter.

container_images[].target

Optional: the target for a multi-stage build.

container_images[].tags
Default: ['latest']
Type: list

A list of tags to be added to the image when promoted.

container_images[].siblings
Default: []
Type: list

A list of sibling projects to be copied into {{zuul_work_dir}}/.zuul-siblings. This can be useful to collect multiple projects to be installed within the same Docker context. A -build-arg called ZUUL_SIBLINGS will be added with each sibling project. Note that projects here must be listed in required-projects.

container_build_extra_env
Type: dict

A dictionary of key value pairs to add to the container build environment. This may be useful to enable buildkit with docker builds for example.

container_registry_credentials
Type: dict

This is only required for the upload and promote roles. This is expected to be a Zuul Secret in dictionary form. Each key is the name of a registry, and its value a dictionary with information about that registry.

Example:

container_registry_credentials:
  quay.io:
    username: foo
    password: bar

container_registry_credentials{}.[registry name]
Type: dict

Information about a registry. The key is the registry name, and its value a dict as follows:

container_registry_credentials{}.[registry name]{}.username

The registry username.

container_registry_credentials{}.[registry name]{}.password

The registry password.

container_registry_credentials{}.[registry name]{}.repository

Optional; if supplied this is a regular expression which restricts to what repositories the image may be uploaded. The registry name should be included. The following example allows projects to upload images to repositories within an organization based on their own names:

repository: "^quay.io/myorgname/{{ zuul.project.short_name }}.*"

opendev-promote-container-image

Retag a previously-uploaded container image.

Analog of promote-container-image job.

This is one of a collection of jobs which are designed to work together to build, upload, and promote container images in a gating context:

• opendev-build-container-image: Build the images.

• opendev-upload-container-image: Build and stage the images in a registry.

• opendev-promote-container-image: Promote previously uploaded images.

The opendev-build-container-image job is designed to be used in a check pipeline and simply builds the images to verify that the build functions.

The opendev-upload-container-image job builds and uploads the images to a registry, but only with a single tag corresponding to the change ID. This job is designed in a gate pipeline so that the build produced by the gate is staged and can later be promoted to production if the change is successful.

The opendev-promote-container-image job is designed to be used in a promote pipeline. It requires no nodes and runs very quickly on the Zuul executor. It simply re-tags a previously uploaded image for a change with whatever tags are supplied by opendev-build-container-image.container_images.tags. It also removes the change ID tag from the repository in the registry. If any changes fail to merge, this cleanup will not run and those tags will need to be deleted manually.

They all accept the same input data, principally a list of dictionaries representing the images to build. YAML anchors can be used to supply the same data to all three jobs.

Job Variables

zuul_work_dir
Default: {{ zuul.project.src_dir }}

The project directory. Serves as the base for opendev-build-container-image.container_images.context.

container_filename

The default container filename name to use. Serves as the base for opendev-build-container-image.container_images.container_filename. This allows a global overriding of the container filename name, for example when building all images from different folders with similarily named containerfiles.

If omitted, the default depends on the container command used. Typically, this is Dockerfile for docker and Containerfile (with a fallback on Dockerfile) for podman.

container_command
Default: podman

The command to use when building the image (E.g., docker).

container_images
Type: list

A list of images to build. Each item in the list should have:

container_images[].context

The build context; this should be a directory underneath opendev-build-container-image.zuul_work_dir.

container_images[].container_filename

The filename of the container file, present in the context folder, used for building the image. Provide this if you are using a non-standard filename for a specific image.

container_images[].registry

The name of the target registry (E.g., quay.io). Used by the upload and promote roles.

container_images[].repository

The name of the target repository in the registry for the image. Supply this even if the image is not going to be uploaded (it will be tagged with this in the local registry). This should include the registry name. E.g., quay.io/example/image.

container_images[].path

Optional: the directory that should be passed to the build command. Useful for building images with a container file in the context directory but a source repository elsewhere.

container_images[].build_args
Type: list

Optional: a list of values to pass to the --build-arg parameter.

container_images[].target

Optional: the target for a multi-stage build.

container_images[].tags
Default: ['latest']
Type: list

A list of tags to be added to the image when promoted.

container_images[].siblings
Default: []
Type: list

A list of sibling projects to be copied into {{zuul_work_dir}}/.zuul-siblings. This can be useful to collect multiple projects to be installed within the same Docker context. A -build-arg called ZUUL_SIBLINGS will be added with each sibling project. Note that projects here must be listed in required-projects.

container_build_extra_env
Type: dict

A dictionary of key value pairs to add to the container build environment. This may be useful to enable buildkit with docker builds for example.

container_registry_credentials
Type: dict

This is only required for the upload and promote roles. This is expected to be a Zuul Secret in dictionary form. Each key is the name of a registry, and its value a dictionary with information about that registry.

Example:

container_registry_credentials:
  quay.io:
    username: foo
    password: bar

container_registry_credentials{}.[registry name]
Type: dict

Information about a registry. The key is the registry name, and its value a dict as follows:

container_registry_credentials{}.[registry name]{}.username

The registry username.

container_registry_credentials{}.[registry name]{}.password

The registry password.

container_registry_credentials{}.[registry name]{}.repository

Optional; if supplied this is a regular expression which restricts to what repositories the image may be uploaded. The registry name should be included. The following example allows projects to upload images to repositories within an organization based on their own names:

repository: "^quay.io/myorgname/{{ zuul.project.short_name }}.*"