Blubber

Wikimedia Release Engineering's Blubber: a BuildKit frontend

Blubber (GitLab repo) (GitHub mirror repo) is a BuildKit frontend developed by Wikimedia Release Engineering for building application container images. Its core functionality lies in defining container images from a minimal set of declarative constructs written in Yaml.

Key features[edit]

Composability, Determinism, Cache Efficiency, and Secure Defaults: Blubber prioritizes these characteristics in its image building process.
Integration with BuildKit: Blubber functions as a frontend for BuildKit, allowing it to work with buildctl and docker build commands.
Multi-platform Builds and Image Attestations: It supports building images for multiple platforms simultaneously and generating attestations such as Software Bill of Materials (SBOM) and provenance metadata.
YAML-based Configuration: Image definitions are written in YAML, promoting readability and ease of management.

Blubber work-alikes[edit]

While Blubber offers a specialized approach to container image building, several tools provide similar functionalities, particularly in the realm of building images from declarative specifications or within container environments:

Buildah: A command-line tool that can build images from Dockerfiles and create container images from scratch or by using an existing image as a starting point. It offers fine-grained control over the image building process and can be used to build images layer by layer.
Kaniko: A tool designed for building container images within a Kubernetes cluster. It doesn't require a Docker daemon and can build images from Dockerfiles and other sources.
Docker Buildx: An extension for Docker that leverages BuildKit's advanced features, including multi-platform builds and enhanced caching capabilities.
Ko: Focuses on building Go applications and packaging them into container images, simplifying the process of creating images for Go projects.
Pack: Separates the build container from the final run container, allowing for more secure and optimized images.
Apko: A build tool that strictly enforces the use of APK packages for image content, enabling the creation of minimal and secure base images, according to Chainguard^[1].

These tools, similar to Blubber, aim to provide more efficient, secure, and controlled ways of building container images compared to traditional Dockerfile-based approaches, catering to various use cases and preferences within the container ecosystem.

README[edit]

(the content below is embedded from the source repo: https://github.com/wikimedia/blubber/blob/main/README.md)

Blubber is a BuildKit frontend for building application container images from a minimal set of declarative constructs in YAML. Its focus is on composability, determinism, cache efficiency, and secure default behaviors.

Examples

To skip to the examples, see the feature files in the examples directory. The examples are implemented as executable Cucumber tests to ensure Blubber is always working as expected by users.

Concepts

Variants

Blubber supports a concept of composeable configuration variants for defining slightly different container images while still maintaining a sufficient degree of parity between them. For example, images for development and testing may require some development and debugging packages which you wouldn't want in production lest they contain vulnerabilities and somehow end up linked or included in the application runtime.

See the copying from other variants example.

Builders

Builders represent a discrete process and a set of files that is needed to produce an application artifact.

See the builders example.

When defining multiple builders, be sure to use the builders field to ensure an explicit ordering.

Similarly to other configuration keys, builders appearing at the top level of the file will be applied to all variant configurations. Builder keys appearing both at the top level and in a variant, will be merged; whereas builders present only at the top level will be placed first in the execution order.

For a particular variant, builders and the standalone builder keys are mutually exclusive, but different styles can be used for different variants. However, note that top level definitions are applied to all variants, so using one style at the top level precludes the use of the other for all variants.

Usage

Blubber used to include both a CLI and microservice for transpiling to Dockerfile text. It is now exclusively a BuildKit frontend that works with both BuildKit's buildctl command and with docker build directly.

To build from Blubber configuration using buildctl, do:

$ buildctl build --frontend gateway.v0 \
  --opt source=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v0.18.0 \
  --local context=. \
  --local dockerfile=. \
  --opt filename=blubber.yaml \
  --opt variant=test

If you'd like to build directly with docker build (or other toolchains that invoke it like docker-compose), specify a syntax directive at the top of your Blubber configuration like so.

# syntax=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v0.18.0
version: v4
variants:
  my-variant:
  [...]

And invoke docker build --target my-variant -f blubber.yaml .. Note that Docker must have BuildKit enabled as the default builder. You can also use docker buildx which always uses BuildKit.

Docker's build-time arguments are also supported, including those used to provide proxies to build processes.

buildctl build --frontend gateway.v0 \
  --opt source=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v0.18.0 \
  --opt build-arg:http_proxy=http://proxy.example \
  --opt variant=pulls-in-stuff-from-the-internet
  ...

Additional options for the Buildkit frontend

The following options can be passed via command line (via --opt) to configure the build process: * run-variant: bool. Instructs Blubber to run the target variant's entrypoint (if any) as part of the BuildKit image build process * entrypoint-args: JSON array. List of additional arguments for the entrypoint * run-variant-env: JSON object of key/value pairs to set in the environment when run-variant is true.

Example usage:

$ buildctl build --frontend gateway.v0 \
  --opt source=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v0.18.0 \
  --local context=. \
  --local dockerfile=. \
  --opt filename=blubber.yaml \
  --opt variant=test \
  --opt run-variant=true \
  --opt entrypoint-args='["extraParam1", "extraParam2"]' \
  --opt run-variant-env='{"SOME_VARIABLE": "somevalue"}'
  ...

Building for multiple platforms

Blubber's BuildKit frontend supports building for multiple platforms at once and publishing a single manifest index for the given platforms (aka a "fat" manifest). See the OCI Image Index Specification for details.

Note that your build process must be aware of the environment variables set for multi-platform builds in order to perform any cross-compilation needed.

Example usage:

$ buildctl build --frontend gateway.v0 \
  --opt source=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v0.18.0 \
  --local context=. \
  --local dockerfile=. \
  --opt filename=blubber.yaml \
  --opt variant=production \
  --opt platform=linux/amd64,linux/arm64 \
  --output type=image,name=my/multi-platform-app:v1.0,push=true

$ docker manifest inspect my/multi-platform-app:v1.0
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": <n>,
         "digest": "sha256:<digest>",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": <n>,
         "digest": "sha256:<digest>",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

Docs[edit]

Roll up your sleeves[edit]

Have at it: https://github.com/freephile/meza/issues/88

References[edit]

↑ https://www.chainguard.dev/unchained/move-over-dockerfiles-the-new-way-to-craft-containers