Skip to content

bpfman Container Images

Container images for bpfman are automatically built and pushed to quay.io/ under the :latest tag whenever code is merged into the main branch of the bpfman and bpfman-operator repositories.

  • quay.io/bpfman: This repository contains images needed to run bpfman. It contains the xdp-dispatcher and tc-dispatcher eBPF container images, which are used by bpfman to allow multiple XDP or TC programs to be loaded on a given interface. It also includes the container images which are used to deploy bpfman in a Kubernetes deployment:
    • bpfman: Packages all the bpfman binaries, including bpfman CLI, bpfman-ns and bpfman-rpc.
    • bpfman-agent: Agent that listens to KubeAPI Server and makes calls to bpfman to load or unload eBPF programs based on user intent.
    • bpfman-operator: Operator for deploying bpfman.
    • tc-dispatcher: eBPF container image containing the TC Dispatcher, which is used by bpfman to manage and allow multiple TC based programs to be loaded on a given TC hook point.
    • xdp-dispatcher: eBPF container image containing the XDP Dispatcher, which is used by bpfman to manage and allow multiple TC based programs to be loaded on a given XDP hook point.
    • csi-node-driver-registrar: CSI Driver used by bpfman.
    • bpfman-operator-bundle: Image containing all the CRDs (Custom-Resource-Definitions) used by bpfman-agent to define Kubernetes objects used to manage eBPF programs.
  • quay.io/bpfman-bytecode: This repository contains eBPF container images for all of the generated bytecode from examples/ and integration-test/.
  • quay.io/bpfman-userspace: This repository contains userspace container images for all of the example programs in examples/.

Multiple Architecture Support

All bpfman related container images that are automatically built and pushed to quay.io/ contain a manifest file and images built for the following architectures:

  • x86_64
  • arm64
  • ppc64le
  • s390x

Locally Build bpfman-operator and bpfman-agent Container Images

When testing or developing in bpfman-operator, it may be necessary to run with updated changes to the bpfman-operator or bpfman-agent container images. The local Makefile will build and load both images based on the current changes: 

cd bpfman-operator/

make build-images
make run-on-kind

Locally Build bpfman Container Image

When testing or developing in bpfman-operator, it may be necessary to run with updated changes to bpfman. By default, bpfman-agent uses quay.io/bpfman/bpfman:latest. To build the bpfman binaries in a container image, run:

cd bpfman/

docker build -f ./Containerfile.bpfman.local . -t quay.io/$QUAY_USER/bpfman:test

Use any registry, image name and tag, above is just an example. Next, build and deploy the bpfman-operator and bpfman-agent with the locally built bpfman container image.

cd bpfman-operator/

BPFMAN_IMG=quay.io/$QUAY_USER/bpfman:test make build-images
BPFMAN_IMG=quay.io/$QUAY_USER/bpfman:test make run-on-kind

To use, the Kind cluster must have access to the image. So either the image needs to be pushed to a registry and made public (make public via the repo GUI after the push) before executing the make run-on-kind command shown above:

docker push quay.io/$QUAY_USER/bpfman:test

OR it can be loaded into the kind cluster after the cluster is running:

kind load docker-image quay.io/$QUAY_USER/bpfman:test --name bpfman-deployment

Now the image should be running in the Kind cluster:

kubectl get pods -A
 NAMESPACE   NAME                               READY   STATUS    RESTARTS   AGE
 bpfman      bpfman-daemon-87fqg                3/3     Running   0          16m
 bpfman      bpfman-operator-7f67bc7c57-bc6lk   2/2     Running   0          16m
 :

kubectl describe pod -n bpfman bpfman-daemon-87fqg
 Name:             bpfman-daemon-87fqg
 Namespace:        bpfman
 :
 Containers:
  bpfman:
    Container ID:  containerd://1777d1810f3648f43df775e9d9af79406eaffc5694aa712da04c3f4e578093b3
    Image:         quay.io/$QUAY_USER/bpfman:test
    Image ID:      quay.io/$QUAY_USER/bpfman@sha256:f2c94b7acff6b463fc55232a1896816283521dd1ba5560b0d0779af99f811cd0
:

Locally Build TC or XDP Dispatcher Container Image

The TC and XDP Dispatcher images are automatically built and pushed to quay.io/ under the :latest tag whenever code is merged into the main branch of the github.com/bpfman/bpfman. If a dispatcher container image needs to be built locally, use the following steps.

Build the object files:

cargo xtask build-ebpf --libbpf-dir ~/src/libbpf/

$ ls .output/tc_dispatcher.bpf/
bpf_arm64_bpfel.o  bpf_powerpc_bpfel.o  bpf_s390_bpfeb.o  bpf_x86_bpfel.o

$ ls .output/xdp_dispatcher_v2.bpf/
bpf_arm64_bpfel.o  bpf_powerpc_bpfel.o  bpf_s390_bpfeb.o  bpf_x86_bpfel.o

Then build the bytecode image files:

bpfman image build -f Containerfile.bytecode -t quay.io/$QUAY_USER/tc-dispatcher:test -b .output/tc_dispatcher.bpf/bpf_x86_bpfel.o
bpfman image build -f Containerfile.bytecode -t quay.io/$QUAY_USER/xdp-dispatcher:test -b .output/xdp_dispatcher_v2.bpf/bpf_x86_bpfel.o

If a multi-arch image is needed, use:

bpfman image build -f Containerfile.bytecode.multi.arch -t quay.io/$QUAY_USER/tc-dispatcher:test -c .output/tc_dispatcher.bpf/
bpfman image build -f Containerfile.bytecode.multi.arch -t quay.io/$QUAY_USER/xdp-dispatcher:test -c .output/xdp_dispatcher_v2.bpf/

Note

To build images for multiple architectures on a local system, docker (or podman) may need additional configuration settings to allow for caching of non-native images. See https://docs.docker.com/build/building/multi-platform/ for more details.

Locally Build Example Container Images

The example images are automatically built and pushed to quay.io/ under the :latest tag whenever code is merged into the main branch of the github.com/bpfman/bpfman. For each example, there is a bytecode and a userspace image. For official bpfman images, bytecode images are pushed to quay.io/bpfman-bytecode and userspace images are pushed to quay.io/bpfman-userspace. For example:

The Makefile in the examples directory has commands to build both sets of images. Image names and tags can be controlled using environment variables. If private images are being generated, both bytecode and userspace images will probably be pushed to the same account, so bytecode and userspace images will need to be distinguished by either fully qualified image names (using IMAGE_TC_BC, IMAGE_TC_US, IMAGE_XDP_BC, IMAGE_XDP_US, etc) or unique tags for each (TAG_BC, TAG_US). See make help in the examples directory and the samples below.

Example Bytecode Container Images

If an example bytecode container image needs to be built locally, use the following to build the bytecode container image, (optionally passing the USER_BC and TAG_BC for the image):

# Build images for all eBPF program types
$ make build-bc-images USER_BC=$QUAY_USER TAG_BC=test-bc
:
 => pushing quay.io/$QUAY_USER/go-kprobe-counter:test-bc with docker
:
 => pushing quay.io/$QUAY_USER/go-tc-counter:test-bc with docker
:
 => pushing quay.io/$QUAY_USER/go-tracepoint-counter:test-bc with docker
:

-- OR --

# Build image for a single eBPF program type, XDP in this example
$ make build-bc-xdp USER_BC=$QUAY_USER TAG_BC=test-bc
:
 => pushing quay.io/$QUAY_USER/go-xdp-counter:test-bc with docker

If a multi-arch image is needed, use (appending PLATFORM):

$ make build-bc-xdp USER_BC=$QUAY_USER TAG_BC=test-bc PLATFORM=linux/amd64,linux/arm64,linux/ppc64le,linux/s390x
:
 => pushing quay.io/$QUAY_USER/go-xdp-counter:test-bc with docker

Note

To build images for multiple architectures on a local system, docker (or podman) may need additional configuration settings to allow for caching of non-native images. See https://docs.docker.com/build/building/multi-platform/ for more details.

Example Userspace Container Images

If an example userspace container image needs to be built locally, use the following to build the userspace container images, (optionally passing the USER_US and TAG_US for the image):

cd bpfman/examples/

# Build all images
$ make build-us-images USER_US=$QUAY_USER TAG_US=test-us
:
 => pushing quay.io/$QUAY_USER/go-kprobe-counter:test-us with docker
:
 => pushing quay.io/$QUAY_USER/go-tc-counter:test-us with docker
:
 => pushing quay.io/$QUAY_USER/go-tracepoint-counter:test-us with docker
:

-- OR --

# Build a single image
$ make build-us-xdp USER_US=$QUAY_USER TAG_US=test-us
:
 => pushing quay.io/$QUAY_USER/go-xdp-counter:test-us with docker

If a multi-arch image is needed, use (appending PLATFORM):

$ make build-us-xdp USER_US=$QUAY_USER TAG_US=test-us PLATFORM=linux/amd64,linux/arm64,linux/ppc64le,linux/s390x
:
 => pushing quay.io/$QUAY_USER/go-xdp-counter:test-us with docker

Note

To build images for multiple architectures on a local system, docker (or podman) may need additional configuration settings to allow for caching of non-native images. See https://docs.docker.com/build/building/multi-platform/ for more details.

Adding Additional Container Images

When adding a new container image to one of the bpfman repositories, whether it be via the examples or integration tests, several steps need to be performed.

  • One of the maintainers of the bpfman quay.io repositories must:
    • Add the image to the quay.io repository.
    • Make the new image public.
    • On the image, provide Write access to the bpfman+github_actions robot account.
  • Add the new image to the bpfman/.github/workflows/image-build.yaml so the image is built and pushed on each PR merge.
  • For examples, update the examples/Makefile to build the new images.

Signing Container Images

Signing eBPF container images is encouraged and can be easily done using cosign. Below is a summary of the steps needed to sign an image.

First, install cosign:

go install github.com/sigstore/cosign/v2/cmd/cosign@latest

Then sign the image. The cosign command will generate a URL. Follow the sigstore URL and login with either GitHub, Google to Microsoft. That will generate a verification code that will complete the cosign command.

cosign sign -y quay.io/$QUAY_USER/test-image@sha256:55fe3cfe46409939876be27f7ed4d2948842918145f6cda167d0c31fdea2046f
Generating ephemeral keys...
Retrieving signed certificate...
:
https://oauth2.sigstore.dev/auth/auth?access_type=online&client_id=sigstore&code_challenge=EwHYBahRxlbli-oEXxS9DoEzEWcyuS_f1lLBhntCVFI&code_challenge_method=S256&nonce=2kR9mJbP0eUxFBAQI9Nhs6LyS4l&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&scope=openid+email&state=2kR9mIqOn6IgmAw46BxVrnEEi0M
Enter verification code: wq3g58qhw6y25wwibcz2kgzfx

Successfully verified SCT...
tlog entry created with index: 120018072
Pushing signature to: quay.io/$QUAY_USER/test-image

Containerfiles

There are multiple Containerfiles in the bpfman repositories. Below is a summary of the files and their purpose.

Userspace Containerfiles

  • bpfman/Containerfile.bpfman.local: This file is used to create a userspace container image with bpfman binaries (bpfman CLI, bpfman-rpc and bpfman-ns). It can be used to run local bpfman code in a Kubernetes cluster with the bpfman-operator and bpfman-agent.
  • bpfman/Containerfile.bpfman.multi.arch: This file is used to create a userspace container image with bpfman binaries (bpfman CLI, bpfman-rpc and bpfman-ns), but for multiple architectures. It is used by the bpfman/.github/workflows/image-build.yaml file to build bpfman multi-arch images on every github Pull Request merge. The resulting images are stored in quay.io.
  • bpfman/Containerfile.bpfman.openshift: This file is used to create a userspace container image with bpfman binaries (bpfman CLI, bpfman-rpc and bpfman-ns). It is used by internal OpenShift build processes.
  • bpfman/examples/go-*-counter/container-deployment/Containerfile.go-*-counter: Where '*' is one of the bpfman supported program types (tc, tcx, tracepoint, etc.). These files are used to create the userspace container images associated with the examples.
  • bpfman-operator/Containerfile.bpfman-agent: This file is used to create a userspace container image with bpfman-agent.
  • bpfman-operator/Containerfile.bpfman-agent.openshift: This file is used to create a userspace container image with bpfman-agent. It is used by internal OpenShift build processes.
  • bpfman-operator/Containerfile.bpfman-operator: This file is used to create a userspace container image with bpfman-operator.
  • bpfman-operator/Containerfile.bpfman-operator.openshift: This file is used to create a userspace container image with bpfman-operator. It is used by internal OpenShift build processes.
  • bpfman-operator/Containerfile.bundle: This file is used to create a container image with all the Kubernetes object definitions (ConfigMaps, Custom Resource Definitions (CRDs), Roles, Role Bindings, Service, Service Accounts, etc) bpfman needs to be deployed in a Kubernetes cluster.

Bytecode Containerfiles

  • bpfman/Containerfile.bytecode: This file is used to create a container image with eBPF bytecode packaged inside. The Containerfile applies labels to the container image describing the bytecode for consumers of the image. See eBPF Bytecode Image Specifications for more details.
  • bpfman/Containerfile.bytecode.multi.arch: This file is used to create a container image with eBPF bytecode packaged inside, but packages eBPF bytecode for multiple architectures. The Containerfile applies labels to the container image describing the bytecode for consumers of the image. See eBPF Bytecode Image Specifications for more details.