Skip to content

Deploying Example eBPF Programs On Local Host

This section describes running bpfd and the example eBPF programs on a local host. When running bpfd, it can be run as a process or run as a systemd service. Examples run the same, independent of how bpfd is deployed, other than requiring sudo or not.

NOTE: When running bpfd as a systemd service, bpfd can only access files owned by the bpfd user, which is created by the install script (./script/setup.sh install). In this case, the install script also copies the examples eBPF bytecode from the examples directory to a directory owned by bpfd (/run/bpfd/examples/). When bpfd is run as a privileged process, the bytecode can be loaded from the examples directory. The example code looks in both locations (/run/bpfd/examples/ first), so if the bytecode is recompiled and bpfd is running as a systemd service, run ./script/setup.sh examples to update the copied version.

Building

To build directly on a system, make sure all the prerequisites are met, then build.

Prerequisites

This assumes bpfd is already installed and running on the system. If not, see Setup and Building bpfd.

  1. All requirements defined by the cilium/ebpf package

  2. libbpf development package to get the required eBPF c headers

    Fedora:

    sudo dnf install libbpf-devel

    Ubuntu:

    sudo apt-get install libbpf-dev

  3. Cilium's bpf2go binary

    go install github.com/cilium/ebpf/cmd/bpf2go@master

Building Locally

To build the C based eBPF counter bytecode, run:

    cd bpfd/examples/go-xdp-counter/
    go generate

To build the Userspace GO Client run:

    cd bpfd/examples/go-xdp-counter/
    go build

Repeat for TC and Tracepoint if desired:

    cd bpfd/examples/go-tc-counter/
    go generate
    go build

    cd bpfd/examples/go-tracepoint-counter/
    go generate
    go build

Running On Host

The most basic way to deploy this example is running directly on a host system. First, start or ensure bpfd is up and running. Tutorial will guide you through deploying bpfd. In all the examples of running on a host system, a bpfd-client certificate is used that is generated by bpfd to encrypt the application's connection to bpfd. The diagram below shows go-xdp-counter example, but the go-tc-counter and go-tracepoint-counter examples operate exactly the same way.

go-xdp-counter On Host

Following the diagram (Purple numbers):

  1. When go-xdp-counter userspace is started, it will send a gRPC request over mTLS to bpfd requesting bpfd to load the go-xdp-counter eBPF bytecode located on disk at bpfd/examples/go-xdp-counter/bpf_bpfel.o at a priority of 50 and on interface ens3. These values are configurable as we will see later, but for now we will use the defaults (except interface, which is required to be entered).
  2. bpfd will load it's dispatcher eBPF program, which links to the go-xdp-counter eBPF program and return a UUID referencing the running program.
  3. bpfctl list can be used to show that the eBPF program was loaded.
  4. Once the go-xdp-counter eBPF bytecode is loaded, the eBPF program will write packet counts and byte counts to a shared map.
  5. go-xdp-counter userspace program periodically reads counters from the shared map and logs the value.

Running Privileged

The most basic example, just use sudo to start the go-xdp-counter program. Determine the host interface to attach the eBPF program to and then start the go program with:

    cd bpfd/examples/go-xdp-counter/
    sudo ./go-xdp-counter -iface <INTERNET INTERFACE NAME>

or (NOTE: TC programs also require a direction, ingress or egress)

    cd bpfd/examples/go-tc-counter/
    sudo ./go-tc-counter -direction ingress -iface <INTERNET INTERFACE NAME>

or

    cd bpfd/examples/go-tracepoint-counter/
    sudo ./go-tracepoint-counter

The output should show the count and total bytes of packets as they pass through the interface as shown below:

    sudo ./go-xdp-counter -iface ens3
    2022/12/02 15:59:34 Unable to read /etc/bpfd/bpfd.toml, using default configuration values.
    2022/12/02 15:59:34 Unable to find primary bytecode file: /run/bpfd/examples/go-xdp-counter/bpf_bpfel.o
    2022/12/02 15:59:34 Using Input: Interface=ens3 Priority=50 Source=/home/$USER/src/bpfd/examples/go-xdp-counter/bpf_bpfel.o
    2022/12/02 15:59:35 Program registered with b6b2107c-f1a3-48ac-a145-1073c0979ba4 id
    2022/12/02 15:59:38 0 packets received
    2022/12/02 15:59:38 0 bytes received

    2022/12/02 15:59:41 4 packets received
    2022/12/02 15:59:41 580 bytes received

    :

Use bpfctl to show the go-xdp-counter eBPF bytecode was loaded.

    bpfctl list
    UUID                                  Type  Name   Location                                                                  Metadata
    b6b2107c-f1a3-48ac-a145-1073c0979ba4  xdp   stats  file: { path: /home/$USER/src/bpfd/examples/go-xdp-counter/bpf_bpfel.o }  { priority: 50, iface: ens3, position: 0, proceed_on: pass, dispatcher_return }

Finally, press <CTRL>+c when finished with go-xdp-counter.

    :

    2022/12/02 16:00:56 64 packets received
    2022/12/02 16:00:56 9280 bytes received

    2022/12/02 16:00:59 64 packets received
    2022/12/02 16:00:59 9280 bytes received

    ^C2022/12/02 16:01:00 Exiting...
    2022/12/02 16:01:00 Unloading Program: b6b2107c-f1a3-48ac-a145-1073c0979ba4

Running Unprivileged

To run the examples unprivileged (without sudo), the following three steps must be performed.

Step 1: Create bpfd User Group

The Tutorial guide describes the different modes bpfd can be run in. Specifically, the Systemd Service section explains how to start bpfd and create the bpfd Users and bpfd User Group. bpfd must be started as a Systemd Service and the examples must be run from a User that is a member of the bpfd User Group.

    sudo usermod -a -G bpfd $USER
    exit
    <LOGIN>

If the user running the userspace program is not a member of the bpfd user group, then the userspace program cannot access the map files shared between the BFP program and the userspace program. If the above step is skipped, then the userspace program must be run with sudo or the program must be granted CAP_DAC_OVERRIDE capabilities (sudo /sbin/setcap cap_dac_override=ep ./go-xdp-counter).

Step 2: Grant CAP_BPF Linux Capability

NOTE: Only for kernel versions prior to kernel 5.19

The examples use a map to share data between the userspace side of the program and the eBPF portion. Accessing this map requires access to the CAP_BPF capability for kernel versions prior to kernel 5.19. Run the following command to grant go-xdp-counter access to the CAP_BPF capability:

    cd bpfd/examples/go-xdp-counter/
    sudo /sbin/setcap cap_bpf=ep ./go-xdp-counter

and

    cd bpfd/examples/go-tc-counter/
    sudo /sbin/setcap cap_bpf=ep ./go-tc-counter

and

    cd bpfd/examples/go-tracepoint-counter/
    sudo /sbin/setcap cap_bpf=ep ./go-tracepoint-counter

Reminder: The capability must be re-granted each time the examples are rebuilt.

Step 3: Start go-xdp-counter without sudo

Start go-xdp-counter without sudo:

    ./go-xdp-counter -iface ens3
    2022/12/02 15:59:34 Unable to read /etc/bpfd/bpfd.toml, using default configuration values.
    2022/12/02 15:59:34 Using Input: Interface=ens3 Priority=50 Source=/run/bpfd/examples/go-xdp-counter/bpf_bpfel.o
    2022/12/02 15:59:35 Program registered with b6b2107c-f1a3-48ac-a145-1073c0979ba4 id
    2022/12/02 15:59:38 0 packets received
    2022/12/02 15:59:38 0 bytes received

    2022/12/02 15:59:41 4 packets received
    2022/12/02 15:59:41 580 bytes received

    :

    2022/12/02 16:00:59 64 packets received
    2022/12/02 16:00:59 9280 bytes received

    ^C2022/12/02 16:01:00 Exiting...
    2022/12/02 16:01:00 Unloading Program: b6b2107c-f1a3-48ac-a145-1073c0979ba4

Passing eBPF Bytecode In A Container Image

bpfd can load eBPF bytecode from a container image built following the spec described in eBPF Bytecode Image Specifications. Pre-built eBPF container images for the examples can be loaded from:

  • quay.io/bpfd-bytecode/go-xdp-counter:latest
  • quay.io/bpfd-bytecode/go-tc-counter:latest
  • quay.io/bpfd-bytecode/go-tracepoint-counter:latest

To use the container image, pass the URL to the userspace program:

    ./go-xdp-counter -iface ens3 -image quay.io/bpfd-bytecode/go-xdp-counter:latest
    2022/12/02 16:28:32 Unable to read /etc/bpfd/bpfd.toml, using default configuration values.
    2022/12/02 16:28:32 Using Input: Interface=ens3 Priority=50 Source=quay.io/bpfd-bytecode/go-xdp-counter:latest
    2022/12/02 16:28:34 Program registered with 8d89a6b6-bce2-4d3f-9cee-9cb0c689a90e id
    2022/12/02 16:28:37 4 packets received
    2022/12/02 16:28:37 580 bytes received

    2022/12/02 16:28:40 4 packets received
    2022/12/02 16:28:40 580 bytes received

    ^C2022/12/02 16:28:42 Exiting...
    2022/12/02 16:28:42 Unloading Program: 8d89a6b6-bce2-4d3f-9cee-9cb0c689a90e

Building eBPF Bytecode Container Image

eBPF Bytecode Image Specifications provides detailed instructions on building and shipping bytecode in a container image. To build go-xdp-counter and go-tc-counter eBPF bytecode container image, first make sure the bytecode has been built (i.e. bpf_bpfel.o has been built - see Building), then run the build commands below:

    cd bpfd/examples/go-xdp-counter/
    go generate

    docker build \
      --build-arg PROGRAM_NAME=go-xdp-counter \
      --build-arg SECTION_NAME=stats \
      --build-arg PROGRAM_TYPE=xdp \
      --build-arg BYTECODE_FILENAME=bpf_bpfel.o \
      --build-arg KERNEL_COMPILE_VER=$(uname -r) \
      -f ../../packaging/container-deployment/Containerfile.bytecode . -t quay.io/$USER/go-xdp-counter-bytecode:latest

and

    cd bpfd/examples/go-tc-counter/
    go generate

    docker build \
      --build-arg PROGRAM_NAME=go-tc-counter \
      --build-arg SECTION_NAME=stats \
      --build-arg PROGRAM_TYPE=tc \
      --build-arg BYTECODE_FILENAME=bpf_bpfel.o \
      --build-arg KERNEL_COMPILE_VER=$(uname -r) \
      -f ../../packaging/container-deployment/Containerfile.bytecode . -t quay.io/$USER/go-tc-counter-bytecode:latest

and

    cd bpfd/examples/go-tracepoint-counter/
    go generate

    docker build \
      --build-arg PROGRAM_NAME=go-tracepoint-counter \
      --build-arg SECTION_NAME=tracepoint_kill_recorder \
      --build-arg PROGRAM_TYPE=tracepoint \
      --build-arg BYTECODE_FILENAME=bpf_bpfel.o \
      --build-arg KERNEL_COMPILE_VER=$(uname -r) \
      -f ../../packaging/container-deployment/Containerfile.bytecode . -t quay.io/$USER/go-tracepoint-counter-bytecode:latest

bpfd currently does not provide a method for pre-loading bytecode images (see issue #429), so push the bytecode image to a remote repository. For example:

    docker login quay.io
    docker push quay.io/$USER/go-xdp-counter-bytecode:latest
    docker push quay.io/$USER/go-tc-counter-bytecode:latest

Then run with the privately built bytecode container image:

    ./go-tc-counter -iface ens3 -direction ingress -location image://quay.io/$USER/go-tc-counter-bytecode:latest
    2022/12/02 16:38:44 Unable to read /etc/bpfd/bpfd.toml, using default configuration values.
    2022/12/02 16:38:44 Using Input: Interface=ens3 Priority=50 Source=quay.io/$USER/go-tc-counter-bytecode:latest
    2022/12/02 16:38:45 Program registered with 0d313a4a-a17c-4c70-81ba-3ecc494b900e id
    2022/12/02 16:38:48 4 packets received
    2022/12/02 16:38:48 580 bytes received

    2022/12/02 16:38:51 4 packets received
    2022/12/02 16:38:51 580 bytes received

    ^C2022/12/02 16:38:51 Exiting...
    2022/12/02 16:38:51 Unloading Program: 0d313a4a-a17c-4c70-81ba-3ecc494b900e

Preloading eBPF Bytecode

Another way to load the eBPF bytecode is to pre-load the eBPF bytecode and pass the associated bpfd UUID to the userspace program. This is similar to how eBPF programs will be loaded in Kubernetes, except kubectl commands will be used to create Kubernetes CRD objects instead of using bpfctl, but that is covered in the next section. The userspace programs will skip the loading portion and use the UUID to find the shared map and continue from there.

Referring back to the diagram above, the load and unload are being done by bpfctl and not go-xdp-counter userspace program.

First, use bpfctl to load the go-xdp-counter eBPF bytecode:

    bpfctl load-from-image --image-url quay.io/bpfd-bytecode/go-xdp-counter:latest xdp --iface ens3 --priority 50
    d541af30-69be-44cf-9397-407ee512547a

Then run the go-xdp-counter userspace program, passing in the UUID:

    ./go-xdp-counter -iface ens3 -uuid d541af30-69be-44cf-9397-407ee512547a
    2022/12/02 17:01:38 Using Input: Interface=ens3 Source=d541af30-69be-44cf-9397-407ee512547a
    2022/12/02 17:01:41 180 packets received
    2022/12/02 17:01:41 26100 bytes received

    2022/12/02 17:01:44 184 packets received
    2022/12/02 17:01:44 26680 bytes received

    ^C2022/12/02 17:01:46 Exiting...

Then use bpfctl to unload the eBPF bytecode:

    bpfctl unload d541af30-69be-44cf-9397-407ee512547a