Skip to content

Deploying Example eBPF Programs On Local Host

This section describes running bpfman and the example eBPF programs on a local host. When running bpfman, it can be run as a process or run as a systemd service. Examples run the same, independent of how bpfman is deployed.

Building

To build directly on a system, make sure all the prerequisites are met, then build.

Prerequisites

This assumes bpfman is already installed and running on the system. If not, see Setup and Building bpfman.

  1. All requirements defined by the cilium/ebpf package
  2. libbpf development package to get the required eBPF c headers

Fedora:

sudo dnf install libbpf-devel

Ubuntu:

sudo apt-get install libbpf-dev

  1. Cilium's bpf2go binary

go install github.com/cilium/ebpf/cmd/bpf2go@master

Building Locally

To build all the C based eBPF counter bytecode, run:

cd bpfman/examples/
make generate

To build all the Userspace GO Client examples, run:

cd bpfman/examples/
make build

To build only a single example:

cd bpfman/examples/go-tc-counter/
go generate
go build
cd bpfman/examples/go-tracepoint-counter/
go generate
go build
cd bpfman/examples/go-xdp-counter/
go generate
go build

Running On Host

The most basic way to deploy this example is running directly on a host system. First, start or ensure bpfman is up and running. Tutorial will guide you through deploying bpfman. In all the examples of running on a host system, a bpfman-client certificate is used that is generated by bpfman to encrypt the application's connection to bpfman. The diagram below shows go-xdp-counter example, but the go-tc-counter and go-tracepoint-counter examples operate exactly the same way.

go-xdp-counter On Host

Following the diagram (Purple numbers):

  1. When go-xdp-counter userspace is started, it will send a gRPC request over unix socket to bpfman requesting bpfman to load the go-xdp-counter eBPF bytecode located on disk at bpfman/examples/go-xdp-counter/bpf_bpfel.o at a priority of 50 and on interface ens3. These values are configurable as we will see later, but for now we will use the defaults (except interface, which is required to be entered).
  2. bpfman will load it's dispatcher eBPF program, which links to the go-xdp-counter eBPF program and return a UUID referencing the running program.
  3. bpfman list can be used to show that the eBPF program was loaded.
  4. Once the go-xdp-counter eBPF bytecode is loaded, the eBPF program will write packet counts and byte counts to a shared map.
  5. go-xdp-counter userspace program periodically reads counters from the shared map and logs the value.

Running Privileged

To run the go-xdp-counter program, determine the host interface to attach the eBPF program to and then start the go program with:

cd bpfman/examples/go-xdp-counter/
sudo ./go-xdp-counter -iface <INTERNET INTERFACE NAME>

or (NOTE: TC programs also require a direction, ingress or egress)

cd bpfman/examples/go-tc-counter/
sudo ./go-tc-counter -direction ingress -iface <INTERNET INTERFACE NAME>

or

cd bpfman/examples/go-tracepoint-counter/
sudo ./go-tracepoint-counter

The output should show the count and total bytes of packets as they pass through the interface as shown below:

sudo ./go-xdp-counter --iface vethff657c7
2023/07/17 17:43:58 Using Input: Interface=vethff657c7 Priority=50 Source=/home/<$USER>/src/bpfman/examples/go-xdp-counter/bpf_bpfel.o
2023/07/17 17:43:58 Program registered with id 6211
2023/07/17 17:44:01 4 packets received
2023/07/17 17:44:01 580 bytes received

2023/07/17 17:44:04 4 packets received
2023/07/17 17:44:04 580 bytes received

2023/07/17 17:44:07 8 packets received
2023/07/17 17:44:07 1160 bytes received

:

Use the CLI to show the go-xdp-counter eBPF bytecode was loaded.

sudo bpfman list
 Program ID  Name       Type  Load Time
 6211        xdp_stats  xdp   2023-07-17T17:43:58-0400

Finally, press <CTRL>+c when finished with go-xdp-counter.

:

2023/07/17 17:44:34 28 packets received
2023/07/17 17:44:34 4060 bytes received

^C2023/07/17 17:44:35 Exiting...
2023/07/17 17:44:35 Unloading Program: 6211

Passing eBPF Bytecode In A Container Image

bpfman can load eBPF bytecode from a container image built following the spec described in eBPF Bytecode Image Specifications. Pre-built eBPF container images for the examples can be loaded from:

  • quay.io/bpfman-bytecode/go-xdp-counter:latest
  • quay.io/bpfman-bytecode/go-tc-counter:latest
  • quay.io/bpfman-bytecode/go-tracepoint-counter:latest

To use the container image, pass the URL to the userspace program:

sudo ./go-xdp-counter -iface ens3 -image quay.io/bpfman-bytecode/go-xdp-counter:latest
2022/12/02 16:28:32 Using Input: Interface=ens3 Priority=50 Source=quay.io/bpfman-bytecode/go-xdp-counter:latest
2022/12/02 16:28:34 Program registered with id 6223
2022/12/02 16:28:37 4 packets received
2022/12/02 16:28:37 580 bytes received

2022/12/02 16:28:40 4 packets received
2022/12/02 16:28:40 580 bytes received

^C2022/12/02 16:28:42 Exiting...
2022/12/02 16:28:42 Unloading Program: 6223

Building eBPF Bytecode Container Image

eBPF Bytecode Image Specifications provides detailed instructions on building and shipping bytecode in a container image. To build go-xdp-counter and go-tc-counter eBPF bytecode container image, first make sure the bytecode has been built (i.e. bpf_bpfel.o has been built - see Building), then run the build commands below:

cd bpfman/examples/go-xdp-counter/
go generate

docker build \
  --build-arg PROGRAM_NAME=go-xdp-counter \
  --build-arg BPF_FUNCTION_NAME=xdp_stats \
  --build-arg PROGRAM_TYPE=xdp \
  --build-arg BYTECODE_FILENAME=bpf_bpfel.o \
  --build-arg KERNEL_COMPILE_VER=$(uname -r) \
  -f ../../Containerfile.bytecode . -t quay.io/$USER/go-xdp-counter-bytecode:latest

and

cd bpfman/examples/go-tc-counter/
go generate

docker build \
  --build-arg PROGRAM_NAME=go-tc-counter \
  --build-arg BPF_FUNCTION_NAME=stats \
  --build-arg PROGRAM_TYPE=tc \
  --build-arg BYTECODE_FILENAME=bpf_bpfel.o \
  --build-arg KERNEL_COMPILE_VER=$(uname -r) \
  -f ../../Containerfile.bytecode . -t quay.io/$USER/go-tc-counter-bytecode:latest

and

cd bpfman/examples/go-tracepoint-counter/
go generate

docker build \
  --build-arg PROGRAM_NAME=go-tracepoint-counter \
  --build-arg BPF_FUNCTION_NAME=tracepoint_kill_recorder \
  --build-arg PROGRAM_TYPE=tracepoint \
  --build-arg BYTECODE_FILENAME=bpf_bpfel.o \
  --build-arg KERNEL_COMPILE_VER=$(uname -r) \
  -f ../../Containerfile.bytecode . -t quay.io/$USER/go-tracepoint-counter-bytecode:latest

bpfman currently does not provide a method for pre-loading bytecode images (see issue #603), so push the bytecode image to a remote repository. For example:

docker login quay.io
docker push quay.io/$USER/go-xdp-counter-bytecode:latest
docker push quay.io/$USER/go-tc-counter-bytecode:latest

Then run with the privately built bytecode container image:

sudo ./go-tc-counter -iface ens3 -direction ingress -location image://quay.io/$USER/go-tc-counter-bytecode:latest
2022/12/02 16:38:44 Using Input: Interface=ens3 Priority=50 Source=quay.io/$USER/go-tc-counter-bytecode:latest
2022/12/02 16:38:45 Program registered with id 6225
2022/12/02 16:38:48 4 packets received
2022/12/02 16:38:48 580 bytes received

2022/12/02 16:38:51 4 packets received
2022/12/02 16:38:51 580 bytes received

^C2022/12/02 16:38:51 Exiting...
2022/12/02 16:38:51 Unloading Program: 6225

Preloading eBPF Bytecode

Another way to load the eBPF bytecode is to pre-load the eBPF bytecode and pass the associated bpfman program id to the userspace program. This is similar to how eBPF programs will be loaded in Kubernetes, except kubectl commands will be used to create Kubernetes CRD objects instead of using the CLI, but that is covered in the next section. The userspace programs will skip the loading portion and use the program id to find the shared map and continue from there.

Referring back to the diagram above, the load and unload are being done by the CLI and not go-xdp-counter userspace program.

First, use the CLI to load the go-xdp-counter eBPF bytecode:

sudo bpfman load image --image-url quay.io/bpfman-bytecode/go-xdp-counter:latest xdp --iface ens3 --priority 50
 Bpfman State
---------------
 Name:          xdp_stats
 Image URL:     quay.io/bpfman-bytecode/go-xdp-counter:latest
 Pull Policy:   IfNotPresent
 Global:        None
 Metadata:      None
 Map Pin Path:  /run/bpfman/fs/maps/6229
 Map Owner ID:  None
 Map Used By:   6229
 Priority:      50
 Iface:         ens3
 Position:      0
 Proceed On:    pass, dispatcher_return

 Kernel State
----------------------------------
 ID:                               6229
 Name:                             xdp_stats
 Type:                             xdp
 Loaded At:                        2023-07-17T17:48:10-0400
 Tag:                              4b9d1b2c140e87ce
 GPL Compatible:                   true
 Map IDs:                          [2724]
 BTF ID:                           2834
 Size Translated (bytes):          168
 JITed:                            true
 Size JITed (bytes):               104
 Kernel Allocated Memory (bytes):  4096
 Verified Instruction Count:       21

Then run the go-xdp-counter userspace program, passing in the UUID:

sudo ./go-xdp-counter -iface ens3 -id 6229
2022/12/02 17:01:38 Using Input: Interface=ens3 Source=6229
2022/12/02 17:01:41 180 packets received
2022/12/02 17:01:41 26100 bytes received

2022/12/02 17:01:44 184 packets received
2022/12/02 17:01:44 26680 bytes received

^C2022/12/02 17:01:46 Exiting...
2022/12/02 17:01:46 Closing Connection for Program: 6229

Then use the CLI to unload the eBPF bytecode:

sudo bpfman unload 6229