bpfman's Integration with the AF_XDP Device Plugin and CNI for Kubernetes
AF_XDP is an address/socket family that is optimized for high performance packet processing. It takes advantage of XDP (an in Kernel fastpath), which essentially runs an eBPF program as early as possible on a network driver's receive path, and redirects the packet to an AF_XDP socket.
AF_XDP sockets (XSKs) are created in Userspace and have a 1:1 mapping with netdev queues. An XSKMAP is an eBPF map of AF_XDP sockets for a particular netdev. It's a simple key:value map where the key is the netdev's queue-id and the value is the AF_XDP socket that's attached to that queue. The eBPF program (at the XDP hook) will leverage the XSKMAP and the XDP_REDIRECT action to redirect packets to an AF_XDP socket. In the image below the XDP program is redirecting an incoming packet to the XSK attached to Queue 2.
NOTE: If no XSK is attached to a queue, the XDP program will simply pass the packet to the Kernel Network Stack.
+---------------------------------------------------+
| XSK A | XSK B | XSK C |<---+ Userspace
=========================================================|==========
| Queue 0 | Queue 1 | Queue 2 | | Kernel space
+---------------------------------------------------+ |
| Netdev eth0 | |
+---------------------------------------------------+ |
| +=============+ | |
| | key | xsk | | |
| +---------+ +=============+ | |
| | | | 0 | xsk A | | |
| | | +-------------+ | |
| | | | 1 | xsk B | | |
| | BPF | +-------------+ | |
| | prog |-- redirect -->| 2 | xsk C |-------------+
| | (XDP | +-------------+ |
| | HOOK) | xskmap |
| | | |
| +---------+ |
| |
+---------------------------------------------------+
The AF_XDP Device Plugin and CNI project provides the Kubernetes components to provision, advertise and manage AF_XDP networking devices for Kubernetes pods. These networking devices are typically used as a Secondary networking interface for a pod. A key goal of this project is to enable pods to run without any special privileges, without it pods that wish to use AF_XDP will need to run with elevated privileges in order to manage the eBPF program on the interface. The infrastructure will have little to no control over what these pods can load. Therefore it's ideal to leverage a central/infrastructure centric eBPF program management approach. This blog will discuss the eBPF program management journey for the AF_XDP Device Plugin and CNI.