Skip to content

eBPF Tutorial by Example 50: Composable Traffic Control with TCX Links

Ever tried attaching multiple BPF programs to the TC ingress path and got frustrated managing qdisc handles, filter priorities, and the tc CLI? Or needed one application's TC program to coexist safely with another's without accidentally overwriting it? Traditional cls_bpf attachment through tc works, but it inherits decades of queueing discipline plumbing that was never designed for the BPF-centric world. What if you could attach, order, and manage TC programs using the same link-based API that XDP and cgroup programs already enjoy?

This is what TCX (Traffic Control eXtension) solves. Introduced by Daniel Borkmann and merged in Linux 6.6, TCX provides a lightweight, fd-based multi-program attach infrastructure for the TC ingress and egress data path. Programs get BPF link semantics (safe ownership, auto-detachment on close, and explicit ordering through BPF_F_BEFORE / BPF_F_AFTER flags) without touching a single qdisc or filter priority.

In this tutorial, we'll attach two TCX ingress programs to the loopback interface, place one before the other, query the kernel's live chain state, and generate traffic to verify execution order.

The complete source code: https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/50-tcx

Introduction to TCX: Why Classic TC Attachment Needed a Rethink

The Problem: Qdisc Plumbing and Unsafe Ownership

Classic tc BPF attachment (cls_bpf) was bolted onto the existing Traffic Control framework. To attach a BPF program, you first needed a clsact qdisc on the interface, then added a filter with a handle and priority. This worked fine for a single operator, but created real problems in cloud-native environments where multiple applications need to attach TC programs to the same interface:

  1. No ownership model: A tc filter del from one application can accidentally remove another application's program. There's no protection against this because classic tc filters are identified by handle/priority, not by the process that created them.

  2. Priority conflicts: Two applications might pick the same priority number. The second attachment silently replaces the first.

  3. Permanent attachment by default: Classic tc filters persist until explicitly removed. If the application that attached a filter crashes without cleanup, the filter remains, potentially with stale program logic.

  4. CLI dependency: Even with libbpf, the attachment model was tied to netlink, the same mechanism the tc CLI uses. This meant your BPF application was sharing a control plane with every other tc user on the system.

These issues became acute in projects like Cilium, where the BPF dataplane needs to coexist with third-party CNI plugins, observability agents, and security tools that all want to hook into TC.

TCX takes a fundamentally different approach. Instead of piggybacking on qdisc infrastructure, it provides a dedicated, qdisc-less extension point for BPF programs at the TC ingress and egress hooks. The key design principles:

BPF Link Semantics: bpf_program__attach_tcx() creates a BPF_LINK_TYPE_TCX link. Like XDP links and cgroup links, TCX links give you safe ownership: the link is pinned to the file descriptor, auto-detaches when the fd is closed, and cannot be accidentally overridden by another application.

Explicit Ordering: Instead of implicit priority numbers, you place programs relative to each other using BPF_F_BEFORE and BPF_F_AFTER. You can also use BPF_F_REPLACE to atomically swap a specific program. All operations support an expected_revision field that prevents race conditions during concurrent modifications.

Chain Return Codes: TCX defines simplified return codes that make multi-program composition explicit:

Return Code Value Meaning
TCX_NEXT -1 Non-terminating; pass the packet to the next program in the chain
TCX_PASS 0 Accept the packet and terminate the chain
TCX_DROP 2 Drop the packet and terminate the chain
TCX_REDIRECT 7 Redirect the packet and terminate the chain

Unknown return codes are mapped to TCX_NEXT for forward compatibility.

Coexistence with Classic TC: TCX links can coexist with traditional cls_bpf filters on the same interface. The kernel runs TCX programs first, then falls through to classic tcf_classify() if present. This allows gradual migration from classic tc to TCX without a disruptive cutover.

Writing the eBPF Program

Our BPF object contains two programs that demonstrate chain composition. Here is the complete source:

// SPDX-License-Identifier: GPL-2.0
#include <linux/bpf.h>
#include <bpf/bpf_endian.h>
#include <bpf/bpf_helpers.h>

#ifndef TCX_NEXT
#define TCX_NEXT -1
#endif

#ifndef TCX_PASS
#define TCX_PASS 0
#endif

char LICENSE[] SEC("license") = "GPL";

__u64 stats_hits;
__u64 classifier_hits;
__u32 last_len;
__u16 last_protocol;
__u32 last_ifindex;

SEC("tcx/ingress")
int tcx_stats(struct __sk_buff *skb)
{
    stats_hits++;
    last_len = skb->len;
    last_protocol = bpf_ntohs(skb->protocol);
    last_ifindex = skb->ifindex;
    return TCX_NEXT;
}

SEC("tcx/ingress")
int tcx_classifier(struct __sk_buff *skb)
{
    classifier_hits++;
    return TCX_PASS;
}

Let's walk through this step by step.

Section Names: SEC("tcx/ingress")

The SEC("tcx/ingress") annotation tells libbpf that this program should be attached to the TCX ingress hook rather than the classic TC classifier. This is not just a naming convention; libbpf maps this section name to BPF_PROG_TYPE_SCHED_CLS with the appropriate attach type for TCX. The corresponding egress variant is SEC("tcx/egress").

Note that SEC("tc"), SEC("classifier"), and SEC("action") are now considered deprecated by libbpf in favor of the tcx/* section names.

Global Variables as Counters

Instead of using a BPF map for counters, we use global variables (stats_hits, classifier_hits, last_len, etc.). The libbpf skeleton exposes these through skel->bss->stats_hits, which makes the user-space code simpler. This is fine for a single-CPU demo; for production use, you would want per-CPU maps to avoid data races.

Return Codes: TCX_NEXT vs TCX_PASS

This is the heart of TCX composition:

  • tcx_stats returns TCX_NEXT, which means "I've done my work, now pass the packet to the next program in the chain." The chain continues executing.
  • tcx_classifier returns TCX_PASS, which is a terminal verdict: the packet is accepted and no further programs in the chain run.

If we had placed tcx_classifier before tcx_stats in the chain, tcx_stats would never execute because TCX_PASS terminates the chain. Ordering matters, and TCX makes it explicit.

User-Space Loader: Attaching and Querying the Chain

The user-space code demonstrates three key TCX operations: attaching programs, ordering them relative to each other, and querying the live chain.

Step 1: Attach the First Program

classifier_link = bpf_program__attach_tcx(skel->progs.tcx_classifier,
                     ifindex, NULL);

This attaches tcx_classifier to the TCX ingress hook on the specified interface. Passing NULL for options means "use defaults", so the program gets appended to the chain. At this point, the chain has one program.

Step 2: Insert the Second Program Before the First

LIBBPF_OPTS(bpf_tcx_opts, before_opts,
    .flags = BPF_F_BEFORE,
    .relative_fd = bpf_program__fd(skel->progs.tcx_classifier));

stats_link = bpf_program__attach_tcx(skel->progs.tcx_stats,
                    ifindex, &before_opts);

The bpf_tcx_opts structure tells the kernel to insert tcx_stats before tcx_classifier in the chain. The .relative_fd field identifies the reference point, which is the fd of the already-attached classifier program. After this, the chain is: tcx_stats → tcx_classifier.

You could equivalently use BPF_F_AFTER with a different reference to achieve the same ordering. The important point is that you express the desired order directly, rather than hoping that two numeric priorities sort correctly.

Step 3: Query the Chain

LIBBPF_OPTS(bpf_prog_query_opts, query);

query.count = 8;
query.prog_ids = prog_ids;
query.link_ids = link_ids;

err = bpf_prog_query_opts(ifindex, BPF_TCX_INGRESS, &query);

After attachment, the loader queries the kernel for the live chain state. The returned data includes:

  • revision: A monotonically increasing counter that changes on every chain modification. This is the value you would pass as expected_revision if you wanted to perform atomic updates.
  • prog_ids[]: The BPF program IDs in chain order.
  • link_ids[]: The corresponding BPF link IDs.

This allows any observer to determine exactly which programs are attached and in what order, which is invaluable for debugging multi-program pipelines.

Step 4: Generate Traffic and Read Counters

The loader sends a UDP packet to 127.0.0.1 (port 9, discard) to trigger the chain, waits briefly, then reads the global variables to verify both programs executed:

printf("  tcx_stats hits      : %llu\n",
       (unsigned long long)skel->bss->stats_hits);
printf("  tcx_classifier hits : %llu\n",
       (unsigned long long)skel->bss->classifier_hits);

If both counters are 1, the chain worked as expected: tcx_stats ran first (recording metadata and returning TCX_NEXT), then tcx_classifier ran second (counting the packet and returning TCX_PASS).

Compilation and Execution

This example requires Linux 6.6+ with TCX support and a recent libbpf.

cd bpf-developer-tutorial/src/50-tcx
make
sudo ./tcx_demo -i lo

Expected output:

Attached TCX programs to lo (ifindex=1)
TCX ingress chain revision: 3
  slot 0: prog_id=812 link_id=901
  slot 1: prog_id=811 link_id=900

Counters:
  tcx_stats hits      : 1
  tcx_classifier hits : 1
  last ifindex        : 1
  last protocol       : 0x0800
  last length         : 46

The revision is 3 because the chain was modified twice: once when tcx_classifier was attached (revision went from 0 to 1), and once when tcx_stats was inserted before it (revision went to 2). The query itself increments the revision to 3.

If you want to inspect the attach behavior without traffic, add -n:

sudo ./tcx_demo -i lo -n

Use -v to enable libbpf debug output, which is helpful for seeing the low-level BPF syscall sequence.

How This Differs from Lesson 20 (Classic TC)

Lesson 20-tc teaches the classic TC data path: creating a clsact qdisc, attaching a SEC("tc") program as a filter, and using __sk_buff for packet inspection. That lesson is still valuable because the packet processing model is identical: TCX programs receive the same __sk_buff context and use the same helpers for packet parsing.

What TCX replaces is the control plane:

Aspect Classic TC (Lesson 20) TCX (Lesson 50)
Attach mechanism Netlink / tc CLI bpf_program__attach_tcx()
Ownership None; anyone can tc filter del BPF link; auto-detaches on fd close
Ordering Implicit priority numbers Explicit BPF_F_BEFORE / BPF_F_AFTER
Multi-program Manual priority management Built-in chain with revision tracking
Section name SEC("tc") SEC("tcx/ingress") / SEC("tcx/egress")
Kernel requirement Any modern kernel Linux 6.6+

If you are building new libbpf-based networking tools, TCX is the recommended interface. Cilium has already migrated from classic tc to TCX for its dataplane.

Summary

In this tutorial, we learned how TCX modernizes TC program attachment by replacing qdisc-based plumbing with BPF link semantics. We attached two ingress programs, controlled their execution order with BPF_F_BEFORE, queried the live chain with bpf_prog_query_opts(), and verified that both programs executed in the correct order. TCX provides safe ownership, explicit ordering, revision-aware updates, and coexistence with classic TC, making it the foundation for composable, multi-program traffic control in modern eBPF applications.

If you'd like to learn more about eBPF, visit our tutorial code repository at https://github.com/eunomia-bpf/bpf-developer-tutorial or website https://eunomia.dev/tutorials/ for more examples and complete tutorials.

References

Share on Share on