Skip to content

Blog

Share on Share on

  • 30 min read

OS-Level Challenges in LLM Inference and Optimizations

Large Language Model (LLM) inference pushes computing systems to their limits, not only in raw compute but also in how the operating system (OS) manages resources. This report examines OS-level challenges for LLM inference and explores potential solutions. We focus first on key bottlenecks – memory management, CPU scheduling, I/O, and real-time constraints – then discuss how kernel-level techniques (like eBPF and custom scheduling) can address these issues. We also consider the impact of system calls and page faults on performance, security/isolation concerns in multi-tenant environments, best practices and emerging research in OS customization for AI, and practical considerations for implementing such optimizations in a research project. The goal is to guide a research initiative by outlining the challenges and then potential solutions and areas for further investigation.

  • 33 min read

WASI and the WebAssembly Component Model: Current Status

WebAssembly (WASM) has evolved from a browser-based technology into a promising runtime for server and embedded applications. Key to this evolution are the WebAssembly System Interface (WASI) – a standardized set of syscalls for WASM outside the browser – and the emerging Component Model, which enables modular, language-agnostic composition of WebAssembly modules. This report analyzes the current status of WASI (including its major runtimes like Wasmtime and Wasmer) and the Component Model, focusing on the technical limitations that impede adoption. We draw from official proposals, runtime issue trackers, academic studies, and industry experiences to highlight critical gaps, unresolved bugs, and feature requests. Case studies illustrate how these challenges block real-world use cases, and we conclude with recommendations for improving WebAssembly’s performance, security, and integrability with host environments.

  • 41 min read

eBPF Ecosystem Progress in 2024–2025: A Technical Deep Dive

Introduction and Summary

Extended Berkeley Packet Filter (eBPF) continues to rapidly evolve, cementing its role as a cornerstone for operating system extensibility. In 2024 and into early 2025, the eBPF ecosystem saw significant advancements across the Linux kernel, tooling, security, networking, and observability domains. This report provides a comprehensive technical deep dive into these developments, with a high-level summary here and detailed sections below. Key highlights include:

This is a AI generated report from OpenAI Deep Reasearch. Let's see how it performs for eBPF.

  • 33 min read

Security Vulnerabilities Study in Software Extensions and Plugins

Introduction

Software extensions and plugins allow customization and added features across many systems – from web servers and databases to browsers, IDEs, and CMS platforms. However, these extensibility mechanisms introduce security risks. Extensions often run with high privileges inside host applications, so a flaw in an extension can compromise the entire system ([PDF] Protecting Browsers from Extension Vulnerabilities - Google Research). Common vulnerability types include memory safety issues (buffer/stack overflows and heap corruptions), privilege escalations, sandbox escapes, infinite loops or livelocks causing Denial-of-Service, and arbitrary code execution (ACE). This report examines historical and recent vulnerabilities in extensions/plugins across various ecosystems, highlighting notable CVEs, trends, case studies of major incidents, and how improved security models like an Extension Interface Model (EIM) could mitigate such issues.

  • 7 min read

Can LLMs understand Linux kernel? A New AI-Powered Approach to Understanding Large Codebases

Ever tried diving into a massive codebase like the Linux kernel and felt like you were swimming in an ocean of code with no land in sight? Trust me, you're not alone. Understanding large, complex, and constantly evolving codebases is like trying to read a never-ending novel that's being written by thousands of authors simultaneously. It's overwhelming, to say the least.

See our arxiv and GitHub repository for more details!

  • 6 min read

Building High-Performance Userspace eBPF VMs with LLVM

We are excited to introduce llvmbpf, a new project aimed at empowering developers with a high-performance, multi-architecture eBPF virtual machine (VM) that leverages the LLVM framework for Just-In-Time (JIT) and Ahead-Of-Time (AOT) compilation.

This component is part of the bpftime project but focuses solely on the core VM. It operates as a standalone eBPF VM library or a compiler tool. This library is optimized for performance, flexibility, and minimal dependencies, making it easy to integrate into various environments without unnecessary overhead.

  • 16 min read

The Past, Present, and Future of eBPF and Its Path to Revolutionizing Systems

This blog post mainly references Alexei Starovoitov's presentation "Modernize BPF for the Next 10 Years" at BPFConf 2024.

Imagine having a Swiss Army knife for your computer's core operations—something that lets you peek inside how data moves, tweak processes on the fly, and monitor everything in real-time. That’s exactly what eBPF (Extended Berkeley Packet Filter) offers. Over the past decade, eBPF has transformed from a simple packet filtering tool into a powerhouse for networking, observability, and security. So, what’s next for eBPF? Let’s dive into its journey, explore where it’s headed in the next ten years, and discuss the challenges and opportunities that lie ahead. This exploration will help us understand how eBPF is shaping the future of modern systems.

  • 6 min read

Simplifying Kernel Programming: The LLM-Powered eBPF Tool

Kernel programming can be intimidating, requiring deep knowledge of operating system internals and programming constraints. Our latest tool, Kgent, aims to change that by making it easier than ever to create extended Berkeley Packet Filters (eBPF) programs. Kgent leverages the power of large language models (LLMs) to translate natural language prompts into eBPF code, opening up kernel programming to a wider audience.

Our paper, "Kgent: Kernel Extensions Large Language Model Agent," was recently presented at eBPF '24: Proceedings of the ACM SIGCOMM 2024 Workshop on eBPF and Kernel Extensions. Let's dive into what makes Kgent a game-changer for kernel programming.

  • 15 min read

The eBPF Evolution and Future: From Linux Origins to Cross-Platform Dominance

eBPF (Extended Berkeley Packet Filter) has become a revolutionary force in operating system kernel programming since its inception. Initially created for network packet filtering, eBPF has evolved into a versatile kernel extension tool, supporting system monitoring, performance analysis, security policy enforcement, and more. As the technology has advanced, eBPF has gained widespread adoption not only on Linux platforms but also on other operating systems like Windows and macOS, showing significant potential as a cross-platform infrastructure technology. In this context, eBPF is driving the evolution of modern operating system architectures, opening new possibilities for cloud-native infrastructure and platform engineering.

  • 8 min read

Implementing an Inline Hook in C in 5 minutes

One of the fascinating aspects of programming comes when we try to alter the behavior of a program while it is running.

In this tutorial, we shed light on one method that can make this possible - an "Inline Hook". We will delve into how you can manipulate the execution flow of a program in the C programming language. By implementing an Inline Hook, we aim to divert the program's execution flow to our function, then returning it back to the normal flow.

  • 15 min read

The Evolution and Impact of eBPF: A list of Key Research Papers from Recent Years

This is a list of eBPF related papers I read in recent years, might be helpful for people who are interested in eBPF related research.

eBPF (extended Berkeley Packet Filter) is an emerging technology that allows safe execution of user-provided programs in the Linux kernel. It has gained widespread adoption in recent years for accelerating network processing, enhancing observability, and enabling programmable packet processing.

This document list some key research papers on eBPF over the past few years. The papers cover several aspects of eBPF, including accelerating distributed systems, storage, and networking, formally verifying the eBPF JIT compiler and verifier, applying eBPF for intrusion detection, and automatically generating hardware designs from eBPF programs.

  • 4 min read

Introducing eunomia-bpf v1.0: Simplifying eBPF with CO-RE and WebAssembly

The world of eBPF (Extended Berkeley Packet Filter) has been rapidly evolving, offering developers powerful tools to monitor and modify the behavior of systems at the kernel level. Today, we're thrilled to introduce the latest milestone in this journey - eunomia-bpf v1.0. This release is a testament to our commitment to simplifying and enhancing the eBPF development experience with CO-RE (Compile Once, Run Everywhere) and WebAssembly.

  • 16 min read

The Secure Path Forward for eBPF runtime: Challenges and Innovations

Yusheng Zheng

Extended Berkeley Packet Filter (eBPF) represents a significant evolution in the way we interact with and extend the capabilities of modern operating systems. As a powerful technology that enables the Linux kernel to run sandboxed programs in response to events, eBPF has become a cornerstone for system observability, networking, and security features.

However, as with any system that interfaces closely with the kernel, the security of eBPF itself is paramount. In this blog, we delve into the often-overlooked aspect of eBPF security, exploring how the mechanisms intended to safeguard eBPF can themselves be fortified. We'll dissect the role of the eBPF verifier, scrutinize the current access control model, and investigate potential improvements from ongoing research. Moreover, we'll navigate through the complexities of securing eBPF, addressing open questions and the challenges they pose to system architects and developers alike.

  • 10 min read

Userspace eBPF Runtimes: Overview and Applications

Yusheng Zheng

In this blog post, we'll dive into the world of eBPF in userspace. While many are familiar with kernel-based eBPF, userspace eBPF runtimes have been making significant strides and offer compelling use cases. We will also compare userspace eBPF runtimes with Wasm runtimes, another popular technology in the cloud-native and edge computing landscape. Among these, we're excited to introduce bpftime. Powered by an LLVM JIT/AOT backend, our benchmarks suggest that bpftime stands out as one of the fastest userspace eBPF runtimes available.

  • 13 min read

bpftime: Extending eBPF from Kernel to User Space

Yu Sheng Zheng, Yu Tong

eBPF is a revolutionary technology that originated in the Linux kernel, enabling sandboxed programs to run within the operating system's kernel. It is used to safely and efficiently extend the kernel's capabilities without altering its source code or loading kernel modules.

In this blog, we are excited to introduce a new open-source user-space eBPF runtime: https://github.com/eunomia-bpf/bpftime. bpftime further expands the capabilities of eBPF, allowing existing eBPF tools and applications, such as BCC tools, bpftrace, Deepflow, etc., to run in non-privileged user space without any code modifications, while using the same libraries and toolchains as kernel eBPF.

bpftime not only provides dynamic tracing or extension mechanisms like Uprobe and system call tracepoints, but also offers an order of magnitude performance improvement over kernel Uprobe. Moreover, like kernel eBPF, it requires no manual code instrumentation or process restarts. bpftime supports inter-process eBPF maps through user-space shared memory, while being compatible with kernel eBPF maps, enabling seamless operations with kernel eBPF infrastructure. Additionally, it includes high-performance LLVM JIT/AOT compilers for various architectures, as well as a lightweight JIT and interpreter for x86. Through performance data and real-world examples, we will demonstrate how bpftime can be effective in the real world and provide insights into its future development. We hope bpftime will bring unprecedented performance and flexibility to system monitoring, analysis, and extension. We also introduced the design and implementation of bpftime at the Linux plumbers 23 conference[2].

  • 7 min read

Use ChatGPT to write eBPF programs and trace the Linux kernel with natural language

eBPF is a revolutionary technology that originated in the Linux kernel and allows sandboxed programs to run in the kernel of an operating system. It is used to securely and efficiently extend the functionality of the kernel without changing its source code or loading kernel modules. Today, eBPF is widely used in various scenarios: in modern data centers and cloud-native environments, it can provide high-performance network packet processing and load balancing; with very low resource overhead, it enables observability of various fine-grained metrics, helping application developers trace applications and gain insights for performance troubleshooting; it ensures secure execution of applications and container runtimes, and more. eBPF has become an increasingly popular technology that helps us to efficiently trace and analyze almost all applications in the kernel and user space.

However, developing eBPF programs or tracing various events generated by the kernel requires certain expertise. For developers unfamiliar with this technology, it can be challenging. In this case, the new ideas brought by our demo tool, GPTtrace, might help you solve this problem. It uses ChatGPT to write eBPF programs and trace the Linux kernel with natural language: https://github.com/eunomia-bpf/GPTtrace

If you are a developer who wants to trace and analyze more efficiently, similar solutions are definitely worth trying. The combination of ChatGPT and eBPF technology will play a more important role in future software development, debugging, and observability scenarios, and it may also bring a new interactive learning paradigm.

  • 6 min read

Simplifying eBPF Development: GitHub Templates and Codespaces for Online Compilation and Execution

Embarking on the eBPF journey can feel daunting, especially when confronted with setting up the perfect environment or making the ideal language choice. But what if there was a streamlined way to immerse yourself in eBPF without the initial hurdles? Look no further! This guide unveils the magic of GitHub templates combined with GitHub Codespaces, empowering you to seamlessly initiate, compile, and run eBPF projects online. Dive in, click once, and turbocharge your eBPF expedition!

  • 15 min read

When Wasm Meets eBPF: Writing, Distributing, Loading, and Running eBPF Programs with WebAssembly

In today's cloud-native world, eBPF and WebAssembly are two of the hottest lightweight code execution sandboxes/virtual machines. Both of them run high-performance bytecode programs compiled from languages such as C, C++, and Rust, and both are cross-platform and portable. The biggest difference between them is that eBPF runs in the Linux kernel, while WebAssembly runs in user space. We want to make an attempt to integrate them: using Wasm to write universal eBPF programs that can be distributed to different versions and architectures of Linux kernels without the need for recompiling.

  • 9 min read

eBPF Advanced: Overview of New Kernel Features

The Linux kernel primarily released versions 5.16-5.19, 6.0, and 6.1 in 2022, each of which introduced numerous new features for eBPF. This article will provide a brief introduction to these new features, and for more detailed information, please refer to the corresponding link. Overall, eBPF remains one of the most active modules in the kernel, and its functionality is still rapidly evolving. In a sense, eBPF is rapidly evolving towards a complete kernel-level programmable interface.

  • 3 min read

Progress of eunomia-bpf in March

The eunomia-bpf project is an open-source project aimed at providing a set of tools for writing and running eBPF programs more conveniently in the Linux kernel. In the past month, the project has made some new progress. Here is an overview of these advances.

  • 11 min read

eunomia-bpf 0.3.0 Release: Easily Build, Package, and Publish Full eBPF Applications by Writing Kernel-Mode Code

Introduction to eunomia-bpf

eBPF, derived from BPF, is an efficient and flexible virtual machine component within the kernel. It executes bytecode at various kernel hook points in a secure manner, enabling developers to build performance analysis tools, software-defined networks, security solutions, and more. However, there are some inconveniences when it comes to developing and using eBPF applications:

  • Setting up and developing eBPF programs is a complex task. It requires handling interactions and information processing between kernel mode and user mode, as well as configuring the environment and writing corresponding build scripts.
  • Currently, it is difficult to achieve compatibility and unified management among tools written in different user mode languages like C, Go, Rust, etc. There is a challenge in integrating various development ecosystems, such as supporting multiple architectures, languages, and kernel versions. How can we package, distribute, and publish binary eBPF programs in a standardized and convenient way? Additionally, there is a need to easily adjust mounting points, parameters, and other aspects of eBPF programs.
  • How can we make it easier to use eBPF tools? Is it possible to download and use them with just one command from the cloud, similar to Docker? Can we run eBPF programs as services, allowing hot updates and dynamic insertion/removal through HTTP requests and URLs?

eunomia-bpf is an open-source eBPF dynamic loading runtime and development toolchain designed to simplify the development, building, distribution, and execution of eBPF programs. It is based on the CO-RE lightweight development framework of libbpf.

  • 15 min read

Wasm-bpf: Bridging WebAssembly and eBPF for Kernel Programmability

Authors: Yu-Sheng Zheng, Mao-Lin Chou

Wasm was initially developed as a secure sandbox for browsers, and has since evolved into a high-performance, cross-platform, and multi-language software sandbox environment for cloud-native software components. The lightweight containerization provided by Wasm also makes it a suitable runtime for the next generation of serverless platforms. Another exciting trend is the rise of eBPF, which enables cloud-native developers to build secure networking, service mesh, and observability components. It is gradually penetrating and deepening into various components of the kernel, providing more powerful programmable interaction capabilities in kernel space.

Wasm-bpf is a new open-source project[1] that defines a set of abstract eBPF-related system interfaces and provides a corresponding development toolchain, library, and a universal Wasm + eBPF runtime platform instance. This allows applications in any Wasm virtual machine or Wasm lightweight container to sink and extend their use cases into kernel space, gaining access to almost all data from both kernel and user space. This enables programmable control at the entire operating system level in multiple areas such as networking and security, greatly expanding the applications of the WebAssembly ecosystem outside the browser.

  • 11 min read

Wasm-bpf: A Common eBPF Kernel Programmability for Cloud-Native Webassembly

Author: Yusheng Zheng, Mao-Lin Chen

Originally developed with a browser-safe sandbox in mind, Wasm has evolved to make WebAssembly a high-performance, cross-platform and multilingual software sandbox environment for cloud-native software components, and Wasm lightweight containers are well suited as the next-generation serverless platform runtime. Another exciting trend is the rise of eBPF, which enables cloud-native developers to build secure networks, service grids, and multiple observable components, and which is also gradually penetrating and penetrating deeper into kernel components, providing more powerful kernel-state programmable interactions.

Wasm-bpf is a new open source project [1] that defines a set of abstractions for eBPF-related system interfaces and provides a corresponding set of development toolchains, libraries, and generic Wasm + eBPF runtime platform instances, giving applications in any Wasm virtual machine or Wasm lightweight container the ability to sink and extend usage scenarios to the kernel state, accessing almost all data in the kernel state and The eBPF runtime platform instance allows applications in any Wasm virtual machine or Wasm lightweight container to sink and expand their usage scenarios to the kernel state, access almost all data in the kernel state and user state, and achieve programmable control over the entire operating system in many aspects such as networking and security, thus greatly expanding the WebAssembly ecosystem in non-browser application scenarios.

  • 7 min read

如何在 Linux 显微镜（LMP）项目中开启 eBPF 之旅？

eBPF 为 Linux 内核提供了可扩展性，使开发人员能够对 Linux 内核进行编程，以便根据他们的业务需求快速构建智能的或丰富的功能。

我们的 LMP(Linux Microscope) 项目 是为了充分挖掘 ebpf 的可能性而建立的，项目以构建 eBPF 学习社区、成为 eBPF 工具集散地、孵化 eBPF 想法和项目为目标，正在大力建设中。之前我们在 LMP 其中的 eBPF Supermarket 中包含了大量由个人开发者编写的 eBPF 工具，覆盖了网络、性能分析、安全等多种功能，我们正在尝试把其中的一些程序迁移到 eBPF Hub，一些规范化的 eBPF 程序库，可以随时下载运行，或嵌入大型应用程序中作为插件使用。

我们尝试在 eBPF Hub 中，基于 eunomia-bpf 开发框架创建符合 OCI 标准的 Wasm 和 eBPF 程序，并利用 ORAS 简化扩展 LMP 的 eBPF 分发、加载、运行能力。

  • 8 min read

Running the ecli on Android 13

Author: CH3CHOHCH3

This article mainly records the author's exploration process, results, and encountered issues when testing the support level of the high version Android Kernel for CO-RE technology based on libbpf in the Android Studio Emulator. The testing method used is to build a Debian environment in the Android Shell environment and attempt to build the eunomia-bpf toolchain and run its test cases based on this environment.

  • 14 min read

在 WebAssembly 中编写 eBPF 程序和使用 libbpf

Authors: Yu Tong, Zheng Yusheng

eBPF (extended Berkeley Packet Filter) is a high-performance kernel virtual machine that runs in the kernel space and is used to collect system and network information. With the continuous development of computer technology, eBPF has become increasingly powerful and is used to build various efficient online diagnostic and tracing systems, as well as secure networks and service meshes.

WebAssembly (Wasm) was initially developed for browser security sandbox purposes. As of now, WebAssembly has evolved into a high-performance, cross-platform, and multi-language software sandbox environment for cloud-native software components. The lightweight nature of Wasm containers makes them suitable for running as the next-generation serverless platform runtime or for efficient execution in resource-constrained scenarios such as edge computing.

Now, with the help of the Wasm-bpf compilation toolchain and runtime, we can use Wasm to write eBPF programs as cross-platform modules, while using C/C++ or Rust to write Wasm programs. By using eBPF programs in WebAssembly, we not only enable Wasm applications to benefit from the high performance and access to system interfaces of eBPF, but also allow eBPF programs to leverage the sandboxing, flexibility, cross-platform nature, and dynamic loading of Wasm. Additionally, we can conveniently and quickly distribute and manage eBPF programs using Wasm OCI images. Combining these two technologies will provide a completely new development experience for the eBPF and Wasm ecosystems!

  • 5 min read

在 WebAssembly 中使用 Rust 编写 eBPF 程序并发布 OCI 镜像

作者：于桐，郑昱笙

eBPF（extended Berkeley Packet Filter）是一种高性能的内核虚拟机，可以运行在内核空间中，以收集系统和网络信息。随着计算机技术的不断发展，eBPF 的功能日益强大，并且已经成为各种效率高效的在线诊断和跟踪系统，以及构建安全的网络、服务网格的重要组成部分。

WebAssembly（Wasm）最初是以浏览器安全沙盒为目的开发的，发展到目前为止，WebAssembly 已经成为一个用于云原生软件组件的高性能、跨平台和多语言软件沙箱环境，Wasm 轻量级容器也非常适合作为下一代无服务器平台运行时，或在边缘计算等资源受限的场景高效执行。

现在，借助 Wasm-bpf 编译工具链和运行时，我们可以使用 Wasm 将 eBPF 程序编写为跨平台的模块，使用 C/C++ 和 Rust 编写程序。通过在 WebAssembly 中使用 eBPF 程序，我们不仅让 Wasm 应用获得 eBPF 的高性能、对系统接口的访问能力，还可以让 eBPF 程序享受到 Wasm 的沙箱、灵活性、跨平台性、和动态加载的能力，并且使用 Wasm 的 OCI 镜像来方便、快捷地分发和管理 eBPF 程序。例如，可以类似 docker 一样，从云端一行命令获取 Wasm 轻量级容器镜像，并运行任意 eBPF 程序。通过结合这两种技术，我们将会给 eBPF 和 Wasm 生态来一个全新的开发体验！