Project Overview
Modern datacenters and enterprise clusters increasingly incorporate programmable networking devices, such as reconfigurable switches, SmartNICs/DPUs/IPUs, and network-attached accelerators. Yet today’s distributed systems continue to treat the network merely as a communication channel, leaving its computation, memory, and coordination capabilities largely unexploited.
The dSys-PNF (Distributed Systems over Programmable Network Fabric) project treats programmable networks as first-class computing resources and co-designs distributed system software with programmable networking hardware. Rather than simply offloading packet processing, we redesign distributed systems around the capabilities of programmable network devices, enabling computation, caching, scheduling, synchronization, and data management to execute throughout the network fabric. Our research spans the entire software stack from programming models, compilers, and runtime systems to schedulers, operating systems, distributed services, and network functions. By integrating distributed systems with programmable networks, we demonstrate that the network can evolve from a passive communication medium into an active execution platform, greatly improving application performance, scalability, energy efficiency, and programmability.
Challenges
Programmable networking devices change where computation can execute. However, exploiting these devices is hard because developers must simultaneously reason about:
- Application decomposition
- Computation placement
- CPU/NIC/Switch/Accelerator coordination
- Communication costs
- Hardware heterogeneity
- Memory management
- Runtime orchestration/scheduling
- Hardware-specific optimizations
Existing software stacks expose programmable networking devices primarily as hardware accelerators, forcing programmers to manually partition applications, optimize data movement, and tune implementations through trial and error. As network bandwidth grows and networking hardware becomes increasingly programmable, this complexity intensifies.
Our Approach
Our insight is that elevating the hardware capabilities of programmable networks into application-native software abstractions makes efficient distributed systems possible. We ask: How should distributed systems be redesigned when programmable networking becomes an execution substrate? In this project, we have developed systems spanning multiple layers of the software stack, from programming models and runtime systems to operating systems and automated optimization tools.
- Programming abstractions that hide heterogeneous hardware details
- Compilers and runtimes that automatically partition computation
- Scheduling mechanisms that exploit programmable datapaths
- Distributed system frameworks that transparently integrate SmartNICs
- Operating system support that manages heterogeneous compute resources
- Automated analysis tools that guide hardware/software co-design
Here are the systems we have built.
- ASK provides generic in-network aggregation for key-value streams (ASPLOS’23).
- Xenic accelerates distributed transaction processing using an on-path SmartNIC (SOSP’21).
- Clara generates automated offloading insights for network functions on SmartNICs (SOSP’21).
- CalQ realizes the calendar queue abstraction for packet scheduling on RMT switches (NSDI’20).
- iPipe offloads actor-based distributed applications onto SmartNICs (SIGCOMM’19).
- E3 explores the energy efficiency benefits of running Microservices on SmartNICs (ATC’19).
- Floem is a data-flow based programming system for SmartNIC-accelerated applications (OSDI’18).
- AFQ approximates fair queueing mechanisms on RMT switches (NSDI’18).
- IncBricks is an in-network caching fabric with basic computing primitives (ASPLOS’17).