CS 736 Reviews - Spring 2017

Summary

Arrakis is an operating system which tries to eliminate the kernel from the data path to as great an extent as possible without compromising process isolation. For achieving this, the authors of this paper make use of the concepts of virtualization and provide a abstractions to the applications to directly access the hardware.

Problem

In networked operating systems, the I/O applications can suffer a lot of overhead due to the factors like Network Stack, Storage Stack and Application overheads and the overhead caused by Kernel crossing Kernel.
The authors use user space libraries to access hardware directly thereby preventing the intervention of kernel.
This improves performance significantly over traditional operating systems.
The authors seek to find and demonstrate if protection and high performance are contradictory in the context of operating systems.

Contributions

>The architecture of Arrakis is such that it provides an “ideal” set of hardware features - for multiple processes accessing a single physical device while making use of virtualizable hardware (Single-Root I/O Virtualization) for the same.
> Arrakis uses the abstractions of control plane and data plane.
> Kernel as control plane performs tasks like naming, associating each application to directories and files, transmit filtering, rate limiting. The data plane will involve multiplexing of packets based on filter expressions, directly to queues that can be managed entirely in user space.
> The virtualization for network devices is provided as virtual network interface cards (VNICs). The equivalent virtualization for storage is provided by virtual storage interface controllers (VSICs). Each of these have queues and rate limiters associated with them.
> The prototype of Arrakis is built on BarrelFish Library OS and a thorough performance evaluation and comparison is made.

Evaluation

Evaluation is done using four separate applications to compare the performance of Arrakis and Linux kernel and it is shown that the Arrakis can offer better throughput than Linux kernel. It is shown that User applications like Redis, Memcached being modified to use user space libraries to access hardware directly without any intervention by kernel results in improvement of performance significantly over Linux kernel.

Confusion

Working of Caladan is not very clear.

1. summary
The paper proposed Arrakis, an Operating System that removes the kernel from IO data path. They divided the kernel's function for IO into two planes, control plane and data plane, in which control plane manage naming and coarse-grained allocation, while data plane is put down to the hardware and contacted with application directly. This design will reduce the performance reduction from IO stack and copying data cross kernel, and expriments show that Arrakis can provide better throughput and latency than linux.
2. Problem
With the new hardware which should have improved the performace to perform IO, the main latency in IO path is in kernel, while enforcing the whole stack, copyies and checks. So removing kernel from IO path an apparent solution. Meanwhile, the user-level IO shoule be implemented without loss of isolation, protection, and security.
3. Contributions
1). The paper conducts the experiment to show the overhead of each step in the processing of package and storage. The table 1 and table 2 is interesting and clearly shows what we need to pay attention and what part can we improve.
2). They proposed the architecture to have two planes, control plan and data plane. It is a clear way to decouple responsibility and make it easier to add the features like resource allocation. 3). They implemented a prototype to shouw the potential benefits of user-level IO.
4. Evaluation
1). The paper evaluated Arrakis on four workloads to demonstrate the performance and scalibility gain of this design.
2). The improvement of performance can be expected, but I wonder that why did not they compare Arrakis with pure Barrelfish.
5. Confusion
1). Confused about so-called "Physical function" and "virtual function" and how they work.
2). I'm wondering the contribution of this work. It is a good and old idea, but with the work like U-Net, SR-IOV, and high-dependence on hardware that can virtualize itself, what makes it no-trivial? (Workload is heavy, but ...this counts?)

Summary: Improved hardware speed, and sluggish and sometimes unnecessary kernel calls motivated authors to build an OS (Arrakis) to explore the effects of removing kernel from the data path for most of I/O operations, without compromising protection or requiring modifications to existing applications compliant to POSIX API. Arrakis achieves this by virtualizing devices and giving its access directly to userspace and thus obviating kernel path.

Problem: Recent steep improvement in Ethernet speed and low latency persistent memories have outsmarted the kernel performance in I/O intensive applications, particularly in client-server computing model. Much of the execution time is spent in mundane OS tasks such as demultiplexing, security checks, context switch, and filesystem meta-data management. Arrakis eliminates (or minimizes) these OS overheads for most of I/O functions by making use of advances in hardware like Single Resource I/O Virtualization (SR-IOV) capable adapters.

Contributions: 1. Complete removal of kernel calls in some I/O paths and minimal kernel interaction in many other I/O intensive tasks.
2. Separate network stack for each application allows smoother transmission of packets, and lesser OS involvement such as context switches, and lock contention etc.
3. Usage of SR-IOV capable adapers as virtual devices (in form of virtual functions).
4. Packet filters and virtual storage areas (VSAs) provided by Virtual Network Interface Cards (VNICs) and Virtual Storage Interface Controller (VSICs) respectively.
5. Coarse-grains hardware protection mechanism for storage under the assumption that VSAs will be allocated in large chunks.
6. Separates file naming from implementation. Also, only files which are accessed by other applications need to be in kernel virtual file system.
7. Provides low level APIs for interacting with virtual network interfaces or virtual storage areas.

Evaluation: Authors first identify bottlenecks in OS tasks of client-server computing model applications and then compare those with native and POSIX version of Arrakis, each time running with a separate I/O intensive distributed application (such as Memcached Key Vale Store, Redis NoSql Store, HTTP Load balancer, and IP-layer middlebox). Both versions of Arrakis performed better than Linux in terms of latency and throughput. Native version was better than POSIX version, however latter is required for backward compatibility reasons. Authors have done a detailed evaluation. A perform test with non-IO intensive non network application to show that there is no performance degradation for other applications would have been perfect.

Confusion: 1. Who manages the control plane – kernel?
2. The paper doesn’t talk a lot about potential vulnerabilities. E.g. Who controls the software (Hypervisor) responsible for managing virtual functions? How secure is it?

1. Summary
The paper outlines an effort to extend the authors previously proposed approach of multikernels to network server operating systems with low latency IO devices. The authors motivate the need of such an approach. They also show how limited direct access to hardware abstractions can lead to significant gains in performance and dealing with low latency devices.
2. Problem
The current operating systems are not well prepared to handle low latency IO devices.
3. Contributions
- An working system which is reminiscent of exokernel approach.
- In my opinion a good approach to favor hardware abstractions and multiplexing over a generalized hypervisor like kernel abstraction, while still maintaining POSIX compliance.
- A good analysis of overheads in monolithic kernel abstractions(even when tuned for a particular task). This is further used as motivation for operating systems to move towards providing lower overhead interfaces to IO devices to enable their optimal use.
4. Evaluation
The paper does a good job in convincing the readers with an extensive evaluation using a representative set of benchmarks. But personally I feel they could have provided additional data while running the system as VMM host whilst managing multiple guests (including non similar kernels such as a tuned linux). Currently this is only left as discussion and simplified as a simple and natural extension the approach. As we all know the devil is in the details and VM guests are a very common workload configuration in their minimal target domain.
5. Confusion
The authors talk at length about the use of such an operating system and its usefulness in presence of fast IO devices such as NICS and fancy block devices with support of hardware virtualization and more features.
-> I would like to know would this approach still be worth it if such support was absent?
-> Also will it work for device classes such as a GP-GPU or a Xeon Phi like co-processor with/without hardware support ?

1. Summary
Arrakis is an operating system that optimizes for applications running on a machine in a datacenter. It allows applications to access virtualized hardware devices for networking and storage with minimal time in the kernel.

2. Problem
Servers that use Linux spend a large amount of time in the kernel for IO operations. Despite many optimizations, the Linux abstractions for networking and storage provide much lower latency and throughput than the raw hardware. This problem has been exacerbated in recent years with the development of extremely fast hardware that does not fit the traditional Linux abstractions. In particular, virtual network interface cards (VNIC's) can multiplex and demultiplex multiple queues of packets at the hardware level, and virtual storage interface controllers (VSIC's) do the same for storage.

3. Contributions
The authors implement Arrakis on top of Barrelfish, another OS. Arrakis provides several new abstractions that are useful in a datacenter environment. Each application receives events on VNIC's and VSIC's through doorbells, special IPC endpoints. VNIC's provide filters, which allow the application to transmit packets from a queue or receive packets into a queue that satisfy particular criteria. VSIC's provide virtual storage areas (VSA's), persistent storage that can be shared among multiple applications. Rate specifiers can be assigned to the queues for both VNIC's and VSIC's to limit ingoing and outgoing traffic. All of these abstractions are close to the underlying hardware, as desired.

Arrakis provides a light wrapper around networking hardware, which only performs multiplexing of packets, not processing. Most of the network stack is in the user space. In terms of storage, Arrakis provides low-level commands to manipulate hardware caches and VSA's, as well as higher-level constructs such as a filesystem and persistent data structures.

4. Evaluation
The authors implement Arrakis/P, which is POSIX-compliant, and Arrakis/N, which deviates from POSIX to achieve better performance. The paper contains a microbenchmark for packet processing, and several macrobenchmarks using real-world tools such as memcached and haproxy. In general, Arrakis/P outperforms Linux, and Arrakis/N outperforms Arrakis/P. The benchmarks were very thorough and show that Arrakis performs much better than Linux, given VNIC's and VSIC's.

5. Confusion
I was confused about several of the terms in the paper, such as VNIC, VSIC, and VIC. Are these terms used outside the paper? In addition, I do not fully understand the difference between the control plane and the data plane, and how different aspects of Arrakis are associated with one over the other.

Summary

The paper describes Arrakis Operating System which reexamines the role of kernel in I/O operations and removes it from the data path (Application level to/from I/O devices) in as many operations as possible.

Problem

Arrakis is targeted mainly towards network server operating systems. In these servers the time spent inside the OS code has become the main bottleneck when compared to performing the actual data transfer. This is due to the latest hardware advancements. Arrakis aims at giving applications direct access to virtualized I/O devices. The challenge is to do this without compromising on the various protection mechanisms provided by traditional OSs.

Arrakis claims to remove (or improve) various overheads involved during the I/O operations. Like the following: 1. Packet processing in the network stack 2. Kernel crossings 3. Scheduling 4. Copying data to/from kernel and user space.

Contributions

Arrakis assumes the underlying hardware can be presented to the user applications as multiple virtual instances of itself. Each of these instances have unique memory mapped register files, descriptor queues and interrupts. This ensures that the control plane can map each of these instances to unique protection domains. This leads protection across user applications.

The Arrakis control plane manages virtual device instances, and kernel involvement is minimized for data plane operations. Applications perform I/O via a user-level I/O stack library.
Arrakis introduces a hardware-independent layer for virtualized I/O which captures the functionality needed to implement in hardware the data plane operations of a traditional kernel.
To validate the approach on the storage side, given the limited support available from storage devices, the authors developed prototype VSIC support by dedicating a processor core to emulate the support expected from hardware

Evaluation

Arrakis was evaluated using a read-heavy memached workload, a write-heavy Redis workload, a web server farm workload with an HTTP load balancer, and an IP-layer ‘middlebox’ in the same configuration.
For memcached, Arrakis/P achieves 1.7x the throughput of Linux on one core, and attains near line-rate at 4 cores. Linux throughput plateaus at 2 cores. Arrakis/N gains an additional 10% throughput. It required 74 lines of code to be changed in memcached to port from POSIX to Arrakis/N.
Much of the performance improvement is attributed to the avoidance of cache misses and socket lock contention.

Confusion

The exact implementation of Caladan and its role in improving latency was not clear to me.

1.Summary
This paper talks about Arrakis, a new operating system which provides applications with direct access to virtualized I/O devices which leads to many performance gains. Arrakis uses advanced hardware I/O virtualization to provide security and protection.

2. Problem
In traditional operating systems, Kernel mediates access to device hardware by server applications which leads a lot of overheads. Earlier these overheads were not significant as compared to disk access times and data rates over networks. But the combination of low latency persistent memories and high speed internet has made these overheads significant.

3. Contributions
Authors have tried to split the role of kernel which in traditional operating systems provide both process isolation as well as network and disk security. Applications can directly perform I/O and network operations without kernel mediation. Modern hardware I/O virtualization (through VNICs and VSICs) is used to provide security and protection. Associated with each VIC are queues and rate limiters. Queues are used for user-space send and receive operations. Filters are predicates on header fields that the hardware use to determine whether to send/receive the packet or discard it. VSAs are used by the applications to store multiple sub-directories and files. Since all these components are present in hardware and does not involve kernel mediation, this leads efficient I/O and network operations.

4. Evaluation
Arrakis was implemented on top of Barrelfish operating system and was evaluated on four cloud application workloads which cover a lot of different use cases. Arrakis performed better as compared to linux on these workloads in terms of latency and throughput. Better performance on a variety of use cases puts a strong case for the claims made by the authors.

5. Confusion
I would like to know more about virtual storage areas and transmit/receive filters.

Summary
Arrakis is a new Operating System design to obviate the kernel intrusion in the I/O data path to achieve better performance without the compromise in isolation. Arrakis achieves this through advances in hardware (and with some software changes) which are capable of providing virtual instances. The authors have chosen Network and Persistent I/O for the detailed explanations and evaluations for the Operating System.

Problem
Having advances in hardware, which are capable for high performance, are under utilized due to the kernel intervention in data paths of Network and Persistence. From the table 1 in the paper, it is evident that Network stack, Kernel crossing, Copying and Scheduling, which are part of kernel, take most of the time in an I/O. In networking, the OS is responsible for (de)multiplexing the packets to different application; Because of this, lock contention and cache effects further reduce the performance.

Contributions
-> A new operating system, arrakis, which supports virtualized instances of hardware and provide unfiltered access to applications through libraries.
-> The authors demonstrate how user-level I/O benefits several applications like Redis, HTTP load balancer and IP middlebox.
-> Isolation is provided, without compromise on performance using transmit, receive and rate control filters (privileged operations).
-> Packets are directly delivered to user space; each application has an individual network stack where packets are delivered to the core where the application is running. Thus reducing cache invalidation.
-> Arrakis also provides a POSIX API support for application which can run unmodified. A native version of the arrakis APIs provide better performance including zero-copy I/O.
-> The virtual instances can be directly assigned to each of the guest operating systems like a regular physical devices without worrying about multiplexing the device. With hardware and its registers directly mapped to userspace, these guest OSes will perform better.
-> Arrakis uses IOMMU to provide the direct access of I/O devices to applications; virtual addresses generated by the application are mapped using this hardware.
->Arrakis provides doorbells, filters and rate specifiers as the abstractions to application to access control plane. Doorbell is an IPC end-point used to notify the events. Filter are used to check the applications for their behaviors and rate specifiers are used to throttle the traffic to balance the load.
-> The authors have developed two shared libraries that interface with the Virtual devices; Extaris for user-level network stack and Caladan for persistent library.

Evaluations
The authors have focused on evaluating real test cases that are practical and widely used like Redis persistent NoSQL sore, HTTP load balancer, Memcached Key-Value store and IP middlebox. Authors evaluate each of this use cases on tuned Linux and two version of Arrakis: POSIX and Native variants. Authors show that, in most of the cases, Arrakis P or N performing 2x or more than Linux. They also show how Arrakis scales with the increased CPU where as Linux doesn't (sometime lower performance due to overheads and cache effect). Authors provide detailed analysis of their OS and it gives enough quantitaive information to convince the reader of their claims. With providing protection and high performance, author prove that these two are not contradictory.

Questions / Confusions
1. What actually is a control plane ? How is it different from data plane ?

2. Filter creation capabilities are exposed to the application. Doesn't this compromise security ?

Summary
The paper proposed Arrakis, a new operating system that aims at reducing the overhead of process abstraction with a focus on modern client-server computing. With Arrakis, as stated by the paper, applications have direct access to virtualized I/O devices, allowing most I/O operations to skip the kernel entirely, while the kernel is re-engineered to provide network and disk protection without kernel median of every operation.

Problem
With networking and storage I/O speed increased dramatically, kernel overhead, including network stack costs, scheduler overhead, kernel crossings and copying of packet data, is starting to become the bottleneck of execution time performance.

Contribution
With the support of single-root I/O virtualization, Arrakis can have applications directly interact with virtualized I/O devices at the same time providing protection with IOMMU, eliminating the kernel overheads.

The design of Arrakis also does not require complex modifications on applications if they are written to the POSIX API. Arrakis is also flexible enough to support a wide range of I/O patterns.

A library of persistent data structures called Caladan was designed to take advantage of low-latency storage devices.

Evaluation
The authors did a solid job of explaining the Arrakis design with the paper and also actually implemented the system and evaluated its performance carefully. Many of the ideas embeded in the design like letting applications to interact with virtualized I/O devices directly, in my opinion, would inspired the community as operating system design progresses. However, one of my concern is that it seems the design of the system is very much geared towards networking. I was wondering whether does the performance of general computing also stand out?

Confusion
1. How does virtual file system (VFS) help data sharing?
Can you explain more on how does Caladan work and what are the benefits we can get from it?

Summary

The main idea of the paper is to provide a new abstraction which avails applications to access the low level hardwares without any kernel crossings improving the overall performance in terms of latency and throughput. This is achieved because of the ability of hardwares to be virtualised.

Problem

In current OSs all network and storage traffic is routed through the kernel, which requires a great deal of overhead in terms of context switches, copying data to and from the kernel and user space. This paper tries to solve this problem making kernel intervention as minimum as possible so that applications can use the low level hardwares to their fullest capabilities.

Contributions

The control plane part of OS manages virtual interface cards(VIC). Control plane handles applications’ requests to create/delete VIC, filters and allocate hardware resources and associate virtualised hardware with user applications.
User applications use user space libraries to access hardware directly without any intervention by kernel. This improves performance significantly over traditional operating systems.

Evaluation

Through a series of experiments, which involve read-heavy, write heavy load pattern, large number of HTTP requests made to web servers via an HTTP load balancer and IP-layer middle box, authors show that Arrakis offer noticeable improvements over Linux in terms of throughput across a variety parameters.

Confusion

Paper doesn’t differentiate very clearly the responsibilities of Data Plane and Control Plane, basically what functionalities of OS fall into Control plane and what into Data plane.

1. Summary
Arrakis is an operating system which revises the implementation and roles of the kernel in a way that nearly removes it from data path. In an Arrakis system, the I/O operations, mainly the network and file system operations are sped up by bypassing the kernel entirely or by keeping the kernel mediation minimal.
2. Problem
In today’s networked systems, due to the introduction of high speed internet and low latency storage devices, the demand for high performance has been rising. It is observed that a big chunk of program execution in a server is operating system code. One of the biggest challenges in systems design has been reducing this overhead of the operating system process abstraction. The authors revive a 20 year old idea of mapping the network hardware directly into user space. The paper is focussed on the implications of such an implementation where kernel is nearly removed from the data path.
3. Contributions
Arrakis provides an architecture where kernel involvement is minimal for most I/O operations while keeping the application design agnostic to these changes. One of the major building blocks in achieving this is Single-Root I/O virtualization. Using this technology each application can get its own virtualized device instance which is provided with unique memory mapped register files, descriptor queues, and interrupts.
The virtualization for network devices is provided as virtual network interface cards (VNICs). The equivalent virtualization for storage is provided by virtual storage interface controllers (VSICs). Each of these have queues and rate limiters associated with them. Additionally VNICs provide filters, which help in filtering out the packets or IPs and VSICs provide VSAs. VSAs are persistent segments for the VSICs.
Arrakis also introduces a new design principle for fie name lookup which is to segregate file naming from implementation. To leverage the advantages of low latency storage devices, Caladan, a library of persistent data structures is designed. Arrakis is implemented in two interfaces. One is Arrakis/P which is designed to be POSIX compatible and the other is Arrakis/N which is the native interface. The native interface achieves zero copy I/O.
4. Evaluation
The paper starts with detailed analysis of packet processing overhead in Arrakis and Linux. By examining the time taken at different layers during packet processing, the authors motivate the problem of the overhead of the operating system process abstraction. The paper uses Redis NoSQL as one of the main network services to measure and compare performance.
The paper also presents the results of running four standard cloud application workloads. With these experiments, the paper examines the major sources of performance overhead, the change in latency and throughout of Arrakisa and other performance gains of native Arrakis interface. The results are well supported by the results of empirical analysis on Linux and Arrakis systems.
5. Confusion
1. What is the fallback mechanism in Arrakis in case of any security violation by a rogue client?
2. Hypervisor software creates and deletes the virtual functions. How does this process run and avoid huge overheads? Where does it fall in the system design?

Summary:
This paper describes the design of a new network server operating system, Arrakis, that avoids kernel mediation for every interaction between application and device hardware by splitting the role of kernel into two. Arrakis targets I/O hardware which support virtualization and thus application can conduct I/O through their protected virtual device instance eliminating kernel intervention. Techniques like transmit filters and rate limiting enable Arrakis to provide the same process isolation and protection offered by traditional OS.
Problem:
With faster hardware available today, device I/O time has reduced significantly. Majority of the time spent (70%) in I/O operations is due to kernel overhead in performing tasks like demultiplexing, security checks, indirection at various layers of network stack, kernel crossing, context-switch etc. Even very simple functions like lookup and storage performed by application need to traverse the kernel multiple time per request. Arrakis fixes this removing kernel from I/O data path along with providing the necessary protection.
Contributions:
> The main idea of this paper is to allow applications to have direct access to virtualized I/O devices bypassing the kernel on most of the hardware access. This approach is close to Exokernel design.
> Achieves higher performance without compromising security. Transmit filters, rate limiters and virtualized hardware provide the necessary process isolation and security.
> Arrakis brings the separation of data plane and control plane. Kernel as control plane performs tasks like naming, associating each application to directories and files, transmit filtering, rate limiting. The data plane will multiplex/demultiplex packets based on complex filter expressions, directly to queues that can be managed entirely in user space.
> Arrakis provides POSIX compatibility which can allows applications to migrate without any modification.
Evaluation
One impressive approach adopted by the paper is presenting the test results of I/O intensive application, UDP echo server, which shows that about 70% of the time spent in packet processing is due to kernel. Arrakis is evaluated on four cloud application workloads: a read-heavy load pattern common in memcached distributed object caching system, a write-heavy load pattern to the Redis persistent NoSQL store, large number of individual client HTTP requests made to a farm of web servers via an HTTP load balancer and an IP-layer middlebox. All of these show significant throughput gain, upto 9x compared to linux.
Confusion:
Due to limited support from storage devices, they had to emulate the storage stack (VSIC). How expensive this would be, if hardware support is not provided in future. Is it reasonable?

1. Summary
The paper describes a new network server OS - Arrakis, designed to improve the efficiency of I/O intensive applications. The kernel is removed from the data path for all I/O and thus re-engineered to only provide network and disk protection. OS, thus, is primarily the control plane.

2. Problem
Despite high speed ethernet and low latency memories, I/O intensive applications still incur huge overhead because they need to traverse the kernel multiple times for each request. Most of the time is spent on execution of OS code such as delivering interrupts, multiplexing and demultiplexing packets. Furthermore, the advent of virtualization furthers provides an opportunity
to allow user applications to directly access virtualized I/O devices.

3. Contributions
The novel contribution of the paper is the design of new OS from data plane and control plane perspectives (typically considered in networks).
The design of Arrakis greatly leverages the virtualization support for I/O. Assuming networks devices are present as multiple Virtual Network Interface Card (VNIC) and storage controllers as Virtual Storage Interface Controller (VSIC), Arrakis uses their capabilities to remove the kernel from the data plane. Each VNIC can multiplex/demultiplex packets directly to queues managed directly in user space. Likewise, VSICs provide independent storage command queues. There is no need for kernel to intervene which thus takes upon the functionality of control plane. Arrakis’ control plane is used to request resources from the system and direct I/O flows from user programs.

4. Evaluation
Arrakis OS was implemented on fork of Barrelfish codebase. To justify Arrakis, authors use microbenchmarks to clearly point out the OS overheads in network workloads. Arrakis is evaluated on four cloud application workloads and shows improvements of 5x in latency and 9x in throughput as compared to Linux implementation.

5. Confusion
I am not totally clear about the exact demarcations between functionalities in data plane and the control plane. Could you explain in the context of Arrakis ?

1.Summary
Arrakis is a new operating system designed to keep the kernel intervention to minimum without compromising process isolation mainly in case of network client and server applications. The application code has direct access to the virtualized input and output devices and thus eliminating the overhead of executing the kernel code for every IO access. Arrakis performs much better than the Linux for variety of server workloads.

2. Problem
The applications spend significant amount of time executing the operating systems code. In case of network servers, 70% of the time is spent in processing the packets and also nearly 20% of time in copying the packet from kernel space to the user space and vice versa. Storage overloads also include parameter and access control checks, blocks and inode allocation, virtualization and metadata updates etc.

3. Contributions
-> Arrakis focuses on the hardware which implements virtualization in the hardware layer and presents its multiple instances to the upper layer. Example, NICs are exposed as VNICs, storage controller as VSICs. The control plane which is the kernel maps these virtual instances to a separate protection domains (applications/processes). Then the applications communicate directly with the hardware and thus the data plane is implemented in the application.
-> Each VIC is associated with queues and rate limiters. The filters (transmit/receive) act as a predicate on the network packet header fields and helps the hardware in deciding whether to send(receive) the packet in question.
-> Arrakis provides the applications the direct access to the virtual storage areas (VSAs) where application stores the files, directories and metadata. These can be exported to the kernel virtual file system (vfs) to make them visible to the other applications. VSA’s can also be mapped to multiple applications and thus can facilitate the sharing of data between the applications.
-> Arrakis handles the asynchronous notification of events using doorbells.
-> Arrakis also designed a persistent data structure called Caladan.

4. Evaluation
Arrakis has been evaluated on four cloud application workloads: the read heavy load pattern, write heavy load pattern to the Redis persistent NoSQL store, HTTP load balancer and the IP-layer middlebox. Arrakis gives lesser latency and greater throughout than the Linux under all the above mentioned workloads.

5. Confusion
Please elaborate on the implementation of Caladan in Arrakis.

1. Summary
The paper describes an operating system targeted at I/O intensive software in the datacenter that provides a new abstraction by allowing applications to have direct access to virtualized I/O devices skipping the kernel entirely. The kernel is designed to provide only network and disk protection without mediating every operation, thus providing significant performance improvement.

2. Problem
Server applications perform simple but I/O intensive operations which traverse the kernel on each request with significant overhead. With the emergence of high speed I/O devices and persistent memories with low latencies, it is important to improve the efficiency of these I/O operations by significantly reducing the kernel overheads.

3. Contributions
The paper describes two planes – the control plane whose functionality is implemented in the kernel and the data plane which involves interactions between applications and I/O devices. Arrakis tries to remove the overhead of kernel crossings by moving the control plane out of the path in the data plane, thus improving efficiency of I/O operations. It is possible to easily achieve this abstraction where applications directly interact with the hardware due to virtualizable hardware. Significant improvements are shown over the Linux kernel while maintaining POSIX compliance, which can be dropped for further performance improvements.

4. Evaluation
The paper makes a strong case for the ideas it proposes by detailed profiling of overheads in the Linux kernel and comparing them to those in Arrakis. It uses four applications that are representative of server workloads and I/O intensive for evaluating performance. But Arrakis seems to significantly depend on abstractions provided by hardware. It is not clear how the system would perform when there are a large number of applications which want to use the hardware directly.

5. Confusion
The ideas in this paper are enabled by technology improvements in I/O hardware. Are there other domains that could leverage better hardware and use improved abstractions for performance benefits?

1. summary
This paper describes a system to allow direct access to IO devices such as the network card and storage devices without kernel intervention.
2. Problem
Requiring that all network and storage traffic be routed through the kernel requires a great deal of overhead in a system. Context switches and copying data to/from kernel to user space memory occupies much of the time that is required for this traffic. A system that allows the kernel to be bypassed for this traffic could effectively eliminate much of this overhead.
3. Contributions
A method of mapping IO devices into the memory space of user programs is demonstrated in this paper. It takes advantage of hardware virtualization capabilities already available in many NICs and other devices to present direct access to user programs using the interface without requiring kernel intervention outside of initial allocation. Instead libraries can directly utilize the IO functions using the virtual NICs while the NIC can similarly deliver traffic to the proper program’s traffic queue. Similar functionality is provided for storage elements to allow storage to also be able to bypass kernel overheads. This is all implemented on top of both the POSIX interface as well as a native interface. The POSIX interface allows for software compatibility but lacks the extra speedup factors of the native interface.
4. Evaluation
To evaluate the system, they fully implemented it on top of the Barrelfish operating system. Through a series of experiments, they show it to offer noticeable improvements across a variety of network use cases. The clear improvements across a variety of usage cases on top of an actual functioning implementation lends a large amount of credibility to the system. They can clearly show that their system shows improvement by removing the kernel overheads they had initially thought to be problematic.
5. Confusion
What would happen when the limit on supported VNICs was reached? Would a program attempting to access the network just not work or does it fall back to kernel intervention?

Side note: I was disappointed they didn’t have any Dune references hidden in the paper – that I noticed at least.

1. Summary
The paper presents the operating system Arrakis which seeks to remove a lot of the overhead of kernel mediation for I/O operation. The paper reviews the methods on which it adds the virtualized necessary to remove the kernel mediation and evaluates the system compared to a well-tuned Linux system.

2. Problem
A large amount of time is spent executing operating system code which represents large amounts of overhead with I/O operations. This is particularly bad when looking at server systems which have large amount of I/O operations.

3. Contributions
Although not overly covered in the paper there is significant overhead if incoming messages are delivered to different CPUs than the one on which the user-level process is scheduled but with Arrakis packets are delivered directly to the user space so this overhead is eliminated. Arrakis removes most kernel mediation of I/O operations instead setting up virtualized I/O devices. These virtualized I/O devices are directly accessible by applications. With packets being delivered directly to the user-space according to the paper this removes what took 10% of the time on a regular Linux system. Arrakis achieves this virtualization by using multiple virtual network interface cards (VNIC) and virtual storage interface controllers (VSIC). These VNICs have multiple DMA queues inside so they can directly send and receive from the user-space bypassing the CPU. A strong point of the system is that it provides these speedups while still supporting the POSIX API and only if you want some of the further improvements that the system offers do you need to utilized the Arrakis’s native API. This would seem to be a strong selling point for the system as its compatible with existing applications.

4. Evaluation
The system is actually implemented using existing hardware and by modifying the Barrelfish OS. The paper uses four separate workloads to test out the system and looked at the improvements of latency and throughput that Arrakis provides. When looking at packet processing the overhead is reduced to almost half what the linux system overhead was and with the Arrakis’s native interface this overhead is a tenth of what the linux system has.

5. Confusion
I was confused as to how secure this system could still be if the kernel usually handles security. The paper mentions transmit and receive filters but does not elaborate on how they are used.

1. Summary
Arrakis is the continuation of the novel idea of moving the network stack into userspace presented in U-Net. They modernize the idea with hardware previously not available and add insight to how to take advantage of other advancements such as NVM.

2. Problem
The client-server computing model has become a staple in modern computing. Efficient use of this model requires incredibly low latency for high packet rates. The researchers wanted to explore how we can better support lower latency network communication and expanded it to storage as well.

3. Contribution
The contributions here are primarily adapting the previous idea of U-Net to the modern era of computing (SR-IOV, VT-d) and studying the latency of the network stack. They implemented a prototype operating system on top of Barrelfish to remove the kernel from the network stack and extended the idea to the storage stack. The insight they provide for the requirements to do this with storage is important as it lays the foundation for creating this hardware. One of the important considerations is how an application can manage it’s own memory layout and expose this to other applications using the VFS.

4. Evaluation
The paper evaluates Arrakis using the POSIX API and a native interface, true zero-copy, against the linux kernel. As shown in Table 1, a significant reduction in processing overhead is demonstrated. They additionally evaluated the solution on memcached and redis showing significant throughput gains as would be expected with operations on in-memory data structures. There is a question to why they did not present any barrelfish statistics pre-Arrakis and only compare against the monolithic kernel.

5. Discussion
It’s interesting to see how software is evolving with hardware support. U-Net was ahead of its time, because clearly the idea survived and this paper can serve as validation for them. I’m curious to see how we continue to move towards are more cooperative design between software and hardware.

1. summary
The paper claims that OS abstraction provided by kernel for network and IO operations bears considerable overhead and complexity. The paper claims that the contributions from the paper enable fast and customizable IO/Networking user mode abstractions without comprising system protection unlike previous related work.
2. Problem
Current OS provides network and IO abstractions via system calls exposed by kernel. Hence, every such call requires kernel mediation. This mediation incurs additional performance overhead and complexity. As a result, hardware (HDD, NIC) cannot be used up to its full potential.
3. Contributions
This paper proposes to perform I/O and Network Operations in user mode library instead of kernel mode. The paper claims doing so reduces the overhead imposed by kernel and complexity. However, hardware support is required to achieve user-mode networking and IO operations. The authors have designed OS called Arrakis for benchmarking this approach. The Arrakis requires virtualizing NICs(VNIC) and storage controllers(VSIC). Each of these VICs can be interfaced via user-mode libraries bypassing kernel invocations. These VIC provide functionalities like queues, filters, QoS provided by kernel stack. The user-mode IO/Network stack can perform these functions as well depending upon the hardware features. The Arrakis control plane manages these VICs instead of kernel. An interface via shared library is provided to interact with Control Plane and demand resources. The VSIC provide Virtual Storage Areas to applications for storage abstractions. The Arrakis provides interface for Network and IO operations with virtualized interrupts for event handling. The Arrakis provides two interfaces – Native APIs and POSIX APIs for network operations. The POSIX APIs work with existing applications while native APIs need tweaking. The IO interface provides POSIX compliant APIs.
4. Evaluation
The authors have described exhaustive evaluation of the Arrakis and its performance comparison with Linux IO stacks. In almost all benchmarks, Arrakis’s APIs seem to outperform Linux’s IO stack performance. The Arrakis native interface seems to be better than posix compliant Network APIs. The paper has described experiments evaluating performance comparison for Memcached, Redis, HTTP Load balancer, IP Layer Middlebox. The results clearly show that Arrakis’s IO and network interface perform better than existing Linux IO and networking stack.
5. Confusion
What are the benefits of Virtual Storage Areas?
Scalability of Control Plane as a service?
The API seem to be asynchronous, how to get acknowledgement of operation completion?

1. summary
The paper claims that OS abstraction provided by kernel for network and IO operations bears considerable overhead and complexity. The paper claims that the contributions from the paper enable fast and customizable IO/Networking user mode abstractions without comprising system protection unlike previous related work.
2. Problem
Current OS provides network and IO abstractions via system calls exposed by kernel. Hence, every such call requires kernel mediation. This mediation incurs additional performance overhead and complexity. As a result, hardware (HDD, NIC) cannot be used up to its full potential.
3. Contributions
This paper proposes to perform I/O and Network Operations in user mode library instead of kernel mode. The paper claims doing so reduces the overhead imposed by kernel and complexity. However, hardware support is required to achieve user-mode networking and IO operations. The authors have designed OS called Arrakis for benchmarking this approach. The Arrakis requires virtualizing NICs(VNIC) and storage controllers(VSIC). Each of these VICs can be interfaced via user-mode libraries bypassing kernel invocations. These VIC provide functionalities like queues, filters, QoS provided by kernel stack. The user-mode IO/Network stack can perform these functions as well depending upon the hardware features. The Arrakis control plane manages these VICs instead of kernel. An interface via shared library is provided to interact with Control Plane and demand resources. The VSIC provide Virtual Storage Areas to applications for storage abstractions. The Arrakis provides interface for Network and IO operations with virtualized interrupts for event handling. The Arrakis provides two interfaces – Native APIs and POSIX APIs for network operations. The POSIX APIs work with existing applications while native APIs need tweaking. The IO interface provides POSIX compliant APIs.
4. Evaluation
The authors have described exhaustive evaluation of the Arrakis and its performance comparison with Linux IO stacks. In almost all benchmarks, Arrakis’s APIs seem to outperform Linux’s IO stack performance. The Arrakis native interface seems to be better than posix compliant Network APIs. The paper has described experiments evaluating performance comparison for Memcached, Redis, HTTP Load balancer, IP Layer Middlebox. The results clearly show that Arrakis’s IO and network interface perform better than existing Linux IO and networking stack.
5. Confusion
What are the benefits of Virtual Storage Areas?
Scalability of Control Plane as a service?
The API seem to be asynchronous

1.Summary
This paper is introduces the Arrakis operating system that separates the data plane and control plane for most IO operations, allows most IO operations to skip the kernel entirely. Thus, this system achieves both good throughput and latency compared to traditional linux system.

2.Problem
The performance of network and storage system improves a lot due to new hardware technology, so, OS kernel, which serves as a mediation, introduces huge overhead for IO operation, like multiple data copy and managing software multiplex. This problem is especially severe for IO intensive services such as http server or cache server. Existing research mainly focus on optimizing OS kernel code, but there are some new hardware that can provide virtualized interface and multiplex data flows similar to OS kernel. So, the author want to utilize these new hardware features to solve the poor IO performance of linux system.

3.Contributions
The key idea of this paper is, OS should just be a control plane that setup and direct the IO flows, data transfer should bypass OS kernel and happens directly between virtualized hardware and user level code. I think the idea is similar to the idea of Exokernel.
In order to do this, hardware are virtualized and presented as virtual interface cards(VIC) with private command queues and filters. This almost provides a hardware implementation of data operations previously provided by kernel.
Kernel does not provide IO functions like read or write, instead, user level libraries like user level network stack or user level storage API provide commands that can manipulate corresponding VIC.
The IO part of the OS kernel becomes a control plane interface, it takes the application’s request about create or delete VIC, create or update filter. OS allocates hardware resources, and associate virtualized hardware with user application. Then, user applications use user-space libraries to communicate directly with hardware.

4.Evaluation
Arrakis is implemented based on Barrelfish OS, Arrakis/P uses POSIX interface, and Arrakis/N is the native interface that supports true zero copy. All the experiments are in multicore environment, and focus on IO intensive benchmarks to test network or storage performance. In almost all cases, Arrakis is 2 or more times faster than Linux, and it performs really well on network services when data processing time is short and IO time is dominant. Using Arrakis/N API, the zero copy method can push Arrakis to almost the maximum throughput of driver.

One question about the evaluation section is, this paper compares Arrakis with Linux system, not Barrelfish, so you can not tell whether the good performance comes from this new IO design, or comes from the Barrelfish. But personally I think it is because of the new IO design.

5.Problem
How to guarantee protection and safety without OS? Like in the virtualized hardware, if one command queue has problem, will it affect other queues or other section of this system? Who provides the protection?

1. summary

This paper presents Arrakis, a modern implementation of an old idea which decouples the control and data plane of a computing system. It is particularly aimed for the server systems. The basic idea is to take the kernel away from the datapath of the the applications into a separate control plane and provide applications a direct access to the I/O devices (The paper considers the storage disk and network card). Most of the OS services are provided as a library service to the applications while the kernel services are available through a separate control plane interface. The authors have implemented Arrakis by extending the open source barrerlfish operating systems and their evaluation shows significant improvement in performance for 5 server side applications.

2. Problem
The need for the faster OS arises because due to advancement in I/O technology, both the network and storage have become very fast. Consequently, most I/O intensive applications end up spending majority of their time traversing the kernel space. The authors found that for Redis the time spent in the kernel for read and write operations was more than 60% of the total operation time making the kernel traversal a bottleneck for application performance.

3. Contributions
Motivated by the above problems the basic principle behind the Arrakis design is to skip the kernel for read and write operations. This is enabled by two major components.
First the Single-Root I/O Virtualization (SR-IOV) which is a hardware technology that which can dynamically create virtual instances of one underlying physical IO device. Each of these instances is exposed to a user application which talk to it directly instead of through the kernel. Some of the functionalities of the Kernel e.g., protection, multiplexing and I/O scheduling are now implemented directly inside this virtual I/O device. The second major component of Arrakis is the library OS that now runs with the application. It can provide a standard (POSIX compliant) interface to the applications and allows them to talk directly to the hardware through its southbound interface. This library can be further tuned for every application making more optimizations possible. The authors have also implemented a persistent datastore library Caladan.
In addition the authors also discuss a hardware model which describes the set of idea features in the hardware for an Arrakis like Operating System.

4. Evaluation
The evaluation of this paper are two fold. First the authors benchmark the kernel overhead for the server side applications e.g., Redis and UDP echo server etc. motivating the need for kernel bypass. The second is the performance evaluation of 5 server applications. The throughput results show that Arrakis almost always performs better than Linux and also scales much better with the number of cores. The linux flattens off pretty fast because with the increase of number of cores, the overhead of locking/synchronization increases pretty fast resulting in a lot of cycles being spent there.

4. Confusion
I didn't quite understand how are the files being shared between different programs. In the Control Plane Interface section it says that "In Arrakis, a single filter acn safely express both a listening socket and all subsequent connections to that socket, allowing server side tcp connection establishment to avoid kernel mediation". What does that mean?

1. Summary
Arrakis provides applications direct access to virtualized I/O devices (network and storage) and thus brings their performance close to the hardware performance. It relies on the recent advances in hardware virtualization such as SR-IOV to provide multiplexing and security and thus eliminates the kernel from the data path entirely.

2. Problem
Historically, the kernel overheads have been insignificant to the latencies associated with hardware I/O devices. However recent hardware trends have drastically reduced these latencies and made these kernel overheads the biggest barrier to improved application performance. With the arrival of advanced hardware virtualization techniques such as SR-IOV, Arrakis proposes that we can get rid of the current abstractions in the kernel related to NICs and storage I/O, and provide the applications direct access to the virtualized I/O devices.

3. Contributions
Through its use of SR-IOV, IOMMU and other adapters, Arrakis is able to provide applications direct access to virtualized I/O devices called Virtual Interface Cards (VICs) and eliminate expensive overheads associated with the kernel. For each of these virtualized device instances, the physical device provides unique register files, descriptor queues and interrupts, allowing separate protection domains.

Arrakis is then able to successfully remove the kernel from the data path by offloading some of the functionality traditionally performed by the kernel to the hardware. Multiplexing and demultiplexing is performed through queues and rate limiters functionality present in VICs. The enforcing of security checks is done through.

Arrakis provides two network data plane interfaces - a native interface which supports zero-copy I/O and a POSIX compatibility interface, allowing legacy applications to run. It also provides a storage data plane interface.

4. Evaluation
The authors first perform a series of microbenchmarks to determine the breakdown of OS overheads associated with network transfer and storage I/O latencies, to make the case for Arrakis. Then they run the same micro benchmarks on Arrakis, and are able to demonstrate improvements of 2-5x in latency.

They also use a wide variety of macro-benchmarks — several cloud applications. This is a fair representative of the type of applications that will run on Arrakis since it seems to target commodity servers in a datacenter. The results also show that the throughput achieved in Arrakis/Native is much better than that of Linux and very close to the hardware performance.

5. Confusion
Can you explain how Caladan works?

Summary:
The paper presents a new operating system designed to remove the kernel from I/O data path without compromising process isolation.
Arrakis uses device hardware to deliver I/O directly to user-level library, thus allows applications to skip the kernel and have direct access to virtualized I/O devices.

Problem:
Earlier due to slow network and storage, kernel overheads did not dominate the total I/O execution time. With the advent of high speed Ethernet and low latency persistent memories, I/O intensive applications spend most of the time executing operating system code. The goal of this paper is to remove these kernel overheads without impacting the protection and provide I/O performance closer to achievable by raw hardware.

Contributions:
Arrakis takes advantage of I/O hardware with support for virtualization. Arrakis assumes the availability of network cards that allow creation of multiple virtual network interface cards (VNICs) that each may have multiple send and receive queues and storage controllers that expose multiple virtual storage interface controllers that each can provide independent storage command queues.
The Arrakis control plane manages these virtual device instances.
Applications perform I/O through their protected virtual device instance without requiring the kernel intervention. Applications rely on a user-level I/O stack library to perform these operations.

Evaluation:
The authors extended Barrelfish multicore OS code base to implement Arrakis. They used four cloud based applications for evaluation. The results are compared with linux kernel. Since application is in control of the whole packet processing flow, it avoids locks contention, cache misses, kernel overheads etc. Hence much better performance compared to linux. Arrakis native interface implementation is claimed to provide even better performance than Arrakis with POSIX API interface. But the native interface implementation does not seem very useful due to backward compatibility and portability issues.

Confusion:
Could you explain how are directories and files being shared between processes in Arrakis, especially when the owner process is not active or dead and some other process wants to access those files?
They claim that with kernel bypass, a remote cache miss and a remote event delivery are similar in cost at a physical level. I don’t understand how would this be possible?

1) Summary

Improving hardware I/O device speeds and hardware support for virtualized devices is making new I/O models possible and necessary. The Linux kernel I/O stack contains a great deal of overhead which decreases the benefits of this new hardware. Arrakis uses new hardware abilities to remove the majority of this overhead by giving userspace access to virtualized devices directly.

2) Problem

I/O stacks contain a lot of overhead. With improving device speeds, the kernel tends to become a performance bottleneck in I/O intensive workloads. In particular, the authors look at network server workloads, such as those found in the cloud.

Networking stacks tend to contain a great deal of multiplexing and demultiplexing overhead. Moreover, if a process receives too many interrupts due to network events, performance can be degraded.

Likewise, storage stacks can introduce a great deal of overhead. Meanwhile, many server workloads such as databases do not want the file abstractions offered by the OS, instead building their own formats inside files.

Thus, the added overhead of the kernel for these workloads does not improve programability. It is simply dead weight. The only feature these applications want from the kernel is protection and resource allocation.

3) Contributions

The main contribution of this paper is a new use for virtualized devices. The authors observe that VNICs already offer isolation and multiplexing functionalities. By using the capabilities of the hardware, the kernel can be moved out of the user's way. The authors propose that the same could be done for storage devices, as these devices already contain logical addressing modes. The user can be given direct access to a virtual device. The device will enforce isolation, so the kernel does not need to intervene. This improves performance significantly over traditional operating systems, such as Linux. The result is Arrakis, an operating system optimized for cloud applications, which tend to be heavy on storage and network workloads.

4) Evaluation

The authors do an excellent job motivating the problem and specifying the domain for which their solution is designed. They profile the overhead of the storage and network stacks in Linux in detail, and they show where this overhead has been reduced or eliminated in Arrakis. In their evaluations, they use a series of micro- and macro-benchmarks that are representative of common server workloads.

However, one workload that is conspicuously missing from their set of benchmarks is a virtualization benchmark. The cloud is built with virtual machines, so one would expect Arrakis to be evaluated as a host OS for a group of VMs. The results of the paper are not as complete as they could be (though, in my opinion, they are still convincing).

Another weakness is the rather important assumption of advanced and non-existant hardware; Arrakis assumes the feasibility of a great number of advanced hardware capabilities to achieve its goals of high performance. In fact, all of Arrakis's performance benefits hinge on the validity of this assumption. The fact that some of the hardware they assume (e.g. VSICs) does not even exist seems to suggest that the design is slightly fragile.

5) Confusion

I did not completely understand how the file naming system works in Arrakis. In particular, how are files exported from a custom file system that process manipulates directly through a VSIC?

1. summary
This paper introduces the Arrakis network server operating system. In Arrakis, applications have direct access to virtualized network devices and storage thus skipping the kernel to improve performance, in terms of both latency and throughput compared with well-tuned Linux. Security in Arrakis is also guaranteed by making virtualized resource assignment a privileged operation controlled by OS.

2. Problem
Servers spend significant amount of time executing operating system codes and this overhead has become more salient with the high speed Ethernet and non-volatile low latency storage. For example, in the UDP echo server experiment in the paper, Linux spent 70% time in the network stack, and using a system level Linux stack also causes more cache and lock contention since the network response may be scheduled to run on a different processor from the user level application. In the fsync experiment, OS storage stack also causes huge overheads.

3. Contributions
Arrakis proposes a method to grant user level processes access to network and storage through resource virtualization and provides corresponding abstract models for hardware virtualization. Network device is virtualized as virtual network interface card (VNIC) which implements data network processing data structures (originally in system memory in Linux) in user space memory and uses transmit and receive filters to restrict the nodes the process can talk to. Storage is virtualized as virtual storage interface card (VSIC), which is like implementing segmented virtual memory for storage. VSA also changes the way storage is managed. In Arrakis, each process can organize its storage (VSAs) in its own format, and other processes need to talk to the process that owns the storage to access them, or through Arrakis application library.

Authors implemented Arrakis based on Barrelfish OS and also implemented drivers for several commercial devices including Ethernet Adaptor, IOMMU and RAID controllers. They also develop a mechanism to emulate VSIC on hardware with limited virtualization support be dedicating a core to run the VSIC functionality. Apart from Arrakis interface, authors implement POSIX interface which allows programs to run without modifications, though lowering performance improvements

4. Evaluation
Authors evaluated Arrakis using four cloud application workloads, including memcached key value store, Redis NoSQL store, HTTP load balancer and IP-layer Middlebox. For all these benchmarks, Arrakis outperformed Linux in both latency and throughput and was proved to be more scalable. Arrakis native interface achieved much higher performance improvement than POSIX interface.

5. Confusion
Since the storage is organized by process, how does Arrakis access the storage created by the process when the process is deleted from the machine? Does the library need to bookkeep the storage format of every process ever run on the machine?

How does Caladan work? Why can it outperform Linux?

1. Summary
This paper introduces a new operating system designed to remove OS overhead in I/O operations. The key idea is to use virtualized hardware and move most of I/O code from kernel to user-level library. Protection and QoS can be enforced via feature such as filter and rate limiter.

2. Problem
The problem arises from the unsatisfactory overhead of the operating system for I/O intensive software. Many I/O intensive software (such as HTTP server) spend much of time executing operating system code including delivering interrupt, copying network packets, maintaining file system meta data and so on. Even though much of research has been devoted to optimize kernel code for I/O operations, the one of state-of-the-art operating systems - Linux still have much worse latency and throughput for network and file system stacks compared to raw hardware. With the appearance of virtualized hardware device, the authors can implement a new I/O stack (both for network and file system) with most of code in user-level based on these new hardware.

3. Contributions
The main contribution of this paper is designing a model for user-level I/O stack OS on virtualized hardware device, and conduct 4 kinds different cloud application workloads to demonstrate its efficiency.

The hardware under this OS has virtual interface card (VIC), consisting of queues, filter, rate limiter and so on. Application directly communicates with underlying virtual interface card to conduct data transfer without OS intervention. The data transfer is achieved via DMA queues between application's memory and hardware. The queue can be implemented as circular buffer with packet descriptors. Notification of events such as I/O completion is implemented via hardware virtualized interrupt (with name of doorbells in paper). I/O operations such as read and write data can be more efficient than traditional OS design, because traditional OS has overhead such as copying packet data from user memory to kernel, and OS context switch between kernel and user processes. Filters can be used to determine whether to send/receive the packet based on packet header. For example, filters can be useful to protect web servers from processing too many malicious connection request attacks. Traffic control is implemented through rate limiter. Without OS processing overhead, QoS can be implemented with hardware support in a more efficient way. Only certain critical operations such as creating and deleting VIC, installing doorbell and filter are privileged, which allows user application to gain performance as much as possible with little OS intervention.

4. Evaluation
This paper implemented a prototype by extending a multicore OS (Barrelfish). The authors implemented user-level network stack (Extaris) with POSIX API and its native API (asynchronous I/O), based on lwIP. To demonstrate its performance, the authors compare their prototype with Linux for 4 different benchmark workloads. Workloads consist of read-heavy load on memcached, write-heavy load on Redis, large number of client requests to HTTP load balancer and IP-layer middlebox. The result shows prototype performance is generally 2x better than Linux in terms of both latency and throughput, both in one and multiple cores. Prototype with native API is better than that with POSIX API. The result is promising because prototype with POSIX API gains much better performance than Linux without changing any application's code.

5. Confusion
I am curious about how hardware I/O virtualization actually works?

In evaluation, prototype with native API (asynchronous I/O) is better than the one with POSIX API. Does it mean asynchronous I/O is better than synchronous I/O. What advantages/disadvantages does it have?

1. Summary
Arrakis improves system performance for network servers by allowing applications to directly access virtualized I/O devices (disk and network) with only minimal kernel intervention. By using virtualized hardware, Arrakis still provides adequate security and isolation to individual processes.

2. Problem
In the past, much I/O optimization (particularly in Linux) has focused on improving the code inside the kernel itself. This ignores one big issue: the network and file system stacks in Linux are contained inside the kernel. If the interfaces could be brought into the user space and used without involving the kernel at all, a great speedup could be achieved. This is especially important in the context of today’s increasingly performant flash-based storage systems.

3. Contributions
This paper shows the exokernel idea is practical and makes a lot of sense in certain contexts. It demonstrates how hardware changes intended for virtual machines can be used in native OSs to bypass the kernel while still providing safety. It also suggests modifications for disk controller that would be useful for exokernels to leverage them effectively.

4. Evaluation
The authors implement their changes on top of the Barrelfish multicore OS. They extend Barrelfish to include the needed user libraries for the Network and Disk interface. They leverage network hardware with virtualization designed to support virtual machines. They emulate a disk controller with virtualization by running code to emulate the controller on a separate dedicated core connected to a RAID system.

To evaluate performance, they run several real and synthetic workloads. Arrakis performs best on workloads that require little processing of data and mostly calls into the network stack; this is true of their “echo” benchmark and the http-middlebox application. This is because Arrakis avoids the overhead of the syscalls and performs most of the functionality in the user space.

They also see performance due to natural packet steering: since each process has its own private network interface, packets sent to a particular application always arrive at the correct core. This is in contrast to Linux, which must first receive the packet and then steer it to its correct core. This allows the Arrakis applications to scale near-linearly with the number of cores available.

Finally, they see a pretty crazy increase in write performance on a noSQL store due to their private file systems and virtualized disk interfaces.

5. Confusion
I don’t understand how their persistent data storage library (Caladan) performs better than a conventional log-based FS; they sound very similar.

1. summary
This paper presents a system, Arrakis for network server. Arrakis removes the kernel mediation in the traditional systems. User-level applications can directly access the hardware with protection provided by a control plane. The removal of kernel mediation have gains in both latency and throughput compared with the Linux system.

2. Problem
The process abstraction and kernel mediation of operating systems introduce much overhead. The high speed Ethernet and low latency persisent memories has made the problem more salient, especially for client-server computing. The overhead mainly comes from delivering interrupts, demultiplexing and copying network packets, and maintaing file system meta-data. Many efforts have been made to optimize the kernel but the problem has not been fundamentally solved. Now the Linux system still
has a much worse latency and throughput than direct access of the raw hardware.

3. Contributions
1. The Arrakis restructs the hardware, kernel and applications. The authors first research the overheads of network stacks and find many overheads are due to the kernel mediation. Thus the authors replace the kernel mediation with direct access of hardware devices in Arrakis. Arrakis also supports the protection as the kernel. It maps the hardware devices information (register files, decriptor queues etc.) to unique memory so that the control plane can map the device to a seperate protection domain. 2. The authors build the prototype of Arrakis on Barrelfish OS. 3. The
authors do a comprehensive evaluation, measuring the latency and throughput in four cases and show the improvements of Arrakis.

4. Evaluation
The authors compare Arrakis and Linux under four cloud application workloads. The first is a read-heavy load pattern (memcached key-value store). The sceond is a write-heavy load pattern (Redis NoSQL store). The last two are a large number of individual client HTTPs request via an HTTP load balaner and an IP-layer middlebox. In all cases the Arrakis has a 1.5x-3x throughputs of Linux. The comparison is in favor of Linux thus we can conclude that Arrakis outperforms Linux in several network cases.

5. Confusion
In table 1, the Arrakis does not have a scheduler, but what alternative way Arrakis provides for scheduling?
What is the use of VSA? There are two abstraction layers in storage, VSIC, VSA? Why not only use VSIC as VNIC in network?

1. Summary
Traditional Operating system are too hands on. They try and monitor every bit of I/O. With a growing demand for sever/client type of I/O the operating systems mediation is slowing things down. These folks have come up with an OS known as Arrakis that doesn't slow things down so much, by not taking part in it all. Now most of the I/O work is done in the Userspace, with user programs using a library to communicate with the hardware directly. They claim that with minor tweaks to hardware and software they can still preserve the security traditional operating systems provide.

2. Problem
In short- I/O takes too long over the network and it's not because the ethernet cable is slow or disk IO is terrible. The CPU overhead cause by the OS handling all the I/O is causing this slow down.

I/O delay is categorised into 4 categories:
* Packet processsing at Hardware layer
* OS scheduling delay
* Switching to and fro between user and kernel mode
* Data copy (I don't think Arrakis gets rid of this)

Arrakis provides performance boosts by removing the first two delays.

3. Contributions
Made their own OS - Arrakis which achieves the following abstractions:

*) Minimize kernel involvement for data-plane operations. Remove kernel mediation for most I/O. I/O is routed from applications address space without compromising security and isolation properties.

*) The traditional read/write code was in the kernel, now it lies in user space in specialised libraries. The control plane plays a role in this read/write but i have no idea how(refer confusion 2)

*) Done by hardware that can present multiple instances of itself to the operating system and the applications on computer. So the the users see multiple versions of the network card and multiple versions of storage even though there is only one. And it seems like the OS doesn't handle any of this. (Refer: Confusion number 1). The applications access these guys through multiple Virtual controllers (which are Queues it seems)

*) Traditional filesystem look up files by their full path, which means internally jumping through multiple levels of inode indirection. Instead, Arrakis provides applications direct control over VSA storage allocation: an application is free to use its VSA to store metadata, directories, and file data. The library exports a file server and everyone queries this server to get files.

*) Just like files, the network I/O which was previously done by the OS, is now done by a library.

4. Evaluation
I don't really want to evauluate the system as I have so many outstanding confusions. The people who wrote this paper make strong claims about the speedup in Table I and Table II. They design an experimental

5. Confusion

This paper confused me in so many ways:

* So there is one piece of hardware and it's the hardwares responsibility to pretend to both the OS and the applications that there are multiple of them? Usually in traditional Operating systems, the OS knows what's underneath and it provides the abstractions like closed off processes only to the User Applications. Does this mean the hardware people have to write firmware for Arrakis and general applications that do this?

* How is the control plane not just what the OS would do otherwise? Applications ask resources from it. It seems like it is doing all the work the OS would otheriwse do: interface are VICs, doorbells, filters, VSAs, and rate specifiers.

* I just can't understand how any of this still secure. The whole point of the OS's doing I/O was for security. Now there are libraries that can be accessed by user level programs. I'm guessing these are read only, and these libraries have all the code needed to handle I/O and interface with virtual hardware?!?

* The whole bit of how the network data plane interface and storage data plane interface communicate is unclear to me. So is the conclusion, they've just made an OS that does all the other stuff but I/O and written some specialised hardware and software to make I/O more efficient?

1. Summary
Traditional Operating system are too hands on. They try and monitor every bit of I/O. With a growing demand for sever/client type of I/O the operating systems mediation is slowing things down. These folks have come up with an OS known as Arrakis that doesn't slow things down so much, by not taking part in it all. Now most of the I/O work is done in the Userspace, with user programs using a library to communicate with the hardware directly. They claim that with minor tweaks to hardware and software they can still preserve the security traditional operating systems provide.

2. Problem
In short- I/O takes too long over the network and it's not because the ethernet cable is slow or disk IO is terrible. The CPU overhead cause by the OS handling all the I/O is causing this slow down.

I/O delay is categorised into 4 categories:
* Packet processsing at Hardware layer
* OS scheduling delay
* Switching to and fro between user and kernel mode
* Data copy (I don't think Arrakis gets rid of this)

Arrakis provides performance boosts by removing the first two delays.

3. Contributions
Made their own OS - Arrakis which achieves the following abstractions:

*) Minimize kernel involvement for data-plane operations. Remove kernel mediation for most I/O. I/O is routed from applications address space without compromising security and isolation properties.

*) The traditional read/write code was in the kernel, now it lies in user space in specialised libraries. The control plane plays a role in this read/write but i have no idea how(refer confusion 2)

*) Done by hardware that can present multiple instances of itself to the operating system and the applications on computer. So the the users see multiple versions of the network card and multiple versions of storage even though there is only one. And it seems like the OS doesn't handle any of this. (Refer: Confusion number 1). The applications access these guys through multiple Virtual controllers (which are Queues it seems)

*) Traditional filesystem look up files by their full path, which means internally jumping through multiple levels of inode indirection. Instead, Arrakis provides applications direct control over VSA storage allocation: an application is free to use its VSA to store metadata, directories, and file data. The library exports a file server and everyone queries this server to get files.

*) Just like files, the network I/O which was previously done by the OS, is now done by a library.

4. Evaluation
I don't really want to evauluate the system as I have so many outstanding confusions. The people who wrote this paper make strong claims about the speedup in Table I and Table II. They design an experimental

5. Confusion

This paper confused me in many ways:

* So there is one piece of hardware and it's the hardwares responsibility to pretend to both the OS and the applications that there are multiple of them? Usually in traditional Operating systems, the OS knows what's underneath and it provides the abstractions like closed off processes only to the User Applications. Does this mean the hardware people have to write firmware for Arrakis and general applications that do this?

* How is the control plane not just what the OS would do otherwise? Applications ask resources from it. It seems like it is doing all the work the OS would otheriwse do: interface are VICs, doorbells, filters, VSAs, and rate specifiers.

* I just can't understand how any of this still secure. The whole point of the OS's doing I/O was for security. Now there are libraries that can be accessed by user level programs. I'm guessing these are read only, and these libraries have all the code needed to handle I/O and interface with virtual hardware?!?

* The whole bit of how the network data plane interface and storage data plane interface communicate is unclear to me. So is the conclusion, they've just made an OS that does all the other stuff but I/O and written some specialised hardware and software to make I/O more efficient?