CS 736 Reviews - Spring 2017

1. summary
The paper presents a documentation for the implementation of Linux-Vserver, a Container based Operating System. They present the various challenges they faced while implementing the system as well as attempt to show an improvement over hypervisor based virtualization in certain cases.
2. Problem
OS level Virtualization, specifically how to reduce the overheads of virtualization and make it more efficient.
3. Contributions
This paper seems to written like a documentation for linux-Vserver for the academic audience. As is said "Its a journey not a destination". The paper seems to get lost in the gory details of implimenting a container based OS. But it does do a good job at identifying the various benefits and major disadvantages of such a system. The benefits are more efficient use of the resources because you are sharing/virtualizing the OS layer. The biggest drawback of this technique is that it makes providing true isolation among different VM's difficult and leads to a whole host of issues regarding security.
4. Evaluation
The authors try to do an indepth analysis of the Linux-Vserever and comaper it to native linux and Xen hypervisor. On the surface the evaluation seems to be thorough and well detailed especially when the authors talk about the improvements in efficiency in section 4. But my major concern with the evaluation is in section 5 where the authors seem to be biased towards their containers and feel like they are showing the native linux is better than a virtualized linux because you have a scheduler. Which to me is a bit of a pointless observation.
5. Confusion
The paper even though is a great effort in trying to explain the concepts and implementation details of containers at some level leaves a desire for more cleaner explanation of the detailed mechanisms involved instead of the we did these hacks to make it work style that the authors chose. I was really left confused by the security and isolation related issues in Containers. And secondly how did they prove their system was isolating correctly, in section 5.1 it seems to me they show that the linux scheduler works, it is not a convincing argument in my opinion.

This paper compares the architecture of the 2 types of VM technologies, Hypervisors and Container based Operating System(COS) with the specific examples of Xen hypervisor and VServer COS respectively. The paper demonstrates the tradeoff of one approach over the other when trying to achieve full isolation and efficiency.

Problem:
Isolation and Efficiency are two separate goals in the design of Virtual Machines technology. Hypervisors like Xen provide full isolation at the cost of efficiency. On the contrary, when the need for high efficiency trumps isolation, containers make the ideal choice as they are lightweight alternative to hypervisors and achieve isolation through a shared virtualized OS image and set of system libraries.

Contributions:
The paper details the design of Container Based OS approach with the specific example of VServer with specific focus on the mechanisms to achieve Resource, Security and Performance isolation and how it compares with Xen Hypervisor.

For CPU isolation, VServer implements token bucket filter for CPU scheduling providing a mechanism for flexible isolation policies based on reservation and shares.
For supporting network virtualization VServer provides a similar approach using Hierarchical Token Bucket queuing.
For storage isolation, VServer allows to have a cap on the storage limit on both memory and disk storage.
For security isolation, VServer provides mechanisms like process filtering, kernel data structure modifications and fake init process.
VServer allows file sharing between different guests by allowing 'hardlinking' of the files in the shared file system and makes use of COW, when a guest VM wants to write to a file.

Evaluation:
The authors use micro-benchmarks to measure and compare the overhead of the different virtualization approaches on primitive OS operations. They also perform System-bechmarks to compare and analyze the Network bandwidth, CPU and disk I/O performance overhead in VServer and Xen normalized against Linux. The evaluation shows that network and disk IO is more efficient in VServer and is close to near native performance compared to Xen.

Confusion:

I didn't quite understand the context of chroot system. Can you elaborate on this?
Can you throw some light on how exactly does live migration work in the case of COS?

Summary
This paper describes the design and implementation of Linux-VServer. VServer is light weight way to provide OS virtualization. The main difference between Hypervisor based approach and VServer is that in VServer kernel is time shared among VMs which improves efficiency trading off isolation, which is helpful in scenarios such as high I/O HPC cluster, Grids, distributed hosting.

Problem
Hypervisors provide full isolation between virtual machines, which comes at the cost of efficiency. However, there are usage scenarios which demand high level of performance, where it is acceptable to sacrifice isolation for efficiency. VServer solves this problem by running all VMs with the same kernel image, and using a host VM to multiplex hardware resources ensuring sufficient level of isolation between VMs.

Contributions
The major contributions of this paper are fault, resource and security isolations.

Fault isolation which is to avoid affecting one VM’s defect to other. This is achieved through separation of namespaces and access control.

In resource isolation, CPU isolation is provided by implementing token bucket filter on top of the standard CPU scheduler, Network isolation is achieved using HTB queues and Disk isolation is managed through completely fair queuing (CFQ) I/O scheduler.

Security isolation is achieved by a special file attribute - Chroot Barrier, which avoids unauthorised modification of root.

Evaluations
The paper uses a variety of benchmarks to test three systems VServer, Xen and unvirtualized linux kernel. It evaluates efficiency and isolation aspects of VServer and compares it with Xen. VServer performs better almost across all the benchmarks than Xen, which suffers because of the overheads with virtualizing the virtual memory hardware. Disk performance in Xen is 25-35% less than Linux while VServer slightly exceeds the Linux performance. Scale tests show that VServer scales well while Xen3 degrades with the increasing VMs.

Confusions
How is docker different from linux-VServer?
Would all the virtual servers are affected in case of host kernel failure?

1. summary
The paper compared two kinds of virtualization, hypervisors and containers via specific case study of Xen and VServer. They proposed that for certain workloads in which efficiency and performance are prefered than isolation, container-based virtualization is a better choice. And they provided the study of COS's isolation model and analysis of performance and scalability compared to Xen.

2. Problem
It is obvious that hardware-based virtualied is not efficient enough, and the sharing can be happened in upper-level, OS. Container is a more light-weight virtualization method than hypervisor. This paper want to answer the problem that how much you will gain if you choose container-based virtualization instead of hypervisors and what is the loss.

3. Contributions
a). Detailed and organized understanding of performance, isolation, scalability through both microbenchmark and real-system benchmark from the view of comparision between container and hyperisors.
b). Proposal of Fault isolation, resource isolation and security isolation with respect to VServer and Xen's behavior.

4. Evaluation
This paper performed micro-benchmark and system-benchmark for latency of operation, network bandwidth and io bandwidth comparison.

5. Confusion
a) Not fully understanding why container-based system with contextualization of PID space per VM is beneficial to implementation of checkpoint. PID is also reused within VM, checkpointing need to handle this anyhow.
b) How dese VServer map VMs to resource controller?
c) How is chroot barrier related to the main point?

Summary

The paper describes the architecture and design decisions made in Linux-VServer and compares it against the Xen hypervisor in terms of various crucial performance metrics.

Problem

An ideal VMM would provide complete isolation between the VMs while providing the best performance possible for all the VMs. Isolation is commonly measured based on how well a system provides :

Fault isolation 2. Resource Isolation and 3. Security Isolation.

Vserver by using Container based Operating Systems (COS) while providing two of the above isolations it attempts to outperform one of the most successful Hypervisor based VMM Xen in terms of number of cycles, memory, link bandwidth and disk space provided per VM. It argues that in real use cases performance is usually more important than isolation.

Contributions

VServer introduces new ways of providing the two isolations it guarantees to provide:

Resource Isolation :

For CPU scheduling it uses shares and reservations to distribute the CPU time amongst the VMs. VMs with reservations are served first and then the idle time is distributed among the other VMs according to their share. It also adds a new variable to each VM called token bucket in which tokens are accumulated when the VM is idle and the tokens are spent when the a process of the VM is running.
For sharing network bandwidth VServer again similarly uses two variables called the reserved rate and a share where reserved rate corresponds to the rate at which the VM can send packets and the share determines how much of the remaining bandwidth in the system (if any) is shared with this VM.
Memory and Disk are multiplexed by limiting things like the number of pages locked, maximum resident size and anonymous pages for memory and the number of blocks and inodes for disk.

Security Isolation:

a. VServer uses a global PID space across all VMs. To ensure different PIDs when a VM boots all processes belong to a default host VM.

b. VServer allows controlled access to the networking subsystem with no full virtualization. Packets are tagged with identifiers to avoid snooping.

Evaluation

The paper compares VServer with Xen and Linux across various micro benchmarks like various system calls and mechanisms like page faults and macro benchmarks of running a wide variety of workloads and shows how VServer out performs Xen in almost all the benchmarks.

Confusion

Please elaborate on the architectural differences between COS and Hypervisor like the utility of having three platforms like hosting platform(why an OS image and a host VM?)and application.

1. Summary
This paper compares the implementation, isolation, and throughput of container-based virtualization and hypervisor-based virtualization. In particular, the paper compares Linux VServer with Xen.

2. Problem
Hypervisors provide high levels of isolation, because they virtualize the underlying hardware to give each virtual machine the illusion of a dedicated physical machine. However, providing such an abstraction causes several low-level calls to run slowly and limits throughput. Containers provide lower levels of isolation, because containers virtualize only the kernel. The benefit is that containers can perform a variety of workloads with nearly the same throughput as if the underlying operating system performed the workload itself.

3. Contributions
The authors did not implement Linux VServer and Xen in this paper - both projects already existed at the time of writing. The paper begins by explaining qualitative tradeoffs between container-based virtualization and hypervisor-based virtualization. For example, hypervisors can support multiple kernels, while containers can update a kernel in place while it is running. The paper also defines three types of isolation. Fault isolation is the guarantee that bugs in one guest do not affect other guests. Resource isolation is the ability to limit resource usage for a guest, so that other guests can also access those resources. Security isolation is the guarantee that two guests cannot access the same objects. Then, the paper explains the mechanisms by which VServer guarantees resource isolation and security isolation.

4. Evaluation
In Section 4, the authors perform the lmbench microbenchmark to compare execution times of various low-level tests in pure Linux, a VServer container, and a Xen virtual machine on a uniprocessor. In several cases, Xen takes much longer to run, even though VServer finishes the test in almost the same time as pure Linux. Next, the authors measure TCP bandwidth on the same three systems on both a uniprocessor and a multiprocessor, noting that Linux and VServer are again comparable, but Xen has lower throughput. Several macrobenchmarks that are IO-intensive and CPU-intensive show similar behavior. Performance degrades on both a uniprocessor and a multiprocessor with multiple containers, and even more significantly with multiple virtual machines.

In Section 5, the authors show that containers can achieve close to the desired CPU reservation, but virtual machines may not. Furthermore, if a single container or virtual machine tries to hog disk resources, then other containers have lower throughput, as expected, and surprisingly, other VM's also have lower throughput. The evaluation would have been better if the authors explored how well containers and virtual machines isolate other resources, such as memory. Overall, the authors made many generalizations from VServer and Xen to all container systems and hypervisors, ignoring other designs completely.

5. Confusion
(1) How is fault isolation different from security isolation?
(2) The paper explains that containers only provide two of the three types of isolation. Which one is missing, and why?
(3) What are weights and caps, in the context of CPU reservations?

1. Summary
In the fast – growing technology of virtualization, there’s always a trade – off between efficiency and isolation. Linux-VServer is one virtualization solution which tries to maximize both isolation and efficiency as much as possible. It differs from the popular hypervisor-based approaches in many aspects, including the huge difference in kernel footprint, while having some stark similarities with them as well.
2. Problem
One fundamental problem system designers face is deciding the level of compromise between process isolation and sharing objects in order to achieve efficiency. Hypervisors effectively provide complete isolation, but end up heavily penalizing efficiency. There has been a rise in new computing scenarios like HPC cluster, Grids, distributed hosting which demand high level of performance by efficiently using the system resources. The paper presents Linux-Vserver, a COS virtualization approach targeted at achieving high level of performance while providing robust techniques for isolation.
3. Contributions
The two major contributions that the paper discusses are – the techniques involved in the design and implementation of COS based virtualization and the analysis of contrasts and similarities between COS-based approach and hypervisor-based approach.
The COS technology provides a means to isolate the VMs at the system call/ABI layer, which necessarily means that the hosted VMs share a single kernel. This achieves a lower kernel footprint without compromising the performance. This approach also allows us to perform live system update. The paper also points out the similarity in multiplexing CPU and other resources in COS and hypervisor based technologies.
COS-based technology uses Token Bucket Filter for CPU scheduling to provide a means to allow flexible isolation policies based on reservation and shares. Similarly network bandwidth allocations are provided using Hierarchichal Token Bucket queuing discipline. The concept of tagging the packets with context id is introduced to differentiate packets from different processes in VServer. VServer also allows to have a cap on the storage limit on both memory and disk storage.
VServer provides mechanisms like process filtering, kernel data structure modification, faking init process to provide security isolation. This also simplifies process migration between VMs, with the help of special spectator VM. To avoid unauthorized modification of root, VServer provides a file attribute called chroot barrier. VServer allows file sharing between different VMs via FS Unification and ensures its integrity by providing CoW.
4. Evaluation
A huge chunk of the paper is dedicated to the evaluation and comparison of system efficiency, performance and isolation. The paper analyzes these various factors by running micro-benmarks, N/W bandwidth benchmarks, macro benchmarks. Though hard to quantify, the paper provides evaluation of isolation using Hourglass and multi-OSDB benchmark, the authors have done a good job in this regard.
After all the evaluation and discussions, one could clearly see that the trade-off between isolation and performance is almost Pareto efficient. While hypervisor based technology has strong support for network stack, COS technology does really better in I/O performance. Also COS technology totally avoids the issue of virtualizing I/O. The paper does justice in explaining these concepts.
5. Confusion
1. Can you please explain the working of network bandwidth allocation in VServer, especially the tagging of packets with context IDs?
2. How does COS approach help in isolating group of users in a data centre?
3. Can you please explain the relation between docker and COS technology?

Summary: This paper presents design and implementation of Linux-VServer, which was developed to provide system virtualization with high degree of isolation and efficiency. Linux-VServer provides greater efficiency, but slightly weaker isolation than hypervisor based OSs like Xen. The paper describes container based techniques and compares benchmark metrics of Xen (hypervisor based OS) with Linux-VServer (container-based OS).

Problem: Traditional hypervisors prefer isolation to efficiency. Such systems are suitable for use cases where different VMs belong to different entities and need to be protected / isolated from each other. However, there are cases where all applications in each VM run based on a single kernel image and primary focus is efficiency instead of isolation. HPC is example of such a use-case. Containers try to address this issue by achieving greater efficiency at the cost of slightly weaker isolation.

Contributions: This paper introduces various techniques and precise definitions of fault isolation (limiting an issue on one VM to affect others), resource isolation (resource management of multiple virtual machines) and security isolations (providing limited access of logical objects to VMs) as way to quantity isolation in order to be able to compare both hypervisors and containers. Container segregates namespaces and access control to provide fault isolation, uses reservation based token bucker filter system to provide resource isolation, and allows access of individual process namespaces to kernel only to provide security isolation. Separate IP for each VM also helps in security isolation. Common files are shared using Copy-On-Write (COW) technique to improve efficiency. Efficiency is also improved by allowing process migration from one VM to another VM. Overall, VServer (COS) trades isolation for efficiency (performance and scalability).

Evaluation: Authors compare performance of Linux-VServer (COS) with Xen (Hypervisor) with various workloads such as network traffic and high CPU and I/O intensive workloads and observe that VServer provides performance almost similar to native system while Xen performs marginally slower. VServer performs better than Xen in scalability test as well owing to virtualization overheads of Xen. Authors don’t evaluate multiple COSs and hypervisors which I believe is a shortcoming in the evaluation.

Confusion: 1. How does process migration work? When is it useful?

1. Summary
This paper talks about Linux VServer, a container based operating system and compares it with hypervisors. It describes various techniques used by the VServer to provide better efficiency while trading off isolation. It also evaluates Vserver by comparing it’s performance with a recent version of Xen.

2. Problem
System efficiency and isolation are two factors which are taken into consideration while designing a system to support VMs. Hypervisors perform well to provide isolation to various VMs but they sacrifice the efficiency of the system as some of the resources dedicated to a VM might remain idle. There are some usage scenarios like HPC clusters, the Grid and distributed hosting organizations for which efficiency is more important. Container based operating systems tries to tackle this and provide better efficiency.

3. Contribution
Authors have compared VServer and a recent generation of Xen to show that VServer can provide better efficiency without any substantial impact on isolation. They observed that there is no VM technology that can maximize both efficiency and isolation. Authors described various techniques that they used for providing isolation. CPU isolation is provided by implementing a token bucket filter on top of the standard CPU scheduler. To improve performance, COSs opt to apply access controls on shared namespaces. They also used hierarchical token bucket to provide network bandwidth reservations and fair service among VServer.

4. Evaluation
VServer was evaluated against a recent generation of Xen. Both the systems performed equally for CPU bound workloads. For I/O centric workloads, VServer achieves better performance by more efficient use of system resources and it far surpasses Xen in terms of scalability. Authors also compared these two systems using micro-benchmarks. Execution time for VServer is almost identical to Linux kernel while for Xen3 it is much higher. Comparison between these systems in terms of network bandwidth, CPU utilization and memory utilization is also shown.

5. Confusion
Could you please explain more about IDDs and how they can be used to provide fault isolation.

Summary:
This paper presents the design and implementation of Linux-VServer, a container-based operating system virtualization and compares its architecture against that of hypervisor. Containers provide comparable isolation and higher efficiency, especially for I/O centric workloads by allowing better use of system resources.
Problem:
Commodity OS provides weak isolation and efficient sharing. On the other hand, hypervisor provides full isolation with no sharing between VMs. Containers achieve best of both worlds by striking a balance between isolation and performance.
Contributions
This paper provides a detailed description of the design and techniques of VServer which supports light-weight VMs running on top of single operating system as isolated entities.
The techniques adopted by COS are compared with those used by Hypervisor to weigh degree of isolation and performance.
Resource isolation : A simple token-based scheduling is used to ensure fair CPU sharing.
Security isolation : COS preserves system safety and configuration independence. It provides partial security isolation by having a shared namespace protected via access control mechanism.
Although it provides somewhat weak isolation compared to hypervisors, it is more performant.
Evaluation:
VServer and Xen perform similar for CPU bound workloads. VServer outperforms Xen for I/O centric workloads by providing efficient use of system resources. Scalability test shows that VServer is much better than Xen as it eliminates the overhead of duplicating OS and better CPU allocation.
Confusion:
Not sure how chroot works.

Summary
This paper presents VServer, a container-based OS that provides virtualization at the system call level and aims to provide higher efficient than a Hypervisor with reasonable VM isolation.

Problem
Hypervisors are focused on running unrelated application and thus demanding higher levels of isolation. When multiple VMs are running similar OS distributions, hypervisors fail to take advantage of this information for better performance. Hardware virtualization comes with the cost of lower I/O throughput and scalability. Thus, authors present a Container based OS overcomes these limitations with some compromise in isolation.

Contribution
As clearly specified by the authors, thorough description of the VServer and comparison of VServer with Xen hypervisor, are the contributions of the paper. Authors claim that VMMs have focused largely on isolation by compromising on efficiency and it is not a universal solution; Some applications might focus on efficient more than isolation.
1. Resource Isolation: CPU isolation is provided by a token bucket filter on top of Linux CPU scheduler. This mechanism helps in achieving range of isolation policies. E.g. To provide an upper bound on CPU time allocated for a VM. VServer provides reservation and shared based token accumulation thus enabling system to be paritioned between VMs with higher CPU reservation and VMs which share the unreserved proportion. For I/O fair share and reservations, Hierarchical Token Bucket queuing is used to provide reservation and fair share of service among VMs. Memory and storage limits are provided by limiting the number of block and memory VMs can use. A watchdog deamon periodically checks of memory overload and resets the VM which consumes the highest memory.

2. Security Isolation : Process filtering-- VServer filters processes and provides isolation between processes of other VMs using additional data structures.
Network Separation-- Each of the VMs are allocated fixed IPs. Packets are tagged with VM id for filtering (and handled in network stack) so that packets reach the right VM. Though it sounds inefficient, authors show that t/p numbers are almost indistinguishable to native Linux.
VServer saves space by sharing files that are common across VMs by creating hard link to files across VMs. To overcome the possible security issue with this sharing, VServer provides copy-on-write feature.

Evaluation
As the VServer was introduced to provide better performance when compared to hypervisor based virtualization, authors provide detailed analysis of VServer against Xen3 hypervisor. Though comparison against a hypervisor (Xen) shows that VServer was able to outperform its opponent, it might happen that VServer could under perform against other hypervisors (VMware). The important information for the performance analysis is the difference between Native Linux and VServer. In most of the cases VServer is almost on-par with native Linux. This shows that Containers can provide higher efficient with isolation.

Confusion
When the watchdog kills a VM that has highest memory consumption, isn't it unfair as the VM was promised to provide enough memory (even though it is overbooking, VM doesn't know it) ?

It is not intuitive to understand how I/O can be better than a Hypervisor ? Doesn't every I/O goes through more layers now ?

Container Based OS Virtualization

1. summary
This paper is about new Virtualization system providing more throughput than Hypervisor based Virtual Machines. This new system offers better performance compromising degree of isolation to some extent. Apart from Performance, this system also efficient allows resource sharing among Container based Isolated Virtual Machines unlike Hypervisor based Virtualization.
2. Problem
The overhead of running hypervisor based Virtual Machines is high although it provides isolation. For use cases not requiring high degree of Isolation, Hypervisor based Virtualization bears high cost. Also, resource sharing in this case is not so effective. The paper leverages Linux kernel data structures to provide Isolated Environment.
3. Contributions
The paper proposes Container based OS Virtualization based on Linux. COS provides lot of Isolation features typically classified as Fault Isolation, Resource Isolation and Security Isolation. COS and Hypervisor based VMs provide fault isolation assuming the interface between ‘Host OS’ and ‘Guest OS’ are not buggy. COS can share Process Organization, Memory state with Host OS. Hypervisor VMs have Virtual Memory adding one more layer of address translation. COS provide Security Isolation by providing context based Resource Visibility and Access controls. Hypervisor handles the security isolation of resources. VMs share same in Kernel in COS unlike in hypervisor. This is major difference in COS virtualization. Authors have modified Linux kernel (called VServer) to guarantee fair share of resources among VMs. VServer banks on Linux’s IO and Network Scheduler to provide fairness among VMs. VSphere stores context for every process to hide processes not part of given VM. As a result, VSphere also supports process migration among containers. Some containers virtualize network stack to provide Security Isolation. VSphere relies on static configuration provided at boot time. VSphere supports Filesystem unification which involves sharing of files among VMs with Copy-On-Write feature. Any modifying VM will get its own copy of the file. Thus, storage consumption drastically improves as compared to hypervisor based Virtualization. VSphere also provides Chroot Barrier to ensure Security Isolation to host OS.
4. Evaluation
The Authors have done extensive evaluation running different types of workload to analyse overheads in APIs provided by kernel. In every case, COS virtualization matches the performance of native Linux processes. Hypervisor based Virtualization faces significant performance bottlenecks as compared to native Linux and COS.
5. Confusion
1. Authors have compared COS against Para-virtualization. Full Metal Virtualization can provide better performance and should have been used for comparison.
2. In Dbench test, VServer-UP seems to doing better than Linux-UP. How is this possible?
3. Why do we need chroot barrier?
4. How does VServer handle Network IO Fairness?

1. Summary
The paper talks about the need for system virtualization providing both isolation and efficiency. Using prior work on resource and security containers, Linux-VServer - a container based system is designed, implemented and contrasted with hypervisors.

2. Problem
Hypervisors provide performance isolation for applications running on different VMs. However, this often leads to inefficient use of system resources. For instance, hosting compute farms require multiple copies of server software in VMs and efficiency becomes sensitive here. In real world scenarios, one may find it acceptable to tradeoff isolation for efficiency.

3. Contributions
The paper discusses isolation using container based OS system. Addressing the problem above, VServer trades isolation for efficiency. Isolation is achieved by providing separate name space (context) and access control (filters). It provides CPU isolation by overlaying token bucket filter on top of O(1) scheduler, network bandwidth reservations and fair service using Hierarchical Token Bucket (HTB). VServer uses the same global PID space across all VMs and filters processes to hide those outside a VM’s scope thus ensuring security isolation.

4. Evaluation
VServer (COS system) is evaluated against hypervisor-based Xen3. For all cases, VServer is comparable to non-virtualized Linux kernel. Overheads in network I/O and disk I/O cause significant performance overhead in Xen3. VServer, thus, clearly performs better than Xen3.

5. Confusion
Was this first attempt at studying container-based OS and comparing it to hypervisors ? I see mention of OpenVZ (Another container-based OS). How did it perform ?

Summary:
Two major approaches of providing a virtualized environment are virtualization at the hypervisor level where a hypervisor multiplexes the resources among multiple virtual machines and the container based operating systems. The paper describes VServer [COS] and compares it with Xen. The authors conclude that the COS approaches enforce high degree of isolation while maintaining efficient use of system resources.

Problem:
Virtualization deals with the problem of choosing the tradeoff between isolation of virtual machines from each other and communication between the virtual machines. The main design goal of the hypervisor level virtualization was isolation of resources and not the efficient use of system resources. There is a lot of overhead involved in sharing the data between the virtual machines in the traditional approach to virtualization.

Contributions:
Container based operating system like VServer runs the virtual machines on top of a single operating system as a secured isolated entities. Fault isolation is achieved by having a separate namespace for each virtual machine and also through access control filters. Resource isolation in case of CPU is achieved using token buckets for each VM which are populated based on the reservations and shares. Network isolation is achieved using HTB queues. Security isolation is achieved by having a separate root directory for each virtual machine(done with the help of chroot system call), having a separate IP address for each VM, so that the network packet reaches the correct VM.

Evaluation:
Authors have compared the container based operating system VServer with the Xen, VMM implementation and have showed that the Vserver performs more efficiently on various benchmarks compared to Xen without compromising individual virtual machine isolation.

Confusion:
Could you please elaborate on CPU fair share and reservations in case of COS.

1. Summary
Hypervisors are the de facto standard when it comes to providing security and enabling higher efficiency with a set of resources. However, there remains the knowledge gap between guest OSes and the hypervisor. Containers are a different solution that trades off some freedoms and modes of operation for hypervisors for increased efficiency while aiming to provide the same isolation guarantees.

2. Problem
Hypervisors lack the knowledge of a single OS to schedule efficiently. By moving the layer of abstraction closer to the application, thus making more things known, we can more efficiently schedule the virtual machines as well as eliminate expensive hypercalls. How can we do better than hypervisors and what are the tradeoffs?

3. Contribution
The contributions of this paper are primarily its evaluation of VServer and the contrasting of mechanisms used by hypervisors. VServer mainly uses existing ideas to provide resource isolation. I wouldn’t say any are particularly novel or new. Naturally, VServer requires management of many more data structures as it needs to manage inside an OS. The benefit here is that things like sharing files becomes more natural instead of requiring snooping like in the ESX hypervisor.

4. Evaluation
Overall, the evaluation points to what we would expect from the elimination of heavy synchronization done by the hypervisor to minimal management handled by the OS. While containers do provide a lesser degree of isolation, I assume that the gap has closed in more recent years. It is clearly outlined that this approach is ideal when you are in control and are running as a single tenant.

Ultimately, it seems that VMs and containers each have their use cases with VMs providing a stricter degree of security and minimization of interference while containers provide the ability to better oversubscribe. An area VMs have a leg up is with migration as stopping the world is much more defined than stopping a container. The flip side is if managed properly, live upgrades are possible with containers while not possible with VMs for obvious reasons.

5. Confusion
Could you go into a little detail about your work with IDDs and what you believe is the next direction of virtualization?

1. Summary
Hypervisors provide virtualization with high degree of isolation and less efficiency while container-based operating systems provide efficiency by trading off isolation. This paper describes the techniques used in the Linux-VServer container based system, compares its architecture to that of Xen and discusses their differences.

2. Problem
Compute farms which serve many users usually run jobs in a batch scheduled manner. They tend to care more about raw performance and throughput in terms of jobs completed rather than complete isolation between these jobs. They also tend to have different software configuration requirements while running on the same operating system kernel. Therefore, running them in multiple virtual machines results in less efficient use of system resources.

3. Contributions
The paper describes the architecture of Linux-VServer container-based operating system. Separation of name spaces (contexts) and access controls (filters) are used to provide resource and security isolation. Control over physical resources is delegated to a host virtual machine. VServer maintains fair CPU scheduling and I/O bandwidth with a token based mechanism using reservations and shares. It also places limits on the amount of memory and disk a VM can acquire. VServer enforces security isolation by making a number of kernel modifications. It reuses the global PID space across all VMs. It shares the networking subsystem between all VMs, but only lets VMs bind sockets to a set of IP addresses specified dynamically or at the time of VM creation. Files common to more than one VM are hard linked on a shared filesystem and protected using copy-on-write attribute.

4. Evaluation
The paper evaluates performance and scalability of VServer against Xen-based virtualization. Xen loses in performance to VServer as it virtualizes the virtual memory hardware. Xen also has higher overhead in network I/O and disk I/O. Also, it is found that CPU scheduling is more efficient in VServer due to lower overheads in the COS based approach. But the evaluation is only VServer against Xen, and not a container-based system against a hypervisor-based system in general. The paper does not explain why VServer is a good representative for container-based systems and why Xen a good representative for hypervisor-based systems and not, say, VMware ESX Server.

5. Confusion
How do live system updates work in container-based systems?

Summary
The paper introduced container-based operating system virtualization (COS) with a focus on Linux-VServer, as an alternative to hypervisors like Xen. The authors made comprehensive comparison between the COS and hypervisor focusing on the design trade off they made with isolation versus efficiency.
 
Problem
The authors argued that even though hypervisors are providing superior isolation (even full isolation), there are many scenarios valuing efficiency (performance and scalability) more than isolation. One example can be oversubscribed hosting organizations trying to reduce the marginal cost per customer VM. Meanwhile, container also provides a lighter solution than traditional VMs within a hypervisor.
 
Contribution
The author claimed two contributions:
1. The paper provides the first thorough description of the techniques used by Linux-VServer to academic audience.
2. A comprehensive comparison between VServer and Zen.
 
From my perspective, the following ideas, might oriented from other related work are pretty aspiring.
1. Use both reservation (indicates the amount of outgoing bandwidth dedicated to a VM) and share (the share of excess capacity) to promote both fair share and performance. The paper mentioned three modes: capped reservation, "fair best effort" and work-conserving reservation for different needs.
2. Token bucket filter (TBF) mechanism for CPU sharing.
3. As VServer does not fully virtualize the networking subsystem, tag network packages with VM identifier is needed.
4. Sharing libraries from similar OS distributions, as mentioned in VMWare ESX as well.
 
Evaluation
• In many cases, the authors used the reduced number of lines of code as an argument to say the system is more reliable and light weight. Is the number of lines of code really a good measure light-weight or reliability?One of main component of system code contribute to system code is various drivers which might never get used and longer code does not necessarily means less reliability.
• Brian in my reading group brought up a good point, as mentioned in the paper, table 2 "shows results only for those which represent approximate 2x or great discrepancy between VServer-Up and Xen3-UP." I think it might be okay to only list the features with larger discrepancies in detail. However, the paper might also want to mention how other features looks for complete information.
• I like how the author made many clarifications, like the definition of VM and the definition of efficiency (performance and scalability), which made the paper easier to understand and read.
• For table 1, I am wondering is there a single hypervisor satisfies all the checked features for hypervisors and is there one COS satisfies all the features checked for COS. If not, then how many features does a single capable hypervisors or COS have? I feel like it would be helpful to make this clear.
 
Confusion
1. How exactly does VServer design networking virtualization and isolation? What are the design trade off with VServer's "not fully virtualize the networking subsystem approach", specifically, how does this help VServer achieve native Linux networking performance at GigE+ line rates?

1. Summary

This paper describes the design and implementation of Linux vServer which is a light weight way to provide operating system virtualization. Linux vServer belongs to the container based class of virtualization platform. Unlike hypervisors, the major goal of container-based virtualization is to achieve better efficiency while trading off VM isolation for that. The paper also shows that vServer provides much better performance in virtual OSes (esp the I/O performance) compared to Xen while still providing similar level of isolation.

2. Problem

Sharing and Isolation are two competing goals in the design of operating systems. While hypervisors provide a high degree of isolation by providing virtual interfaces to VMs which might be running the same kernel and OS, this isolation comes at the cost of efficiency. However, for some scenarios, e.g., high I/O HPC farms it might be acceptable to sacrifice isolation for efficiency. Containers provide a way to achieve that.

3. Contributions

I think the major contribution of this paper is the design and implementation of VServer's isolation mechanisms. This includes resource isolation and security isolation.

For CPU resource isolation, Vserver implements token bucket filter on top of the scheduler. In this scheme, each VM gathers tokens while it is not running depending upon the reservation or sharing model and then spends one token for each clock tick while it is running. Similarly, for Network virtualization, VServer implements hierarchical token buckets in which there is a bucket associated with each of the guest VMs. Packets generated by each VM are tagged in the server and are then copied into corresponding bucket. For storage isolation, VServer fixes and upperbound on the number of disk blocks and inodes for a VM. For memory, it uses a watchdog demon which can reset the VMs which might be taking too much memory. To provide security isolation, Vserver filters the processes such that each VM has a view of process inside only itself. A VM cannot access processes that belong to some other VM. Finally, to be storage efficient, VServer implements file sharing across guest images where files common to multiple images can be hardlinked in the shared file system. Later on, if a guest wants to write to a shared file COW is used.

4. Evaluation

In the the performance of VServer is compared with Xen against both micro and macro benchmarks. For the LMBench OS benchmark, the performance in terms of execution time for VServer is almost identical to native linux while for Xen that time is much higher. This in general can be attributed for the high memory management overhead in Xen. Similarly, the network and disk IO is also considerably more efficient in VServer compared to Xen.

5. Confusion

In section 3.3 it mentions about a spectator VM that allows a global process view for all the guests. I am not sure how is that working and why is it needed. I don't quite understand the chroot section, and the need for chroot here. Can we discuss it a little bit in the class ?

1. Summary
This paper introduces the container, a lightweight virtualization technique which trades some extent of isolation for efficiency. Specifically, the paper presents some implementation details of a container-based OS (COS) Linux-VServer and compares it with a hypervisor Xen in terms of both architecture and performance.
2. Problem
Hypervisors favor isolation over sharing because in datacenter consolidation scenario, different VMs run on behalf of different organizations. However, hypervisor lacks efficiency when multiple VMs run the same kernel, compared to running all the applications in all VM based on a single kernel image, which is the case in scenarios like HPC. Container solves this problem by running all VMs with the same kernel image, and using a host VM to multiplex hardware resources for real VMs which are like isolated execution environments with separate namespaces and different filters based on the host VM.
3. Contributions
Authors proposes to evaluate different virtualization techniques in terms of both efficiency and isolation. Efficiency could be measure with overall throughput and aggregate latency (in Evaluation part). For isolation, authors evaluate container and hypervisor on a conceptual level. In terms of fault isolation, although hypervisor’s small code base makes it seemingly more reliable than container based OS, but hypervisors’ relying on host VM to multiplex devices degrade the overall reliability. In terms of resource isolation, both virtualization techniques can implement complicated resource allocation policies to provide different resource sharing pattern (reservation, shares). For security isolation, COS implementing partial isolation for global namespace are less secure compared with fully isolated scheme. And COS’s isolation directly involves internal operating systems objects like PID and IPC, while hypervisor rely on careful allocation of virtual memory and control on devices and privileged instructions.
Authors also present some implementation details of Linux-VServer, including the shares and reservation allocation policy for CPU and I/O, limits on storage, process filtering among different VMs and network separation by binding IP addresses to VM statically or dynamically.
4. Evaluation
Authors compare the performance of a COS Linux-VServer and a hypervisor Xen. They carefully configure the kernel to ensure that the differences result from different virtualization techniques rather than different kernel configuration.
For network traffic benchmarks, VServer performs similarly to native Linux and outperforms Xen in terms of CPU utilization and throughput on a single core machine, due to Xen’s contention of hypervisor and VM. For CPU and disk I/O intensive workloads, VServer and Xen performs similar for pure CPU intensive workloads while VServer outperforms Xen for disk I/O intensive workloads due to data copying overheads between host and guest VMs. For scalability test, VServer also outperforms Xen as a result of low virtualization overheads and better CPU allocation policy. For resource allocation test, VServer exhibits smaller skew from designated partition due to its complete CPU allocation policy including both shares and reservation. Finally for resource allocation, Xen behaves better than VServer thanks to its full isolation of memory.
5. Confusion
- How does container implement virtualization? From my understanding it’s just providing isolated execution environments as VMs, but I don’t know if this is true.
-The results presented in the paper show that a lot of Xen’s overheads come from copying and communication between host VM and VMs. So why not compare Xen with hypervisors without host VM, like VMware ESX?
- The sharing techniques mentioned in this paper is not very different from ESX, why can containers achieve better sharing than hypervisors with similar techniques, as argued by the authors?
-For the evaluation part, it makes little sense to evaluate performance on single core systems, especially by the time the paper was written in 2007.

1) Summary

Containers offer an alternative to virtual machines with higher performance but weaker isolation. The authors compare a full virtualization implementation Xen with a container implementation Linux VServer. They find that VServer gives near native performance but also expose areas where its isolation is weaker. Meanwhile, Xen gives full virtualization, so isolation is stronger, but performance is lower, especially in IO intensive workloads.

2) Problem

Virtualization technology has grown in popularity in recent year, particularly with respect to server virtualization. However, fully virtualizing machines results in significantly slower IO, among other overheads. Containers offer a solution with significantly lower overheads by sharing a single kernel among all virtualized environments at the cost of weaker isolation. The authors seek to analyze this tradeoff between performance and isolation.

3) Contributions

The main contributions of this paper are a description of the Linux VServer implementation of containers and a comparison between VServer and Xen. The description of VServer reveals many of the challenges of retrofitting Linux with container support, particularly with respect to isolation and implementation. The authors describe several workarounds and hacks that are needed to implement VServer. They also show where shared infrastructure in VServer weakens isolation.

Meanwhile, the analysis of performance and isolation in VServer and Xen quantifies well-known differences between full virtualization and containerization. They find that VServer allows most operations to happen at nearly native speed in terms of latency and throughput. On the other hand, Xen's IO and memory virtualization adds significant overhead for most workloads. Moreover, the authors find that containers are more scalable due to their lower overhead. Finally, they find that both VServer and Xen suffer from performance isolation shortcomings.

4) Evaluation

This paper does a good job at detailing both the benefits and drawbacks of the VServer implementation and containers in general. In their evaluation section, they use a variety of microbenchmarks, macrobenchmarks, and system benchmarks that are representative of typical server workloads. Moreover, the comparison with Xen provides useful information for when each solution is appropriate.

However, overall, I felt that the paper was poorly written. There were many long and hard-to-follow sentences and the organization of the paper's discussion was not very fluid. Worse, there were numerous places where the paper contradicted itself. For example, the abstract states that containers and hypervisors offer comparable isolation, but the section on isolation and the figure summarizing virtualization solutions clearly state that containers have worse security isolation.

Moreover, in my opinion, the implementation of VServer is extremely hacky. The authors detail several quick fixes they are forced to used, such as creating a fake init process or using "spectator VMs". These hacks reflect the fact that Linux was never designed with containers in mind; rather, it is being retrofitted. The sheer number of silly hacks and corner cases indicates that a proper container solution must be built from the ground up. The current model of retrofitting Linux seems unsustainable and produces an unmaintable mess.

Finally, the exact methodology of the evaluation section seems unfair in a few places. First, the authors never explain why they choose Xen and believe it is representative of all full virtualization techniques. Second, in the CPU scheduling section, the authors compare Xen's scheduling against VServer's scheduling... The problem is that Xen was not designed to have a fair scheduler, so the comparison is meaningless. In addition, in the network throughput section, the paper claims that Xen requires ~130% CPU to achieve the same throughput that VServer can achieve at ~70% CPU. But in order to get this ~130%, the authors include the CPU usage of the hypervisor itself. Instead, in my opinion, only the CPU usage of the benchmark itself should be included since this is the workload being measured.

5) Confusion

I am not sure I understood the role PlanetLab in the context of this paper. Why do the authors mention it?

1. summary
This paper compares many elements of the Linux-VServer and Xen virtualization methods.
2. Problem
Complete process isolation is the goal when virtualizing systems but doing so with hypervisor based systems introduces large overheads when switching between the VMM and VMs which is very likely to occur for most server applications. Container based solutions however can virtualize at the system call level allowing for lower overheads with many IO driven tasks.
3. Contributions
The paper analyzes the isolation and performance properties of the VMM and COS solutions. It discusses how all of the isolation properties afforded by a VMM can still be realized by a container based solution. In some instances, such as CPU fair share usage, the container solution can even do better than the Xen VMM. It also demonstrates that the performance of the COS is able to nearly match that of baseline Linux for many different benchmarks aimed at memory, network, and disk usage.
4. Evaluation
A range of benchmarks are used to compare both VServer and Xen. The systems are configured to be as close to identical outside of the virtualization method as possible. The results in many cases seem cherry picked to favor VServer. Table 2, for example, only shows benchmarks with a large discrepancy between systems yet the rest of the benchmarks are not discussed anywhere. There are also a large number of uniprocessor tests that favor VServer and frankly do not seem like the likely usage case for the majority of applications as multiprocessors were becoming common at the time of the study and even their test system was one.
5. Confusion
I’m not sure I understand how the different namespaces work. For example, looking at the filesystem, can the host VM see the entire filesystem but then every other VM can only see a portion of it or do they see all of it but have restricted permissions to most of it? Similarly with the PID namespace if a guest VM ran ‘ps’ would it see all process and not be able to interact with those outside of it or just its own processes?

1. Summary
This paper introduces container as a kind of virtualization technique. Container works like a balance between virtual machine monitor and process, which can provide more isolation than process in traditional OS, and can also reduce the overhead led by VMM.

2. Problems
We often face the dilemma of isolation and efficiency. Virtual machine monitor provides good isolation as if two applications are running on separate machines, but faces some performance overhead even with fancy techniques mentioned in ESX paper. Process has little performance overhead, but can not provide an isolated environment for apps. So, this paper tries to solve this dilemma by allowing multiple VMs to share the same OS kernel, keeping the advantages of both VMM and traditional Linux, and this leads to the Container-based Operating System(COS).

3. Contributions
This paper gives a definition of different aspect of isolations. Unlike efficiency that can be easily measured by benchmark, isolation is hard to quantify. So, this paper introduces fault (limit a bug from one VM to affect others), resource (manage the resource of multiple VMs) and security isolation (each VM has limited access to logical objects like files) to help us analyze the isolation of a virtualization.
This paper also gives a clear description of how COS works, using Linux-VServer as an example. To analyze the isolation of COS, this paper follows the three aspects of isolation. To guarantee resource isolation, it uses token scheduling and token bucket to keep a fair share of CPU and IO resources, also, it adds limitation to the storage and memory usage of VM. To guarantee security isolation, it creates PID space, special IP address and special root directory for each VM, so that each virtualized system can only access its own process, network packages and filesystem. It also has various experiments to evaluate the efficiency of COS.

4. Evaluation
In evaluation section, this paper use Linux-VServer as COS, Xen as virtual machine monitor, and original Linux. This paper uses microbenchmarks to test the syscall performance, COS is similar to native Linux because it does not need to emulate some OS operations, but VMM is much slower. In system benchmarks, this paper tests network throughput and some IO performance, COS also outperforms native linux. For scalability test, this paper changes number of VM from 1 to 8 and run the same benchmark on each VM, COS works better than VMM, especially when the number of VM increases.

5. Confusion
(1). Are there some criterions for choosing the right virtualization method? Given an application, how can we decide which virtualization method to use?
(2). Why is Docker so popular in recent days? Does it has some special features that are different from the container in this paper?

1. Summary
This paper's main focus was to compare the VServer container implementation with the Xen hypervisor implementation and show that container based virtualization can outperform the conventional hypervisor. The paper accomplished this goal by showing that containers can achieve comparable isolation with much higher efficiency than Xen hypervisor.

2. Problem
The problem with hypervisors is that each virtual machine has its own kernel and operating system leading to inefficient usage of system resources. Hypervisors however provide excellent isolation which is essential to users running on the same hardware.

3. Contributions
This paper used the comparison between the VServer and the Xen to show that containers could not only achieve comparable isolation but do it in a much more efficient manner. Isolation is a hard aspect to quantify and the paper uses the first section of comparison to describe three parts of isolation for both the container and hypervisor. Mostly the paper finds the two techniques have similar isolation. Containers do have an interesting token based method of requiring tokens to gain access to the CPU. This is containers method of keeping CPU utilization fair while also allowing for favoritism in the VMs by giving one VM a higher rate of tokens. In terms of efficiency since containers only need the one kernel they make much better usage of the system resources.

4. Evaluation
Since the main point of the paper was comparing VServer with Xen the paper uses a variety of benchmark test to verify their claim on containers. For all tests except one VServer outperformed Xen. However, on the one test where VServer-SMP underperformed Xen there is no explanation as to why other than that the SMP tests perform worse. It also seemed odd that when comparing overhead the paper only showed those benchmarks that did twice as well.

5. Confusion
Why exactly do these virtualization techniques perform so badly with SMP especially since multiprocessing is such a major focus today?

Summary:
This paper describes design and implementation of Linux-VServer, a container based OS system. VServer is built on resource container and security container work. It provides a shared, virtualized OS image consisting of a root file system, a set of system libraries and system executables. Kernel is time shared between VMs which removes the virtualization overheads associated with hypervisors without compromising much on isolation.

Problem:
Hypervisors provide full isolation between virtual machines, providing no more support for sharing between VMs than the network provides between physical machines. There are usage scenarios where VMs run same system software, kernel and similar operating system distributions. These cases require isolation but also have the opportunity for sharing. Using hypervisors to run these applications on separate VMs induces significant overheads and leads to inefficient use of system resources. Container based OS (COS) virtualization provides better system efficiency for such applications while ensuring sufficient level of isolation between VMs.

Contributions:
The paper describes and evaluates a COS based system, VServer and compares it with a hypervisor based system, Xen. COS approach trades isolation for efficiency. It provides resource and security isolation between VMs and trade-off fault isolation for better efficiency. Here by efficiency, they mean performance and scalability. VServer is implemented by minimal modifications to the kernel code. Host VM manages other VMs and applications run on Guest VM. VServer uses separate name space and access filters to provide security isolation between different VMs. CPU isolation is achieved by using a Token Buffer Filter. Hierarchical token bucket is used to provide network bandwidth reservations and fair service. Vserver imposes storage limits on memory and disk usage. It uses file system unification to reduce overall resource usage.

Evaluation:
The paper evaluates efficiency and isolation aspects of VServer and compares it with Xen and an unvirtualized linux kernel. Almost across all the benchmarks, Vserver performs as well as unvirtualized linux kernel whereas Xen performs badly. Xen loses on performance due to virtualization overheads. Disk performance in Xen is 25-35% less than Linux while Vserver slightly exceeds the Linux performance. Vserver is shown to scale better due to lower overhead imposed by COS approach and a better cpu scheduler for keeping competing VMs progressing at the same rate. Vserver and Xen provides comparable results for CPU fair share allocation and both systems get impacted by a competing VM.

Confusion:
I do not understand the part in Vserver Security Isolation section where they say that all processes belong to a default host when a VServer based system boots. Later the processes are migrated to another VM. Why is there a need to migrate these processes?
Why disk performance in Vserver is better than in Linux?

1. Summary
The paper describes Linux-VServer and compares the architecture with the Xen hypervisor. It uses a container-based virtualization to provide isolation at the system call / ABI layer instead of the hardware abstraction layer like in Xen. The authors argue that this approach provides greater efficiency with comparable isolation to hypervisor-based systems.

2. Problem
Hypervisor or VMM-based virtualization had become a common technique for server consolidation, fault isolation and ease of deployment. However with the advent of techniques based on resource and security containers, it became possible to provide isolation at the system call layer itself. This approach would eliminate the context switching overheads between the VMM and OSes and also allow more scalability by supporting packing of more containers in a server. Although there exist other container-based OSes, the authors pick Linux-VServer because it is open-source and is used in production systems.

3. Contributions
The authors make the case that container-based virtualization approaches can provide greater efficiency than VM-based approaches without compromising on isolation. VServer provides the following mechanisms for isolation:

1. Fault Isolation - This is achieved through separation of namespaces (contexts) and access control (filters). Hypervisors also provide isolation through contextualization and filters but apply them at the hardware abstraction layer.

2. Resource Isolation - CPU isolation is provided by the token bucket filter. Tokens are generated based on reservations and CPU shares and decide which VM gets to run. Network isolation is provided through rate-limiting via the HTB queueing discipline in Linux tc. Disk isolation is managed through the Linux CFQ I/O scheduler. Upper bounds on memory and storage can also be set.

3. Security Isolation - VServer provides process filtering by allowing only the kernel access to all the individual process namespaces. It also provides network separation by allowing containers to bind to only certain IP addresses

4. Evaluation
Microbenchmarks show that VServer’s performance is very close to native Linux performance for fork, exec, context switching etc. Its performance is also much better than Xen3, which suffers because of the overheads with virtualizing the virtual memory hardware. Network bandwidth benchmark shows the benefits of sharing the network stack. CPU and disk I/O macrobenchmarks also perform 25-30% better on VServer. Scale tests show that VServer with 8 VMs gives comparable performance to Xen with 2 VMs. CPU and disk I/O isolation tests are also run to provide proof of the isolation capabilities.

5. Confusion
1. Can you provide more info about what the other container-based operating systems were doing? I would especially like to know about the design differences, because some of the choices in VServer (fake init process) seem hacky.

1. Summary
This paper introduces container-based operating system (COS) virtualization method. Specifically, it explains how COS works in two folds, resource isolation (CPU, mem, I/O) and security isolation (proc, net, FS). In addition, COS is compared to VMM (hypervisor) in terms of performance.

2. Problem
The main problem is how to satisfy user case where efficiency trumps the need for full isolation (VMM). VMM abstracts at hardware layer. It's not optimal for VMs running upon same kernel. The full isolation provided by VMM comes with less efficiency, due to trap-and-emulation method. For VMs running with same kernel, COS abstracts at system call layer, to isolate different VMs and have less overhead than VMM.

3. Contributions
The main contribution is how to make a resource and security isolated unit (VM) upon OS layer. It seems like that putting processes into different groups (VMs) is easy, but many detailed problems within resource and security have to be considered.

(1).Resource isolation. CPU must be fair shared among VMs in general case. A simple token-based scheduling is used to enforce fairness. QoS (reservation) can also be quantified by number of tokens one VM has. For network I/O, the Hierarchical Token Bucket (HTB) queuing is used. It enforces network I/O resource fair shared among different VMs, and also supports QoS via tokens. Similarly, disk I/O is shared via completely fair queuing (CFQ) I/O scheduler. To enforce memory and storage usage limitation for each VM, a daemon process (watchdog) is used to monitor resources used by VMs, and take actions when overload happens.

(2).Security isolation. To avoid unwanted interaction between processes across different VMs, OS has to remember each process belongs to which VM. Ideally, each VM has its own process ID space (though not implemented in the paper), to avoid conflict with assumption made by applications for PID (e.g. pstree). For network packet transmission, OS has to remember each packet belongs to which VM, and each IP address belongs to which VM. In this way, OS can forward packets from and to correct VM. File system isolation can be achieved by creating separate root directory of each VM, and the authors introduce a special file attribute - Chroot Barrier to work around the problem that process escapes from current root directory.

To improve efficiency, certain files (e.g. libraries) is shared among VMs by using copy-on-write hard links. Sharing files reduces pressure on storage and improves memory cache.

4. Evaluation
This paper implemented COS inside Linux (VServer), and it conducted comprehensive experiments to compare VServer against a VMM implementation (Xen). The experiments are aimed to compare VServer and Xen for both efficiency and isolation. The evaluation consists of micro-benchmark for OS primitive operations (lmbench), network bandwidth benchmark (Iperf), several file system I/O benchmarks (DD, DBench, Postmark and OSDB), scalability (1-8 VMs on OSDB), CPU isolation (hourglass), and performance isolation (1 VM dd, others OSDB). The experiments show almostly in every case, VServer is better than Xen.

5. Confusion
(1).How chroot works, and why it is designed to allow process escape from the chroot-ed environment?

(2).In Figure 6(a), Xen3-SMP actually has better performance than VServer-SMP, but the authors didn't explain the reason.

1. summary
This paper describes a virtualization approach designed to enforce a high degree of isolation between VMs while maintaing efficient use of system resources.
2. Problem
There is fundamental tension between isolating applications and enabling sharing among them.
- current OSs provide weak form of isolation with generous facilities for sharing
- hypervisors have full isolation bewteen VMs, which heavily favor full isolation over sharing. However, on a single machine, the isolation offered by hypervisors impacts efficiency relative to running all applications on a single kernel."

3. Contributions
The authors claim two contributions
1) first thorough description of the techniques used by Linux-VServer for an academic audience
2) Compare VServer with a recent generation of Xen
- CPU workload: equal performance
- I/O centric workload: VServer has more efficient use of system resources
- scability: VServer far surpasses Xen

4. Evaluation
The authors evaluate the systems based on two metrics: performance, scalability, and tests the three systems VServer, Xen and Linux (as reference) under three cases,
- micro-benchmarks (several system calls). The results show that
Linux-UP, VServer-UP timings are similar within 1%, while Xen3-UP behaves much worse due to the overhead.
system benchmarks, consist of network throughput, macro benchmarks (Disk and CPU intensive operations). The results show VServer outperforms Xen.
- Performance at scale. Number of VMs increases from 1 to 8. VS-UP scales well while Xen3 degrades with the increasing VMs.

5. Confusion
Figure 3. How can applications directly run on virtual machines? Do VMs serve as both VM and OS? Why do we need a shared OS under VMs?
Section 3.3 What is the global PID space? What role does it play in process filtering? Why is this important?
OpenVZ is also container-based systems with features like independent PID space for per VM and fully virtualized networking subsystem. It should outperform VServer. Then what is the meaning of this paper?

1. Summary
A Container-based OS (COS) provides guest VMs with isolation on top of a single shared kernel. By virtualizing the kernel instance, the COS decreases the overheads of virtualization without sacrificing too much of the isolation provided by hardware hypervisors such as Xen.

2. Problem
Traditional VMMs favor full isolation of guests by providing virtualization at the hardware level. However, managing the VMs at the hardware level can lead to large overheads when the VMs and VMM must continually swap out to share the hardware. Some use cases allow for all running VMs to share the same kernel. For these cases, a Container-based OS (COS) allows VMs to be isolated by the kernel itself, leading to much fewer swaps and much lower overheads. Of course, enforcing isolation between VMs using the kernel presents many of the same problems as traditional HW virtualization. Many of these problems can be solved simply and flexibly in software.

3. Contributions
The authors present a way of thinking about different virtualization techniques: a tradeoff between isolation and efficiency. They identify VMs as going too far in the direction of isolation, and present COSs as a viable middle ground for use cases that can tolerate some relaxation in isolation.
They identify issues present in both traditional hypervisors and COSs and compare how the two solve problems differently.
Using benchmarks, they show that COSs are much more performant than hypervisors in certain cases.

4. Evaluation
They compare a COS, VServer, with the more traditional Xen hypervisor. Both run on the same hardware. The Xen hypervisor is configured to rely on a host VM instance to interact with the hardware, exacerbating some of the overheads for Xen.
They first show the overheads of Xen versus VServer for microbenchmarks that perform a lot of memory operations and syscalls. These tests show VServer has almost no overhead compared to a native kernel and Xen performs significantly worse. They also run some classic system benchmarks and come to similar conclusions.
They test the system’s scalability by running multiple copies of OSDB on each. Once again, they see VServer outperform Xen. This section has the interesting result that VServer increases throughput performance running 2 instances instead of 1, while Xen’s performance degrades. Again, they largely attribute this to swapping overhead.
Finally, they present a test that shows VServer’s direct and precise control over the scheduling mechanism allows them to nearly exactly hit a cpu-time-sharing target, while Xen does a much worse job.

5. Confusion
I was confused by the “CPU Fair share” section. What is the difference between what VServer does and what Xen does to time-multiplex the CPU? Why is there such a significant difference? Is it just due to swap overhead?