CS 736 Reviews - Spring 2017

1.Summary:

Singularity project incorporates new design ideas and for building an operating system to overcome the limitations of the conventional operating systems and improve the robustness of the operating systems. The three key architectural features based used in design of the Singularity Operating
system are Software Isolation Processes, contract-based channels and the Manifest based programs. This project serves as an experimental platform for building dependable systems.

2. Problem:

The shortcomings of the contemporary OS are rooted in the design decisions and include security vulnerability, unexpected interactions with applications, failures caused by errant extensions, plugins, drivers and lack of robustness. There hasn’t been significant reengineering of the operating system to benefit from the advancements in the programming languages and the verification tools and to cope up with the changes modern day computers are used.

3.Contributions:

Singularity OS makes use of safe programming languages, good program verification tools to eliminate the preventable defects and enables early detection of errors.
The three key architectural features are software isolated process, contract based channels and manifest based programs.

SIPs are similar to processes in the conventional OS expect that they rely on the programming language type and the memory safety for isolation, instead of memory management hardware. Contract based channels provides inter SIP communication. Channels together with contracts helps in reducing bugs while also providing a clean separation of concerns and hence a better understanding of complex system architecture of the OS.

Manifest provide reliable way to check the expected behavior of programs while also specifying the required system resource requirements and dependencies of the program. Program is run by invoking the corresponding manifest which contains information about the required resources, capabilities and the dependencies of the program with other programs. 

Singularity OS has a microkernel which provides the minimum functionality such as the abstractions of SIPs, contract channels, scheduling and the memory management. SIPs communicate with the kernel through the ABI which provides an interface to the kernel.

4. Evaluation

The paper provides little evaluation of improvement in the robustness offered by the Singularity operating system neither does it suggest metrics on which a comparison can be made. It does not provide a thorough evaluation of its design choice in comparison with that of conventional OS.

5. Confusion

1. Dynamic code inspection seems to be pretty powerful but at the cost of code optimization. What alternative techniques do conventional OS’s make use to solve the problem of dynamic code inspection and similar such problems?

Summary : Design choices of current OSs are rooted in 1960s-70s era and suffer from pervasive security issues and system failures. Singularity projects aims to build a reliable and secure OS by exploiting the latest advances in computer science (especially architecture and programming languages). Fundamental to it are three design decisions – Software-isolated processes for safety, contract-based channels for communication and manifest-based programs for system properties verification.

Problem : Current operating systems share a lot of design decisions which can be attributed to their inspiration from systems designed in 1960-70s. These designs haven’t kept pace with latest technological advances. Such systems also possess security issues, application communication bugs, and system failure issues. Although usage pattern and user requirements of computers have changed significantly, system architecture of current Oss hasn’t changed accordingly.

Contributions : 1. Sing # - A high-level language which is very useful for static verification.
2. Introduced Software-Insolated Processes (SIPs) which are isolated by low cost software verification unlike traditional OS processes that require expensive hardware protection.
3. Contract-Based Channels – resourceful and analyzable communication between SIPs. Allows static verification by Sing#.
4. Manifest-based programs: facilitates static and runtime verification of program and its properties.
5. Exchange heap (in lieu of shared memory) which facilitates low cost ownership transfer of data.
6. Exploring the idea of making specialized programmable processors core to OS scheduling and computing abstractions.

Evaluation : The paper does a good of introducing a novel approach to OS design, improving safety and robustness. It however doesn’t provide a quantitative measure or detailed qualitative measure of improved reliability and security. Also, variety of quantitative measures of performance improvement (except in case of software and hardware isolation) are missing.

Confusion : 1. Scheduling described for threads is confusing.
2. If users are allowed to modify manifests, can’t someone inject malicious code in manifest to compromise security?

1. Summary
Singularity is an experimental operating system designed by Microsoft Research in an effort to re-evaluate legacy design decisions and explore new ones. The operating system has three core ideas: software-isolated processes, contract-based channels, and manifest-based programs.

2. Problem
One of the beautiful things about this project is that they did not set out to necessarily tackle one particular problem such as the overhead of communication, but were empowered to re-evaluate the operating system in today’s context. WIth that being said, central goal was to create a provably reliable, robust system.

3. Contributions
The contribution is a micro-kernel that embodies the three core ideas mentioned above. Software-Isolated Processes pushes the idea of a single address space for performance and locks down what can be run to allow the elimination of hardware isolation. It comes at a rather steep price of what they call sealed code spaces. No dynamically generated code can be executed and everything relies on message passing via channels described next.

Contract-based Channels is rather similar to the idea presented in the U-net paper by Vogels et al. while at Cornell. It uses the idea of message queues and a pinned set of memory to facilitate zero copy. These channels also go one step farther and define state machine semantics that require programmers to identify all cases ahead of time as to push safety earlier into the equation rather than reveal these scenarios later.

Third, Singularity uses a manifest at program creation to verify things such as code resources, required system resources, capabilities and other dependencies. The main use here is to allow static and dynamic verification of properties of the program.

Other ideas are explored such as heterogeneous multiprocessing with the advent of many-core systems and dynamic specialization, the placement of programmable I/O processors, and typed assembly language to generate proof-carrying code.
Additionally, Garbage Collection was identified as a major concern with the use of a type-safe language. A deeper evaluation of the impact of this would be interesting.

4. Evaluation
Evaluation should be taken in a different light as the goal was stated to be “good enough.” They were able to begin evaluation of other techniques only made possible by this micro-kernel.

Given evaluations include the following. Table 1 demonstrates that there is a significant overhead cost that we may be paying towards other areas that can be reclaimed. Figure 5 shows the performance of adding features to facilitate unknown code.

5. Confusion/Questions
It feels like their implementation is at the crossroads of flexibility and scalability. My understanding is that a significant portion of OS code is related to drivers and such. Where does this tradeoff exist from a production perspective as they acknowledge the decision for research freedom?

The lack of JIT is a drawback of the design. How impactful is the lack of JIT from a performance and productivity perspective as a tradeoff to the verifiability of the software?

1. Summary
Though there have been advances in programming languages and verification tools, most design choices in current O.S. have remained untouched. This paper tries to capture Singularity Project’s efforts to re-examine these conventional design choices and explains the key architectural design features and their implementation.

2. Problem
Contemporary operating systems share a number of decisions which are rooted in similarities in their operating environments, programming languages and user expectations. Many of these have not evolved and leveraged the advancements in computer architectures and programming languages which has led to a number of obvious shortcomings. Singularity project re-examines the shortcomings of these O.S. which include – security vulnerabilities, aberrant communication between applications, inherent vulnerabilities of extensions and lack of robustness.

3. Contributions
To re-think the operating systems rather than to evolve it from the traditional designs, Singularity project has built operating system from scratch. To facilitate this “ground up” approach they have also built a new programming language - Sing# and new software verification tools.
Singularity system introduces three main architectural components:
1. Software-Isolated Processes which ensure the protection of programs
2. Contract-Based Channels which facilitate fast, reliable message-based communication
3. Manifest-Based Programs for verification of system properties.
With the introduction of SIP, the process safety is enhanced, inter-process communication is eased and the overheads are lowered. Contract-Based Channels guarantee a robust communication mechanism defined by contracts and enable zero-copy exchange between SIPs. Another notable feature in Singularity is that the kernel and SIP object spaces are garbage collected. Compile-Time Reflection is implemented in Singularity for generative programming. This helps in writing inspection code, constructing bolier-plate code and automating configuration of device drivers. Hardware Protection Domains provide flexibility in the implementation of hardware protection, thus helping in defining the architecture of the system. Also the implementation of SIPs inherently helps in dynamic specialization of processors.

4. Evaluation
The paper provided only a minimal and insufficient amount of evaluation. The only real evaluations presented are process costs in comparison with other Operating systems and the comparison of costs of hardware and software isolation. The paper makes many claims and assumptions with no real proof. For example, it states that the results are so far promising for the construction of an O.S with only SIPs, but that sentence is all we have to it. Also the authors keep postulating many hypothesis throughout the paper, without any strong support or further discussion. For example, they state that MBPs will play a role in significantly reducing the costs of system administration. Again there is no strong explanation to back it or any experimental results pointing to it. Given that Singularity was still work in progress and just a research prototype at that time, this is quite understandable. But the authors should have provided strong empirical proof at least to evaluate the strengths and weaknesses of the 3 building blocks of Singularity System.

5. Confusion
• How is memory virtualization implemented in the Singularity System?
• How exactly is scheduling achieved via two lists? The explanation is confusing in the paper.
• How is smaller code footprint achieved by using SIPs? (mentioned in Heterogeneous Multiprocessing)

Summary

Singularity can be thought as a thought experiment which reexamines new design decisions accommodating advances in modern operating systems. The primary goal of Singularity is to provide reliability and robustness through three main features SIP, Contract based channels for communication and manifest based programs for verification of system properties.

Problem

Singularity addresses many problems persist with contemporary operating systems like security vulnerabilities, unexpected interactions among applications, failures caused by errant extensions, plug-ins and drivers, and a perceived lack of robustness. The second thing is performance lost in Hardware isolation in these contemporary operating systems. (expensive special instructions like interrupts)

Contributions

One of the key contributions of Singularity is software Isolated Processes. Most of the processes share the same address space as Kernel’s. Software verification and language safety ensures that SIP code can’t abuse the kernel privilege under which it runs. This is huge performance benefit over contemporary architecture of kernel space and user space, where for user program to get to privileged access, there needs to be a kernel crossing mechanism (interrupts).

The other main thing is new way of inter-process communication (between SIPs) in Singularity flows through Contract-Based Channels (Bidirectional message conduit) which provides a clean separation of concerns between interacting components. It is defined by state-machine-like contracts whose specifications are verified by the compiler.

Singularity also provides type checking to process creation through Manifest-Based Programs. It defines a bunch of checkable program properties that the Singularity kernel can verify before starting a process.

Evaluation

Singularity is evaluated on it’s performance. Though there is no solid evidence over it is performing better overall than other conventional OS. The main evaluation showed concentrated on performance benefits of Software Isolated process avoiding all kernel crossings in process of process creation.

Confusion
1. It is unclear how Singularity does runtime checks.
2. Who writes manifest for MBPs?
3. The behaviour of scheduler is a bit confusing when it comes to handling unblocked list and preemted lists.
4. It is not clear how Singularity achieves memory virtualization.

1. summary
This paper is about building a new operating system that accounts for programming language and verification tool improvements that have arisen since the initial creation of the more common modern operating systems.
2. Problem
The main problem that is addressed by the Singularity system is that of a lack of security that arises from the design environment of the 60s and 70s in which the more common operating systems were initially conceived. The very limited environment of that period lead to a range of vulnerabilities and unexpected interactions as systems became more feature rich over the years and capabilities were incorporated into the existing design.
3. Contributions
The Singularity system addresses these problems by redesigning an operating system from the bottom up with the idea of heavily enforcing that all programs are entirely isolated and verified to not perform any unexpected interactions outside of their intended functionality. To do this, processes are designed as software-isolated processes which have an entirely enclosed address space which no other process may interact with and outside of which the SIP is not able to interact. Whereas hardware memory management constructs are typically used to enforce similar behavior, SIPs do so through programming language type and memory safety methods. SIPs are only able to interact with other processes by means of contract-based channels. A channel is exclusively between two SIPs and is defined by a set of protocol states and message declarations. To enforce intended behavior within programs, each program it run using a manifest that describes which resources and dependencies the program has which can be used to verify the program meets the requirements of the system.
The strictly enforces isolation of the different processes allows for a very minimal kernel with device drivers and similar being instead run as their own SIP which can then be connected to using a channel to interact. It also allows for trusted code to be able to be run in a privileged level avoiding some of the overhead in context switching and allowing access to privileged instructions if needed.
4. Evaluation
The evaluation of the system that’s provided is somewhat lacking. It is explicitly stated that performance was not a main goal of the project and yet they spend time evaluating some performance metrics. An evaluation which uses only a single benchmark in their comparison. I would have preferred to see an evaluation of how truly isolated the processes were. Many of their design decisions are based around processes definitely being isolated but I would like to see some form of analysis of how true that is if someone is intentionally trying to violate that isolation. Seeing as they don’t use some of the hardware mechanisms that are in place I would be interested in seeing just how wrong it could go since some programs are run at kernel level privilege.
5. Confusion
I am mostly unsure about how much can be guaranteed by a verification tool. Everything seems to depend on them being infallible in their checks which I find hard to believe and once that’s not guaranteed it seems many of their system could easily be broken.

1. Summary: Singularity project is an effort towards building a dependable and trustworthy system from scratch by leveraging safe programming language and sound verification tools to eliminate some of the shortcomings of existing systems.
The Singularity system is based on three key architectural features: software-isolated processes (SIP), contract-based channels and manifest-based programs.

2. Problem: Traditional operating systems like Windows, linux , Mac OS X still hold on to the design decisions made during early development of operating systems and have not evolved with the advancements in hardware and software. This results in shortcomings such as security vulnerabilities, unexpected interaction among applications, driver failures etc. Thus, the singularity project realized a need to re-examine the design decisions and build a more robust system.

3. Contributions: The main contribution of singularity is the prototype of a reliable and robust system with extensive static analysis to detect programmer error early in the development cycle. This is realized by using three key features:
> SIPs, which are sealed processes with no dynamic code generation. It takes advantage of the type and memory safe programming language Sing# to eliminate the cost of hardware protection.
> contract-based channels allow message-based communication between SIPs and facilitate transfer of data through dedicated memory called exchange heap. The code is statically verified whether it conforms to the state machine described in the contract’s protocol.
> Every program is associated with a manifest which describes its resource requirements and dependencies on other programs. This manifest allows static and dynamic verification of the program.
Singularity doesn’t allow sharing pointers to object between SIPs to ensure security. This further enables the SIPs to employ its own independent garbage collector.

4. Evaluation: The paper lacks a evaluation to prove that the system in now more reliable and robust which is the main goal of the project. It does evaluate the execution time for WebFiles benchmark in different configurations of hardware and software isolation to show that hardware isolation can significantly slow down the execution. But this limited evaluation is not convincing enough to say that software isolation is always the best choice.

5. Confusion:
> Paper implies that eliminating hardware protection eliminates cost of maintaining page table. If so, then how does the address translation happen?
> All SIPs share single address space - not clear how are addresses are assigned.
> Who writes the manifest ? programmer or generated by the compiler?

1. Summary
This paper talks about Singularity operating system which was designed from scratch using safe programming language and verification tools. It was aimed at providing more robust, dependable and trustworthy systems. Singularity has three key architectural features: software isolated process (SIP), contract-based channels and manifest-based programs (MBP).

2. Problem
In spite of the advances in programming languages and verification tools, operating systems today are still using some of the design decisions that were made during 1960s and 1970s. These decisions have some shortcomings like widespread security vulnerabilities, unexpected interactions among applications and lack of robustness. Singularity was aimed to make use of advances in computer software and hardware, safe programming languages and verification tools to build an operating system that would overcome the shortcomings present in the current operating systems.

3. Contributions
This paper proposes three architectural features: software isolated process, contract-based channel and manifest-based program.
Software isolated process: They provide an environment for program execution protected from external interference. SIPs do not share any writable memory with any other SIP. This isolation is provided by software rather than hardware and thus reduces the complexity of managing hardware. The communication between SIPs occur through message passing channels. SIP's code is sealed at execution time which allows program analysis tools to detect defects statically. Exchange of data occur in a special area of memory known as exchange heap.
Contract based channels: The communication between SIPs flows through contract-based channels which enable efficient and analyzable communication. Channel contract consists of message declaration and a set of named protocol states. Channel contracts are valuable tools for preventing and detecting mistakes.
Manifest based program: For each program to run on singularity, it has to provide a manifest which describes MBP's code resources, its required system resources, its desired capabilities and its dependencies on other programs. Manifest allows static and dynamic verification of properties of the MBP. For example, type and memory safety, absence of privileged mode instructions and conformance to channel contracts.

4. Evaluation
The paper provides a brief comparison between the execution times for the WebFiles benchmark in six different configurations of hardware and software. The graph in figure 5 shows that the overhead paid by programs running in systems built for unsafe code could be very high and the overhead for runtime for safe code is significantly lower. No other evaluation is provided by the authors and it would be difficult to claim that Singularity is better than other operating systems based on only one evaluation.

5. Confusion
Paper states that SIPs can be created without creating page table and flushing TLBs. It also states that pages for a SIP need not be in contiguous memory location. Then how is virtualization actually achieved here? Does the process has pointers to all blocks of memory that it can access?
Who provides the manifest? If it is provided by the user then how does Singularity prevents malicious programs from lying about their resource requirements?

1.Summary: This paper is about redesigning existing operating system for current state of the art resources like multi-core CPUs, high speed cheap memory, etc. The new design aims to incorporate ease of robust software development and software verifiability without compromising performance.
2.Problem:The current operating systems rely heavily on hardware based protection mechanism, which require frequent switch to kernel mode from user mode degrading performance. The current operating systems do not provide a clean way of handling run-time errors. This not only makes it difficult to debug issues, but degrades user experience.
3.Contributions: The main focus of the paper is to design restrictive Operative System which provides abstractions that ease development of robust applications. The three main abstractions provided are – Software Isolation of Processes (SIP), Contract based channels for inter-process communication and Manifest based programs (MBP) for security. The SIP requires static linking of dependencies and provides software isolation from other SIPs instead of hardware protection. This eliminates the need of traditional virtual memory addressing. Thus, multiple SIPs can run in same address space with isolation guaranteed by verification tools at compile time. The address space is shared by SIPs in one protection domain. SIPs cannot share writable memory except the Exchange HEAP. SIPs access primitive kernel functions via Application Binary Interface (ABI). This eliminates the complexity of virtual memory addressing and gains performance. The SIPs can perform IO or inter-process communication via Contract-based Channels. These channels facilitate message transfer pre-defined rich interface. Channels can provide zero-copy communication within a protection domain. Channels also ease error detection at compile time with type safety support in the OS itself. MBP play a very crucial role in ensuring robustness of the applications and important in compiling, installing and customizing the software.
4.Evaluation: The paper describes an empirical evaluation of the new design by comparing execution times for IO intensive application. The readings state that new design beats existing virtual memory addressing architecture by almost 38% performance gain. The paper has not compared performance for any device driver. Usually device drivers need to be extremely fast to completely utilize the hardware. The paper should have covered at least one such scenario.
5.Confusion: How does Singularity Architecture achieve software protection for Invalid Memory references? How does Singularity provide memory over-committing? Paper has not discussed the increase in compile time of applications due to extensive type safety checks.

1. Summary
This paper covers the Singularity operating system. Singularity as an operating system has three core features of the architecture of the system which are software-isolated processes, contract-based channels, and manifest-based programs.

2. Problem
Singularity as an operating system started as a way to research shortcomings of current systems such as security vulnerabilities among others and to investigate the software isolation of process protection. The authors make an argument that existing systems suffer from these problems as a result of the them having "not evolved far beyond the computer architecture and programming languages of" the 70's. They propose an OS based on Sing#, which is an extension of C#. This OS is based on software isolation of processes.

3. Contributions
Provides insight into the theory of software isolation of process protection . Shows that designing a system in this manner can eliminate many errors early on in the design and implementation of the system. Singularity uses software verification to ensure security.

4. Evaluations
The evaluation of Singularity seemed to be lacking. It appears that the focus of this paper was more to introduce the concepts that the group was working on rather than to compare perform to other OS.

5. Confusions
How does the system manage memory without the mmu and tlb?
Going over the manifest-based programs a little more would be helpful

1. Summary
The paper discusses “Singularity”, an OS which aims to improve dependability and trustworthiness through the use of safe programming languages & program verification tools. It uses these tools to provide software isolation between processes, verify the communication channels, and additionally uses manifest-based programs for system verification.

2. Problem
Existing OSes have several shortcomings which include security vulnerabilities, unexpected failures and a perceived lack of robustness. Many of these problems can be attributed to the failure to evolve beyond the programming languages and systems of the 1960s and 1970s. The authors argue that many recent advances in the programming languages and program verification fields can be leveraged to build a more dependable OS.

3. Contributions
1. Demonstration of Software-isolated Processes (SIPs) as a low-cost alternative to hardware protected processes without reducing safety. SIPs can be created without creating page tables or flushing TLBs, and thus have low overheads. Because of its low costs, SIPs can also be freely used to load extensions and thus avoid unprotected dynamic code loading.
2. A contract-based channel abstraction for communication between processes. It allows for analyzable communication through static analysis. This is extremely useful because communication in traditional systems is impossible to analyze without such abstractions.
3. The MBP can be used as a mechanism to provide static and runtime verification of programs.
4. Far superior IPC performance through zero-copy transfers which are made possible through the exchange heap and the use of contracts to transfer ownership of data.
5. Other contributions such as the Compile Time Reflection used in MBP and the flexibility of choosing a custom garbage collector system for a given SIP.

4. Evaluation
The authors mention that that primary goal is improving dependability. Yet, no empirical evidence is shown to demonstrate the increased dependability. Admittedly, dependability is a tough property to quantify. Even then, a crude quantification could have been possible. Eg. Scraping through existing bug reports in Linux and classifying how many failures could have been avoided through the use of safe languages.

The paper does demonstrate the overheads of providing hardware isolation in Singularity. This shows that having simpler hardware with software-based isolation might result in better performance. Simplifying hardware requirements can help the development of special purpose I/O processors by doing away with memory management units and cache coherency.

The authors also mention that their system achieves superior performance in the shared communication micro-benchmarks, which is expected because of its ability to do zero-copy message transfers. This does seem a little out of place because performance is not listed as the primary goal of the project.

5. Confusion

The contract-based channel abstraction requires explicit message and type declarations to allow static verification. Can this requirement restrict the type of network communication that can happen?

What has stopped these ideas of safe programming languages and program verification from becoming used in mainstream OSes?

1.Summary:
Singularity is an operating system designed as a part of Microsoft research project to overcome the limitations of the conventional operating systems and make the OS more secure and robust. The OS was designed mainly based on the three key architectural features. Software Isolation Processes(SIP), contract-based channels and the Manifest based programs.

2. Problem:
The OS has not been redesigned from the scratch to take the advantages of the advancements in the programming languages and the verification tools. The conventional OS suffers from lots of limitations which include security vulnerability, unexpected interactions with applications, failures caused by errant extensions, plugins, drivers and lack of robustness.

3.Contributions:
The whole idea behind this project was the use of safe programming languages, good program verification tools eliminates the preventable defects and enables the error to be detected in the earlier stages of development and also a good system architecture stops the propagation of errors to the run time.
The three key architectural features were software isolated process, contract based channels and manifest based programs. SIPs are similar to processes in the conventional OS expect that they rely on the programming language type and the memory safety for isolation, instead of memory management hardware. SIPs communicate with the other SIPs by using contract based channels, which are bi-directional message conduit with exactly two endpoints. The data passed to other SIPs by placing it on the exchange heap and sending the pointer to it to the other SIP.
Every program in singularity is associated with manifest. Program is run by invoking the corresponding manifest which contains information about the required resources, capabilities and the dependencies of the program with other programs.
Singularity kernel is the microkernel which provides the bare minimum functionality such as the abstractions of SIPs, contract channels, scheduling and the memory management. The device drivers, network protocols, file systems etc execute outside the kernel in a SIP. ABI gives the interface for the SIPs to communicate with the kernel.

4. Evaluation
The paper has demonstrated the overhead introduced by the hardware isolation level by level. Apart from that, the paper does not evaluate its every design choice with the conventional OS and verify if the choice made was better than the conventional one.

5. Confusion
The paper does not give the clear picture of how the memory is virtualized in this architecture. As in which component here does the job of mapping virtual to physical address spaces.
Why are hardware protection domains necessary when the SIPs themselves can provide the isolation. Is there any case where the software can not provide the isolation on its own and needs the support of the hardware to do that?

Summary

Several design decisions for building operating systems have remained unchanged for decades. This project attempts to redesign some of the major common mechanisms in systems today.

Problem
Singularity project reexamines various design decisions which have remain unchanged for decades. This is required because modern software stacks have many shortcomings. The following are some of them:
Widespread security vulnerabilities; unexpected interactions among applications; failures caused by errant extensions,plugins and drivers; and a perceived lack of robustness.

Major goal of the project is to provide a platform upon which dependable and verifiable system can be built.

These problems never existed during the development of first systems in 1960s and early 1970s because computers were not nearly as powerful, they were limited to harmless technical researches of the field so security was not a big issue and there not as many programming languages as now.

Contributions

OS is written in higher level language so that it can take advantage of the type and memory safety features of modern languages.

The three main architectural differences between Singularity systems and regular OSs:

SIPs - Sealed Processes; Isolation through software; easier context switching, communications between SIPs and inexpensive to create and destroy because SIPs share same virtual address space and the usual overheads of TLB flushing, Page Table creation and GC are absent.
Contract based channels
Manifest based programs - Main purpose of a manifest is to enable static and dynamic verification of MBPs.

The implicit argument is that dynamically linked libraries, self-modifying code, and shared memory are inherently dangerous and should be eliminated.
The idea of restricting the pointers in a SIP to object spaces entirely inside the SIP was impressive. This makes GC flexible and easy to implement.

Evaluation

The paper did not make an attempt to quantitatively evaluate its stated goals of reliability and robustness. Instead it attempted to evaluate speed at various hardware isolation levels. The paper also claims the system to be “no worse than” other popular widely systems. But Operating Systems are expected to perform reasonably fast anyway. Without a significant gain in speeds my opinion is it is not worth mentioning.

Confusion

The paper talked about using Compile Time Reflection for dynamic inspection of existing code and to generate new code but was not clear on how are these achieved.

1. Summary
Singularity is an operating system that prioritizes dependability and trustworthiness in its design. The authors accomplish this by coding most of the system in a custom language, Sing#, and by introducing the new constructs of software-isolated processes, contract-based channels, and manifest-based programs.

2. Problem
Many previous operating systems had security vulnerabilities, and unsafe user programs could interfere with other programs. In addition, most operating systems in 2003, when the Singularity project started, maintained the design decisions made by early operating systems in the 1960's and 1970's, even though hardware and user constraints had changed.

3. Contributions
The authors implicitly note that users usually intend for their programs to be safe, and that operating systems should help with this case. The authors want to show that it is possible to verify the correctness of an operating system, assuming that certain tools in the development of the system are also correct, and for this operating system to run only safe user programs.

Singularity is implemented in a custom programming language, Sing#, which enables developers to write low-level code and verify its correctness. User code runs in software-isolated processes (SIP's), which provide type and memory safety and only allow safe programs to run. SIP's communicate through contract-based channels, which allow efficient IPC and ensure (through static analysis) that both endpoints are always in valid states. All user programs are manifest-based programs. The manifest specifies the resources and dependencies of the program, and it allows the system to reject programs that are unsafe.

4. Evaluation
The paper does not have much evaluation. Figure 5 shows, surprisingly, that Singularity had better performance than other operating systems on the WebFiles benchmark, even though the main goal was dependability, rather than performance. The paper would have been better if it quantified the work required to build Singularity (in units of, say, man-years). If this paper was really part of "A Journey, not a Destination", then hopefully future papers included more complete evaluations.

5. Confusion
The purpose and implementation of compile-time reflection (Section 4.1) were very unclear. The paper introduced the idea of transforms, but did not explain them adequately.

Singularity Review
1/23/2017

1. Summary
The paper presents the Singularity system, a system that aims to address some shortcomings of existing systems and software stacks that has been caused by inheriting system design decisions back in the 60's and 70's. The three key architectural features, as described by the authors, are: "software-isolated processes for protection of programs and system services, contract-based channels for communications, and manifest-based programs for verification of system properties."

2. Problem
Many modern operating systems shares the design decisions that was made 40 or 50 years ago where the environment of computing were dramatically different. Some of the differences, as Hunt and Larus mentioned, were computers were very constrained by speed and memory capacity, were only used by benign specialist and were rarely connected to network or physical devices.

By inheriting the design decisions, many systems exhibit shortcomings including "wide-spread security vulnerabilities; unexpected interactions among applications; failures caused by errant extensions, plug-ins, and drivers, and a perceived lack of robustness".

3. Contributions
The three architectural features, software isolated process (SIP), contract based channels (CBC) and manifest based programs (MBP) contributed some fresh ideas and reconsiderations to system design.

SIP
SIPs serve as placeholders of processing resources and provides the context for program execution. Comparing to processes in many other systems, their data sharing is very restricted. As the authors mentioned, SIPs do not share data, the channels handle communications instead. Hunt and Larus also mentioned that SIPs do not share writeable memory with other SIPs and the code within a SIP is sealed at execution time. SIPs reduce the cost of isolating safe code by taking advantage of the type and memory safety of modern programming languages.

CBC
As the authors described, "a channel is a bi-directional message conduit with exactly two endpoints. A channel provides a lossless, in-order message queue". It turned out that channel contracts can do a great job preventing and detecting mistakes.

MBP
"A manifest describes an MBP's code resources, its required system resources, its desired capabilities, and its dependencies on other programs", as described in the paper. The main reasoning of having manifest is to allow static and dynamic verification of properties of the MBP.

I think the strategies mentioned in the paper, using safe programming languages, using sound program verification tools and stopping the propagation of runtime errors at well-defined boundaries also brings new considerations into system design.

4. Evaluation
I like how the authors address the problem that many classic design decisions might introduce hazard nowadays, especially because computers are much more connected than they were three or four decades ago. The question raised by the authors "what would a software platform look like if it was designed from scratch, with the primary goal of improved dependability and trustworthiness" is interesting and important.

The Singularity system addresses dependability more than performance. This design principle can be seen in many of the design decisions made, where safety is assured by introducing some overhead.

The evaluation portion within the paper could be more comprehensive. It used one set of benchmark but not more which might introduce bias into the evaluation process. But it is a good thing that the author also pointed out that the Singularity project is a journey but not a destination. Looking forward to seeing future works will help Singularity project to bring more insight to system design.

5. Confusion
a. As the author mentioned, "A key experiment in the Singularity project is to construct an entire operating system using SIPs and demonstrate that the resulting system is more dependable than a conventional system." What are some of the way to compare dependability?

b. What are the security principals mentioned in section 3.6?

c. How is a manifest generated for a program?

1. Summary
The paper discusses building a modern operating system from scratch making use of advances in programming languages and verification tools. The paper tries to side-step shortcomings of popular operating systems by using guarantees provided by software and verification using static analysis tools. The goal is to build a dependable and trustworthy operating system.

2. Problem
Most of the operating systems that are widely used today have their roots in languages, tools and hardware from the 1970's. Computers are now used very differently compared to those eras, but operating systems have most of their design decisions rooted in the past. They have security vulnerabilities, lack reliability and are not very robust. Since they form the foundation of almost every software stack, these shortcomings have a wide impact.

3. Contributions
The paper tries to build a software stack from the ground up for modern needs by leveraging advances in other fields of computer science. Widely used operating systems use languages such as C for systems programming since they provide easy access to low level hardware features. But they don't provide type safety and memory safety which can lead to runtime errors due to bugs. The paper shows that it is possible to implement operating systems in languages that provide safety guarantees, such as Sing#. Using the properties of such a language, processes are isolated using software. Communication between processes happen through channels based on contracts which can be verified statically. Also, a program can be checked for correct behavior before it is even run. Even though the goal of the system is to improve reliability, the ideas help in improving performance as well. For example, privileged instructions can be in-lined in processes since safe execution of code is guaranteed, thus avoiding the overhead of system calls. Since isolation is provided by software, processes can exist in the same address space and thus avoid expensive operations such as changing page tables and also TLB invalidations during a context switch.

4. Evaluation
The paper provides a few examples to illustrate the architectural features of the system which it argues provide reliability and isolation. However, only an anecdote is shared demonstrating their claims. Very less information is provided about the actual system built by the authors and its capabilities. The authors claim that the operating system was built for better dependability, but the paper does not provide information on how it achieves this goal for real-life workloads where this is especially a major concern.

5. Confusion
Since code cannot be changed once loaded, is it impossible to run JIT compiled code on a system like Singularity?

1. Summary
The paper presents Singularity a new research idea that develops an operating system using modern knowledge and software. The main part of the paper revolved around security and isolating processes.

2. Problem
The main problem presented in the paper is that current operating systems are legacy software that was first built in a time when there were few users and little need for security. Singularity hoped to solve the security issue in a more efficient way with its Software-Isolated Processes.

3. Contributions
Some of the more interesting aspects of the paper were the discussions on how they isolated specific processes but still allowed for a safe way of communication. All communication between processes is through contract-based channels. These contracts act as state machines that are conditional based on the messages being passed. A Software-Isolated Process can only transfer data through the exchange heap and a process only has a pointer to the item in the exchange heap. A piece of memory in the exchange heap may only be owned by one process at any one time. Another security feature is the manifest-based program which requires programs to have a manifest which describes its required resources, capabilities, and dependencies on other programs. The paper also briefly mentions sing# an extension of C# that adds in some OS support along with support for strong system programming.

4. Evaluation
It does not seem that the quantifiable portion of the paper was the main objective. Instead the paper focused on presenting and justifying new ideas. The paper tried to make it seem that this software approach was superior to the previous hardware isolation approach since it got rid of TLBs and the associated overheads. The paper did report on one benchmark using their own Singularity system to show increased overhead. However, just using one benchmark to prove your case is not very conclusive.

5. Confusion
The manifests seem odd and I was wondering how they are created to make sure that they are trustworthy and not lying about what resources they use or capabilities they can achieve. I was also confused about if or how dynamic code works with manifests.

1. Summary
The authors present a managed code based operating system developed by them. The work is presented as a research effort from the ground up to provide a verified and dependable OS enabled by the advances in technology and theory such as managed runtime languages (java, c#). The main goal is to show that this can be done and as shown by the authors prove to be faster and scalable than traditional OS's.
2. Problem
a. Advances in languages and practices being ignored by main stream OS, which tended to be written in C or assembly leading to a host of security related issues such as buffer overflows.
b. Provide a guideline and identify tools to develop an OS using a managed runtime language that can be verified
3. Contributions
This paper has many small contributions which if observed from today's perspective can be seen as major directions for the OS community. These contributions are:
a. Mechanisms for software based isolation of processes dependent on the compiler.
b. Well defined IPC mechanism in the form of fixed endpoint channels which use exchangeable buffers present in a comman global heap.
c. Verifiable way of introducing new programs and device drivers in the form of Manifest based Programs, which form the basis of Software-Isolated Processes.
d. Garbage collection inside the kernel.
e. Impact of various hardware isolation mechanisms on program execution time.
f. A pathway for typed assembly language.
g. A good prediction and outline of a solution for developing Operating Systems for heterogeneous ISA architectures.
4. Evaluation
I feel the authors could have provided with more detailed evaluation of the system than just providing a runtime comparison for some important OS functions between conventional operating systems and singularity. It would have been nice to see some actual interactions in an unsafe environment with malicious code to test the durability of the system.
5. Confusion
The authors mention the the Reflections API and their simplified implementation called transforms. I was confused by the requirement of such a feature in an OS.

1. Summary
The authors present a managed code based operating system developed by them. The work is presented as a research effort from the ground up to provide a verified and dependable OS enabled by the advances in technology and theory such as managed runtime languages (java, c#). The main goal is to show that this can be done and as shown by the authors prove to be faster and scalable than traditional OS's.
2. Problem
a. Advances in languages and practices being ignored by main stream OS, which tended to be written in C or assembly leading to a host of security related issues such as buffer overflows.
b. Provide a guideline and identify tools to develop an OS using a managed runtime language that can be verified
3. Contributions
This paper has many small contributions which if observed from today's perspective can be seen as major directions for the OS community. These contributions are:
a. Mechanisms for software based isolation of processes dependent on the compiler.
b. Well defined IPC mechanism in the form of fixed endpoint channels which use exchangeable buffers present in a comman global heap.
c. Verifiable way of introducing new programs and device drivers in the form of Manifest based Programs, which form the basis of Software-Isolated Processes.
d. Garbage collection inside the kernel.
e. Impact of various hardware isolation mechanisms on program execution time.
f. A pathway for typed assembly language.
g. A good prediction and outline of a solution for developing Operating Systems for heterogeneous ISA architectures.
4. Evaluation
I feel the authors could have provided with more detailed evaluation of the system than just providing a runtime comparison for some important OS functions between conventional operating systems and singularity. It would have been nice to see some actual interactions in an unsafe environment with malicious code to test the durability of the system.
5. Confusion
The authors mention the the Reflections API and their simplified implementation called transforms. I was confused by the requirement of such a feature in an OS.

1. Summary
This paper presented Singularity Project, which leverages three new features software-isolated processes, contract-based channels to build safer and more reliable OS than conventional ones.

2. Problem
Authors argue that traditional OS has widespread security vulnerabilities caused by unexpected interactions between processes, buggy drivers, plugs-in and failures caused by errant extensions, since its architecture are inherited from OS design in 1960s and 1970s. OS designer need to rethink about ways of building more reliable and secure modern OS.

3. Contributions
The paper proposes three new features for building secure operating system.
The first feature is Software Isolated Process (SIP). Unlike traditional OS which relies on hardware memory management techniques to deliver memory protection, SIP provides secure memory management by restricting process execution environment. Any two SIPs cannot share writeable memory and can only transfer exclusive ownership of data (guaranteed by software verifier and verifiable Sing# language). As a result, multiple SIPs can exist in the same address space as the kernel which reduces hardware address translation overhead (TLB flushes and cache misses) as a side effect. And the original hardware memory management unit can be leveraged to offer another level of protection called hardware protection domain by running unsafe programs (written in c++ for example) in a separate address space. Since SIPs are verified to be correct, privileged instructions can be also run directly in SIPs, eliminating the kernel/user mode switching overhead in conventional OS.
The second feature is contract based channels, which SIPs rely on to communicate with other SIPs. A channel is a bi-directional message conduit with two endpoints. Communication over channels follow contracts written Sing# which can also be statically verified with software verifier. Together with exchange heap, channel can achieve zero-copy exchange of large amounts of data between SIPs.
The third feature is the Manifest-Based Programs. In Singularity, each program is supposed to have a manifest which describes the resource requirements, dependencies and expected behaviors. With the manifest, Singularity can check if the program is safe in terms of resource usage permission, interference with previous MBPs and so on.

4. Evaluation
The paper lacks solid evaluation. In the safety perspective, only empirical analysis is given. In terms of the performance, authors did an experiment comparing the overheads of software isolation and hardware memory management. Although experiments show that software isolation is more efficient, the results are not quite convincing since they only use one set of I/O intensive benchmark. Authors also presented some data to prove that Singularity had the least process cost but did not give much insights either.

5. Confusion
1). Authors argue that manifest makes a system safer since system can verify program’s behavior with software. But the safety burden to actually left to application programmers such as driver developers to write correct manifest which precisely describes application’s behavior in every corner cases. Without similar kind of verification tools for this step, the security problem is still open. Manifest is supposed to include resource requirement. Does this requirement only refers to permission or does it include the amount of resource required? If it includes amount of resources required, how does Singularity handle the overprovisioning problem?
2). The paper introduces a lot of kernel data structures such as handle table and linked stacks. How do they function? The paper does not explain them quite clearly.

1. summary
This paper describes a completely new way to design and build operating systems which they call singularity and presents architectural foundations of that design. Most of the design choices it presents are inspired by advances in programming languages and verification and are central to the design philosophy of singularity like systems.

2. Problem

Many of the shortcomings of the modern day operating systems e.g., security vulnerabilities and unexpected process interactions are a consequence of the fact that many of the design choices in these systems are still rooted in the computer architecture and programming languages of 1960's and 1970's despite significant improvement in hardware. To solve most of these problems a clean slate approach to OS design is needed which takes into account the current day computing infrastructure available. Singularity is a step in that direction.

3. Contributions

Singularity is built around three basic abstractions which are as follows:

- Software Isolated Process (SIPs)

A SIP uses the static program analysis instead of using hardware for protection. So the need for hardware level protection mechanisms e.g., TLB flusing, page table management goes away improving the performance as well. The code of the SIP is sealed at the execution time meaning that a SIP cannot dynamically load code or interact with the memory of some other SIP. A SIP can however access the primitive kernel functions through an application binary interface (ABI) to the kernel.

- Contract Based Channels (CBC)
This is the (only) way for SIPs to communicate in the singularity system. Each CBC has an associated contract with it which specifies the messages that flow through the channel and a set of states which dictate what message will flow for a given state of a SIP. The sing# compiler also statically verifies that messages correctly pass through the channel in accordance with a given protocol state.

- Manifest Based Programs (MBPs)
All the programs in singularity are manifest based program. A manifest is a sort of metadata (but not exactly just metadata) that describes the capabilities, system requirements, and dependencies of an MBP. The primary purpose of a manifest is to allow the verification of static and dynamic properties of an MBP. It is however not clear to me whether it would be possible that a programmer is able to lie about the program in the manifest (how would that be proved?) or are the verification techniques are strong enough to not allow that.

4. Evaluation

This paper is a summary of a large project. Therefore, it mostly has system design discussion and has very little evaluation. The two things evaluated are the basic processor costs in terms of cpu cycles for various operations and the webFiles benchmark. The cpu cycle costs for singularity for process creation etc., are much smaller because many of the hardware operations - like flushing TLBs are evaded in singularity. The other thing it evaluates is the execution times for the WebFiles bench mark which the paper calls the unsafe code tax. There again singularity performs better because the cost for hardware isolation is not being paid.

5. Confusion
Is there a way to formally prove that software based protection actually works and there are no security vulnerabilities around?

1. Summary
The paper talks about the Singularity project aimed at building more dependable and verifiable systems. This is reflected in the three architectural features : software-isolated processes, contract-based channels and manifest-based programs. These features, inspired mostly from programming language mechanisms, form the core of building secure and trustworthy systems.

2. Problem
Contemporary OS’s (Windows, Linux, BSD) have decision decisions rooted in the architecture of late 60s and 70s. There are multiple different shortcomings of such systems : security vulnerabilities, buffer overruns, device driver failures. The researchers believe, most of these can be attributed to the fact that systems have not evolved beyond the architectures present back then. Thus, there is a fundamental need to re-examine the design decisions underlying systems (and software stack). Singularity project resulted from this need.

3. Contributions
The novel contribution of the paper is Singularity as a prototype of how OSs can be built (right from scratch) using programming language mechanisms. This essentially allows for greater dependability and trustworthiness.

Software-isolated processes, defined in sealed code spaces, help in ability of program analysis tools to detect static errors. Communication between different SIPs is facilitated through contract-based channels which are implemented as state machines that allows for easy verifiability. The inherent notion of manifest associated with programs helps in checking program properties prior to process creation. Using Sing#, memory-safe programming language, (as opposed to C - not memory safe) helps in avoiding many programmer errors. The fact that the garbage collector is built into the Singularity kernel also indicates the motivation drawn from PLs in designing memory-safe and robust systems.

4. Evaluation
While singularity is designed to be robust, we do not know if it is more performant than other OSs. Improvements in execution time is shown using the WebFiles benchmark. However, no detailed evaluation of other aspects like IPCs, memory access is done.

The researchers say that Singularity is not backward compatible and portable. How would the end-user developing applications try and test out Singularity ?

5. Confusion
1. It seems like contract based channels are very much similar to how TCP works. We can definitely create state machines for TCP too. Is it not the case that current day communication mechanism are also easily verifiable ?
2. Can we clearly and explicitly define manifest for almost every single program ? If not, what do we do ?
3. I do not clearly understand the consequences/limitations of designing OSs using programming languages principles (type safety, garbage collection). How can we understand them ?

Summary:
Singularity was built to address two primary goals of dependability and trustworthiness by using advances in programming languages and verification tools. It incorporates three architectural features: software isolated processes, contract based channels and manifest based programs.

Problem:
Conventional operating systems suffer from perceived lack of robustness due to security vulnerabilities, unexpected interaction among applications, system failures caused by malicious/buggy extensions, plug-ins and driver code. The authors believe that this is because modern operating systems have not evolved beyond computer architecture and programming languages of 1960s and 1970s.

Contributions:
Conventional operating systems provides process isolation via memory management hardware. Singularity relies on type and memory safety of modern programming languages for process isolation. Processes in singularity are sealed code spaces and are called Software-isolated Processes (SIPs).
SIPs communicate with other processes through contract based channels. Channels enable fast and efficient communication between SIPs because the data being communicated resides on exchange heap and only the pointer to this data is passed over the channel. Hence singularity allows zero copy exchange of large amount of data between SIPs. Channel contract enable static verification to avoid run time errors.
Third important architectural feature of singularity is Manifest based Programs (MBPs). A manifest describes MBP’s code resources, its required system resources, its desired capabilities and its dependencies on other programs. Manifest is used to verify whether the MBPs code meets all the safety properties.
Three features discussed in above 3 points enable Singularity to run all processes in single address space which reduces the overhead of context switches. Since SIP is a closed environment, it makes garbage collection much easier.

Evaluation:
Their goal was to provide dependability which is difficult to measure in research prototype. Instead the authors use Singularity to evaluate the cost of hardware and software isolation. Their conclusion is that hardware protection comes at the cost of 37% performance reduction as compared to their baseline which ran with safe code and no hardware protection mechanisms. These results are less reliable since the comparison is between Singularity and Singularity with added hardware protection. Instead it would have been better if they have evaluated Singularity against a conventional operating system.

Confusion:
Who write manifest for MBP? Is it the programmer’s responsibility? If it is programmer’s responsibility, is it justified to expect programmer to know system level details?
What are security principals? How are they useful?

Summary:
Singularity was built to address two primary goals of dependability and trustworthiness by using advances in programming languages and verification tools. It incorporates three architectural features: software isolated processes, contract based channels and manifest based programs.

Problem:
Conventional operating systems suffer from perceived lack of robustness due to security vulnerabilities, unexpected interaction among applications, system failures caused by malicious/buggy extensions, plug-ins and driver code. The authors believe that this is because modern operating systems have not evolved beyond computer architecture and programming languages of 1960s and 1970s.

Contributions:
Conventional operating systems provides process isolation via memory management hardware. Singularity relies on type and memory safety of modern programming languages for process isolation. Processes in singularity are sealed code spaces and are called Software-isolated Processes (SIPs).
SIPs communicate with other processes through contract based channels. Channels enable fast and efficient communication between SIPs because the data being communicated resides on exchange heap and only the pointer to this data is passed over the channel. Hence singularity allows zero copy exchange of large amount of data between SIPs. Channel contract enable static verification to avoid run time errors.
Third important architectural feature of singularity is Manifest based Programs (MBPs). A manifest describes MBP’s code resources, its required system resources, its desired capabilities and its dependencies on other programs. Manifest is used to verify whether the MBPs code meets all the safety properties.
Three features discussed in above 3 points enable Singularity to run all processes in single address space which reduces the overhead of context switches. Since SIP is a closed environment, it makes garbage collection much easier.

Evaluation:
Their goal was to provide dependability which is difficult to measure in research prototype. Instead the authors use Singularity to evaluate the cost of hardware and software isolation. Their conclusion is that hardware protection comes at the cost of 37% performance reduction as compared to their baseline which ran with safe code and no hardware protection mechanisms. These results are less reliable since the comparison is between Singularity and Singularity with added hardware protection. Instead it would have been better if they have evaluated Singularity against a conventional operating system.

Confusion:
Who write manifest for MBP? Is it the programmer’s responsibility? If it is programmer’s responsibility, is it justified to expect programmer to know system level details?
What are security principals? How are they useful?

Summary:
Singularity was built to address two primary goals of dependability and trustworthiness by using advances in programming languages and verification tools. It incorporates three architectural features: software isolated processes, contract based channels and manifest based programs.

Problem:
Conventional operating systems suffer from perceived lack of robustness due to security vulnerabilities, unexpected interaction among applications, system failures caused by malicious/buggy extensions, plug-ins and driver code. The authors believe that this is because modern operating systems have not evolved beyond computer architecture and programming languages of 1960s and 1970s.

Contributions:
Conventional operating systems provides process isolation via memory management hardware. Singularity relies on type and memory safety of modern programming languages for process isolation. Processes in singularity are sealed code spaces and are called Software-isolated Processes (SIPs).
SIPs communicate with other processes through contract based channels. Channels enable fast and efficient communication between SIPs because the data being communicated resides on exchange heap and only the pointer to this data is passed over the channel. Hence singularity allows zero copy exchange of large amount of data between SIPs. Channel contract enable static verification to avoid run time errors.
Third important architectural feature of singularity is Manifest based Programs (MBPs). A manifest describes MBP’s code resources, its required system resources, its desired capabilities and its dependencies on other programs. Manifest is used to verify whether the MBPs code meets all the safety properties.
Three features discussed in above 3 points enable Singularity to run all processes in single address space which reduces the overhead of context switches. Since SIP is a closed environment, it makes garbage collection much easier.

Evaluation:
Their goal was to provide dependability which is difficult to measure in research prototype. Instead the authors use Singularity to evaluate the cost of hardware and software isolation. Their conclusion is that hardware protection comes at the cost of 37% performance reduction as compared to their baseline which ran with safe code and no hardware protection mechanisms. These results are less reliable since the comparison is between Singularity and Singularity with added hardware protection. Instead it would have been better if they have evaluated Singularity against a conventional operating system.

Confusion:
Who write manifest for MBP? Is it the programmer’s responsibility? If it is programmer’s responsibility, is it justified to expect programmer to know system level details?
What are security principals? How are they useful?

1. summary
This paper introduces Singularity, a new operating system design with the goal of dependability and trustworthiness. Key features are software-isolated process, contract-based channel and manifest-based program.

2. Problem
Traditional operating systems are criticized to have problems such as security vulnerabilities, and lack of robustness. The authors claimed such shortcomings primarily attribute to operating systems still stuck with old (1960/70) computer architectures and programming languages. For example, C and early version of C++ have little restrictions on using pointers, and misuse of pointers is a big source for bugs. In addition, extensions to OS such as device driver is running in kernel mode if OS is designed as monolithic kernel. In monolithic kernel, buggy extension can easily lead to kernel failure and system reboot. To tackle the problem, the authors aimed to design a new operating system with new corresponding architecture and programming language.

3. Contributions
The main contribution of this paper is built a microkernel with a new programming language (Sing#). Security is mainly guaranteed by verifying intermediate code (MSIL).

The first feature in their microkernel is software-isolated process (SIP). All processes (including kernel) share a single address space, and all code is running in kernel mode. (As a side effect, this will lead to fast context switch because no need to switch page table and flush TLB.) They abandon hardware protection such as user/kernel mode for privileged instructions and paging mechanism for memory. To ensure protection, they verify intermediate code of program before program running. Process gets service from kernel via application binary interface (ABI), instead of trapping into kernel by special instructions (such as INT). (As a side effect, this will lead to fast system call because no need to switch from process to kernel.) Process can only have pointers to its own data, so there will be no bugs from shared memory. Process's code cannot be changed at execution time, so there will be no bugs from e.g. code insertion attack. Process communicates with each other through contract-based channels.

The second feature in their microkernel is contract-based channels (their way to implement IPC). The IPC is expressed as contract in Sing#. Contract consists of message declarations and protocol states, which again is verified by Sing# compiler to ensure security. There's an area of memory (exchange heap) specified by kernel for IPC. They enforce strong property in exchange heap such as exchange heap only holds pointers into itself, every block in exchange heap is accessible by at most one process, a process has at most one pointer to a block in exchange heap at any point. Shared memory with such strong property can prevent many security bugs such as one process attacking another by exploiting its buffer in shared memory. Additionally, exchange heap can allow zero-copy exchange of data between processes, where multiple layers' data buffer is always a performance problem for traditional OS (such as buffer of NIC, network driver, TCP/IP, and web server for network packets).

The third feature in their microkernel is manifest-based program (MBP). Every program must first have a manifest, which describes code resources, system resources, desired capabilities and dependencies. Again, manifest is used to verify safety properties. This is useful in the sense that before executing a program, first to verify its manifest, they can make sure the new process not interfering with previous ones, and not misusing any hardware resources. In addition, manifest can also guide code placement, and IPC channels assignments.

Other contributions include hardware protection domains and heterogeneous multiprocessing. Hardware protection domains introduce hardware protection back such as paging and user/kernel modes. The system can decide which part of OS should be running in kernel mode during run time, this may lead to better performance. In heterogeneous multiprocessing, system can specialize many-core processors by dedicating them to specific processes, this may lead to better execution performance for processes.

4. Evaluation
This paper didn't give very strong evaluation of their system. The only evaluation is shown in table 1 and figure 5. Table 1 shows CPU cost for several OS-related operations across 4 OSs. Singularity is obviously better than 3 other OSs. But the authors didn't say much about the details of this experiment. We don't know in what settings they got the result. Figure 5 shows the execution overhead among software protection and different levels of hardware protection. The experiment uses an I/O intensive benchmark based on SPECweb99. It would be good to see if authors can provide experiments under other situations such as CPU intensive benchmark, or use their OS to run Microsoft services.

5. Confusion
I am concerned about the difficulty to developing with Sing#. I am curious whether it's difficult to program with Sing#, due to it has high level requirements for security. Is it easy to extend, change and maintain Singularity with Sing#?

1. Summary
This paper introduces the Singularity system, which has three key architectural features: software-isolated processes, contract-based channels and manifest-based programs.

2. Problems
Contemporary operating systems are based on the architecture, programming language and development tools in the 1970s, mainly to solve the management of memory and hardware. Now this ‘outdated’ design faces obvious shortcomes like security vulnerabilities, driver crashes, problems caused by the unsafe C language, etc.

3. Contributions
After 30 years of development in safe language, verification tools and new OS architecture, the author rethinks the OS design and proposes a system called Singularity to solve the problems mentioned before. Software-Isolated-Process (SIP) provides better isolation than traditional process by static verification and runtime checks, isolation is not only provided by hardware, but also by software like programming language types. Contract-Based-Channels provides verifiable safe communication, messages with number and type of arguments, and non-copy transfer of large data between SIPs. Manifest-Based-Programs not only includes the program, but also includes the configuration, version, expected behavior that can be verified dynamically. All of these features utilize the new techniques in PL, architecture in the past 30 years to provide a more secure OS design.

4. Evaluation
In the performance evaluation section, this paper uses a IPC intensive benchmark, changes the level of unsafe code and measures the cost of cache and TLB flush during context switch. This experiment tries to prove the overhead of managing large amount of user processes, but this paper does not make a direct comparison between Singularity and a real OS in the experiment, making this evaluation less convincing. Also, security is an important aspect of this system, but in this paper, there is not an evaluation or comparison with traditional OS. I think maybe it is because evaluating the security of a system is a difficult question not yet solved.

5. Confusion
This system sounds very advanced, but is someone really use this? Is it because we need to rewrite all the driver or applications in the Singularity format and that is too expensive?
In common OS like Windows or Linux, are there some new features that is based on the idea in this paper?

1 Summary
System is not evolving as fast as hardware, so it is still influenced by the arch and language of early time. This paper proposes a new operating system of micro-kernel written in advance programming language to facilitate the verification and dependability. It has three key points, Software-Isolated Processes (SIP), contract-based channels and manifest-based programs.

For SIP, Processes are completely isolated from each other with the help of software. Contract based channel deals with all the communitation between processes. In order to exchange data, processes only pass ownership of pointer in exchanged heap, this no-copy mechanism makes SIP possible. Manifest based programs mechanism requires each program provides a manifest stating its requirements, capabilities, communication channels, and other configurations. And the system will prepare the resources and do verification accordingly.

2 Problem
In order to make operatring system and hardware compatible, we have to resign it instead of keep inheriting the old OS and add new features. By taking advantage of type safe programming language, static analysis and verification, the author try to build a robust operating system.

3 Contribution
a) To make processes isolated with each other, it use static analysis and program verification, while currenct operating system use virtual memory and hardware to implement protection.
b) Since processes only communicate with each other by the contract based channel, it is easy to use a state machine to ensure the in-order delivery. Making the system more dependable. Using the idea of exchange heap is smart, however, if the sending process wants to preserve a copy of the message, it is inevitable to make a copy which will heavily cut down the performance.
c) The singularity system always requires a manifest to run a program which makes management easier, with type check and so on, the system is even more robust.

4 Evaluation
Although this paper emphasize on dependability, they mainly evaluate the performance of this operating system and it turns out to be comparable with other operating system. However, it is hard to measure dependability does not necessarily mean that we do not need it. Given the difficulty of developing on this system is unclear, more evaluation need to be done until people are convinced.

5 Confusion
a) How can it achieve zero-copy when share data with other process? Changing the ownership of pointer result in changing the ownership of data?

b) What makes contract based channel unique?

1 Summary
System is not evolving as fast as hardware, so it is still influenced by the arch and language of early time. This paper proposes a new operating system of micro-kernel written in advance programming language to facilitate the verification and dependability. It has three key points, Software-Isolated Processes (SIP), contract-based channels and manifest-based programs.

For SIP, Processes are completely isolated from each other with the help of software. Contract based channel deals with all the communitation between processes. In order to exchange data, processes only pass ownership of pointer in exchanged heap, this no-copy mechanism makes SIP possible. Manifest based programs mechanism requires each program provides a manifest stating its requirements, capabilities, communication channels, and other configurations. And the system will prepare the resources and do verification accordingly.

2 Problem
In order to make operatring system and hardware compatible, we have to resign it instead of keep inheriting the old OS and add new features. By taking advantage of type safe programming language, static analysis and verification, the author try to build a robust operating system.

3 Contribution
a) To make processes isolated with each other, it use static analysis and program verification, while currenct operating system use virtual memory and hardware to implement protection.
b) Since processes only communicate with each other by the contract based channel, it is easy to use a state machine to ensure the in-order delivery. Making the system more dependable. Using the idea of exchange heap is smart, however, if the sending process wants to preserve a copy of the message, it is inevitable to make a copy which will heavily cut down the performance.
c) The singularity system always requires a manifest to run a program which makes management easier, with type check and so on, the system is even more robust.

4 Evaluation
Although this paper emphasize on dependability, they mainly evaluate the performance of this operating system and it turns out to be comparable with other operating system. However, it is hard to measure dependability does not necessarily mean that we do not need it. Given the difficulty of developing on this system is unclear, more evaluation need to be done until people are convinced.

5 Confusion
a) How can it achieve zero-copy when share data with other process? Changing the ownership of pointer result in changing the ownership of data?

b) What makes contract based channel unique?

1. Summary
Singularity was a research project developed from ground up, to understand & demonstrate the reliability and robustness of OS built using modern programming languages and verification tools. Dependability and trustworthiness as the goals of the project lead the three major architectural design decisions : Software Isolated Processes, Contract-based communication channels and Manifest-based programs. These decisions form cardinal pillars of Singularity to build a safer and trustworthy OS. Authors posit that due to OS design decision, there have been positive side-effects in performance. But authors haven't provided us with the detailed evaluations results to compare Singularity with existing OSes (Though performance was not the center of focus, it would've given an idea of how fast the system is with these security features), which makes it less convincing.

2. Problem
Authors say that contemporary Operating Systems like Windows or Linux have inherited the design decisions from Operating Systems which were built in 1960's and 1970's. They imply that these OSes haven't evolved at the same rate at which the hardware, software tools and Programming Languages have evolved. Due to this fact, these OSes have inherent shortcoming like driver failures, unexpected process interactions, security vulnerabilities and lack of robustness. Singularity project aimed at re-examining the design decisions to overcome these shortcomings; Extensive use of safe languages to eliminate preventable defects, use of verification tools to filter out a class of programming errors and stopping run-time errors propagating through the system.

3. Contributions
The main contribution of the authors is providing Singularity as a proof of concept of new design ideas that OSes can be built using modern safe languages and verification tools to avoid unnecessary OS intervention in providing security to processes and robustness of the system.

As stated by the authors, a new Operating System, new programming languages, new verification tools and new software architecture are the contributions.

Software-Isolated Processes are the one of the fundamental design aspect of Singularity; SIPs are built using safe modern programming languages to provide cheaper process isolation solution. SIPs are similar to processes in other OSes but with some distinctions; Unlike traditional processes, SIPs do not share memory with other SIPs; SIPs are isolated by software verification and not hardware protection (no hardware overhead). They use communication channels for IPC and to get higher-level functionalities of OS (like file access); SIPs use exchange heaps to share data efficiently; SIPs cannot dynamically load or generate code into itself.

Contract-based channels provide the fundamental communication primitives. Manifest-based programs enable the system to verify the behavior of the MBPs before they are loaded into the system through manifests.

Another important aspect of Singularity is its microkernel which provides very basic functionalities like scheduling, memory management and threads. And all other functionalities like file access, device drivers are implemented as SIPs. One big advantages of this approach is that a crash is a device driver will not make the entire OS go down. For example, a faulty printer driver crash will not make the OS result in trap.

4. Evaluation
The authors provide only evaluation results in the paper: CPU cost and Execution time for WebFiles benchmark. The CPU costs for API, threads, messages and processes seem very promising and they are sometimes 10 times faster. But with only this little information, it would be wrong to conclude that Singularity is faster than other OSes (Other aspects like file access, virtual memory, application flexibility etc). With natively compiled binaries, Singularity would have to provide hardware isolation and there are no measure of how well the system performs with mixed user-level applications (SIPs and complied binaries). It would've been good to know the amount of effort required to port existing application to Singularity: Just recompilation of code ? How much of change required ?

5. Confusion
Below are some of the confusions/Questions.

• How does static analysis catch dynamic allocation and dynamically decided indexes ? (let's say a value received from network socket is used for indexing array)
• 1). In section 3.3.2: Scheduler, at the end of first paragraph. "Whenever a scheduling timer interrupt occurs, all threads in the unblocked list are moved to the end of the preempted list, ... Then, the first thread from the unblocked list is scheduled and the scheduling timer is reset". At this point, there are no threads in the unblocked list, then from where is the scheduler picking threads from ? Does it mean first thread moved from unblocked list is scheduled ?
• How do user application (SIPs) find each other for channel communication ?
• Page 2, Column 2, Para 4, Starting at line 2, "Through a combination of static verification and runtime checks". Wasn't the whole idea of Singularity to avoid runtime checks ?
• How memory Virtualization is implmeneted ? How is it efficient if TLB and MMUs are truned off?
• Who generated manifests ? Compilers or application developers ?

Summary

They big claim is that operating systems have not evolved with other
technology. Keeping advances in compilers and compute in mind, the
OS's are not very reliable. With the goal of making Operating Systems
robust/secure/reliable again, Microsoft invested in a research
prototype called Singularity, which leverage memory safe languages,
advanced verification techniques and the concept of isolated processes
to make OS's reliable again.

Problem

Simply put, current operating systems are not secure. They're riddled
with buffer overflows, errant device drivers, malfunctioning plug-ins
etc. With Singularity Microsoft wishes to design an OS with a clean
interface for isolating it's moving parts.

Contribution

To accomplish their goals, they introduce 3 major abstractions:

- SIP's or isolated processes
- Contract based communication
- Manifest based programs

All processes and even the kernel share an address space, but
they're isolated from each other by the programming language and
software. There is no hardware based virtualization of
memory. Apparently Sing# doesn't need contiguous logical address
space.

So each process has it's own heap, it's own garbage collector and it
can fail independently of everyone.

The issue, is processes need to talk to other processes. To
accomplish that, Singularity has a specific place in memory known as
the exchange heap. Using two way channels, processes can pass data
between each other through this heap.

Only one process can have acccess to any data in this heap.

Example: Process A has a pointer to some data in the exchange
heap. Process B cannot just access this data. Process A relinquishes
it's ownership of the data, only for Process B to pick it up. Thus
it doesn't have to copy over all the data it can just relinquish
access to the pointer that pointed to the data

The exact workings of this contract based channel communication is
ambigous. I would love for someone to draw it out.

Singularity’s manifest-based programs bring
type checking to process creation. A manifest defines a bunch of
checkable program properties that the Singularity kernel can verify
before starting a process. For example, the manifest says what ABI
versions a program needs, what IPC interfaces are required, what
other processes must be started, and so forth. The kernel can check
“type and memory safety, absence of privileged-mode instructions,
conformance to channel contracts,” and other, more specific
properties, such as “that [a device] driver will not access hardware
used by a previously installed device driver.”

Evaluation

It's hard for me to evaluate Singularity. The goal of the project was
to imorove system reliability. A reliable system has minimum human
programming errors (buffer-overflow, privileged access etc)

However singularity uses it's own advanced compiler and Sing# which
would imply it doesn't receive a tonne of interested
developers. Without anyone programming on singularity I'll just have
take Microsofts word for it that the system is reliable. How do we
know about it's vulnerabilities without anyone trying to break it?

They mention that they don't really care about performance, though in
some cases they outperform current OS's, but that was more of side
effect and not intentional.

I don't know if it is reliastic to ignore performance of an OS.

Confusion

I read their other papers on just SIPS
SIP's message passing
and it's still not exactly clear how this message passing is accomplished.

A drawing and would be nice.

How is different from something like TCP? One difference might be that
TCP involves data copying, whereas Singularity has permission
relinquishing.

It also seems like they're putting all their eggs in one basket. i.e
TRUST this programming language and the verifier.
Someone finds a vulnerability in that, then what?

Summary
In this paper, the authors present the Singularity systems with three features, software-isolated processes, contract-based channels, and manifest programs.

Problem
Contemporary operating systems were mostly developed in 1960s-1970s, when the focuses are speed and limited memory and there are limited number of computers used by experts with rare network communication. However, the shortcoming of these systems have been increasingly obvious, including wide-spread security vulnerabilities, unexpected interactions among applications, failures caused by errant extensions, plug-ins and drivers, and a perceived lack of robustness.

Contribution
First, this paper proposes three new features for the system design. The software-isolated processes avoids the unnecessary interaction bewteen processes much, thus increasing the security of local data. It also reduce the overhead of process switch context by removing the page tables and TLBs. The contract-based channels enables fast, verifiable message-based communications. The manifest-based programs allows static and dynamic verification of properties of programs.
Second, this paper builds a new systems based on a new programming language (Sing#). Compared with old operating systems, such a system have better verfiability and dependability.

Evaluation
This paper evaluate two metrics, performance and compatibility with previous systems. It claims Singularity out-performs existing systems. However, it does not provides experimental results to show the superority of its design features to the old operating systems. The breadth of applicability of Singularity is also unclear. Without good metrics and fair comparisons between Singularity and other OSs and exhausting tests under different hardware settings, it is hard to evaluate the system.

Confusion
1. What is the verification process the paper mentions in detail? How does the paper claim its system has better verifiability and dependability?
2. What is the difference between the contract-based channels and the traditional message communication in processes?
3. What is the exact memory usage in SIPs? How does the system do context switch? Any concrete example?

1) Summary

Current OSes have looked almost the same for decades, taking after the inspiration of Unix. The authors consider how an OS could be built from scratch using modern advances in computer science. They propose a design very different from traditional Unix-like OSs, and posit that OSs can be made safer and faster due to advances in programming languages and verification.

2) Problem

Despite advances in programming languages and static analysis, OSes and systems code suffer from numerous bugs and security vulnerabilities. Currently, systems code tends to be complicated and subtle. The authors observe that modern programming languages and static analysis can offer significant help in managing this complexity and removing memory unsafety from the system without significant performance cost.

3) Contributions

The authors propose a simple and elegant OS design with a few straightforward simplifying invariants that are easy to understand. They also propose a programming language, programming constructs, and static analysis tools to enforce these invariants in the system. The aim of these contributions is to use formal methods to curb the complexity and unsafety of low-level or systems code.

The authors' system is designed around software-isolated processes, contractual channels, and manifest-based programs. Software-based isolation can remove the overhead and complexity of using hardware isolation without decreasing safety. The authors further suggest that software isolation may pave the way to simpler and faster hardware. Contractual channels provide a way to pass information in a way that is easy to reason about formally, resulting in the ability to pass large amounts of data without copies. Finally, manifest-based processes provide a way to declare and varify the formal guarantees and assumptions of a program, allowing applications to be proven safe or rejected by the system.

4) Evaluation

The authors (along with common experience) present compelling reasons why OSes should not be implemented in C. The authors demonstrate that a moderately complex system can be implemented using a safe language. However, they do not extensively demonstrate that their system has the benefits they seem to imply.

For example, the authors' primary claim is that Singularity prevents errors and bugs from existing or propagating in the system and user applications. However, they never give evidence to support this claim. For example, they only give one example of a bug that was caught verification. Personally, I would need more than one bug to convince me that the system is worth it. Ideally, I would like to see a count of bugs discovered and prevented by the verifier or the system design, though this may be difficult to measure.

The authors do quantitatively show that hardward isolation has high overhead. They demonstrate that a side-effect of safer and simpler systems is that hardware may also be simpler and faster. While this a useful observation, it does seem a bit out of place in the paper, since the authors explicitly state that performance is not their first goal.

Finally, the authors also state that they have not made their system portable or backward-compatible for the sake of making it easier to use in research. However, they never describe what it would take for such a system to become portable.

5) Confusion

The authors never state any caveats of their system. Apart from the obvious problem of legacy code, why don't we all use systems like this? The paper was written almost 15 years ago!

1. summary
This paper describes the design of Singularity, a microkernel system whose goal is dependability and trustworthiness, mainly there key architecture features: software-isolated procsses, contract-based channels and manifest-based programs.

The process in Sinularity is Software-Isolated Processes (SIP), which accomplishes isolation through static verification and run-time checking instead of traditional hardware isolation. They proposed Hardware Protection Domain, a software-enforced protection boundaries which can host SIPs. This provides a level of indirection and flexible mapping between domains and SIPs, which can be good for legacy code and combination of software and hardware isolation.

The SIPs can only communicate through channels based on contract. This single strategy makes communication of the system more unified and less dependent. And with SIPs, channels and linear types, Singularity allows zero-copy of data transfer between SIPs through pointer passing.

All the code allowed to run in Singularity need to have a manifest, that is manifest-based programs. The manifest contains the code's dependencies, configuration parameters, version information, and will describe the expected behavior of the program which can be checked and verified both statically and dynamically.
2. Problem
They build a completely new operating system and try not to be constrained by the legacy of traditional system to explore the combination of progress in type safe programming language, program verification tools and os architecture to make the system more robust, safe and easy for engineering.

3. Contributions
The contribution of this project is that it can show the possibilities of totally different kind of os design, and the utilization of PL and verification to make the system robustness. This system may not be used by a large number of people, but it can give insight and lessons to related research and system design. Like the verification can be considered for complex system design and programming language can also have language-level properties like SIP and channel in language VM to provide isolation. This may like the open question in math, which can spark a lot work.
4. Evaluation
The dependence can not be easily evaluated, so they do some evaluation of performance. Although it is not the system's primary goal, they do have some performance gain mainly through static verification and reduced complicated context switch. The evaluation of performance is used to demonstrate that their design will not hurt performance.

The evaluation of costs of hardware and software isolation is interesting. But I do wondering will the safe code of SIPs hurt the deployment and initialization of the system.
5. Confusion
1). In the scheduler section (3.3.2), is there any thread left in the unblocked list to be scheduled when a scheduling timer interrupt?
2). The two ends of communication in Listing1

1. Summary
Singularity is a research OS built by the authors from the ground up to focus on isolation of processes and verifiability of code.

2. Problem
Conventional OSs (Linux, Windows, OS X) do an inadequate job of isolating processes from each other and from the kernel. Because kernel code and driver code operate in the same memory space and execution context, a buggy driver can bring down the whole system; similarly, a buggy extension to a user program can easily cause the whole program to crash. The authors also highlight the inherent lack of safety checks in the C language, used to write most of the code in the mainstream OSs. C is also not designed to be easily verifiable, making it hard to check code for trustworthiness.

3. Contributions
The authors' main contribution is a demonstration of a working OS written (mostly) in a modern, verifiable, type-safe language. This allows them to package software into verifiably isolated “Manifest-Based Programs” that are guaranteed to behave well during execution. These verifiable guarantees mean that the OS does not have to go to great lengths to protect processes from one another, and hence may run them all in the same address space. This makes context switches a much more lightweight operation.
This choice of language leads to the authors' second contribution, the design of the OS as a series of isolated modules that communicate through well-defined “channels” and cannot interact through shared memory. This strict isolation presents some challenges to overcome over a more traditional OS, which often leans heavily on shared memory and closely interacting processes to function. By building Singularity, the authors show that such a design is possible.
It is important to note that the authors' contributions are the proof that these things are possible, not Singularity itself. As the authors put it “Singularity is a laboratory for experimentation in new design ideas, not a design solution.”

4. Evaluation
This paper does not feature a large amount of empirical observation. As mentioned above, the main evaluation is building a working prototype that proves these design goals are feasible to meet. Figure 5 shows some shady numbers about the “unsafe code tax”: running Singularity, they raise the level of untrustworthiness of the code, forcing the system to do things like flushing caches on context switches. They then conclude conventional OSs perform like Singularity with all code marked as “untrustworthy.” This evaluation would have been better if they compared their runtime to a commercial OS. As these commercial systems are orders of magnitude more optimized, I suspect Singularity would lose in a more direct comparison. As such, it is unknown how well this system actually performs.
In the conclusion, they admit to dispensing with any worry about performance or compatibility. While Singularity is not designed to be a commercial OS, being completely incompatible with old driver code would be a huge barrier to adoption. Also, as the authors point out, it is unclear if it is possible to build a fully verifiable or trustworthy OS. There are many steps involved to building such a system, and the Garbage Collector and compiler still stand out as weak points in Singularity.

5. Confusion
I was confused by the discussion on Contract-Based Channels. Must all contracts be implemented as strict state machines? That seems like a very big and inflexible design constraint.