i. What is IPC?
ii. Does it move the same a complexity to an App? NO: app doesnŐt need all of the kernel
i. Predated modern virtual machines – they are like an exokernel
ii. Paper presented at HotOS workshop – for new ideas, not fully-baked systems. Later published 2 papers demonstrating performance, flexibility, sharing
iii. Trying to be inflammatory – to get attention.
- operating systems getting big and bloated. QUESTION: HOW BIG IS A KERNEL? Linux?
- abstractions offered useful for low-perf apps (sometimes), but
mismatch between offering and what apps need makes programming
difficult, hurts performance.
- Example: automatic virtual memory. App may have better sense of its
needs, e.g. what pages to replace and when
CLAIM: impossible to provide abstractions good for all applications
Lets examine claim:
1. What is an OS? code that securely multiplexes + abstracts hardware.
- Securely multiplex: why? so can run multiple programes
- Abstract: why?
- Makes multiplexing easier (at higher level, conceals some
e - details. E.g. file systems)
- Easier to write program - don't need to deal with low-level
2. Lots of good ideas proposed but not adopted by OS - demonstrates
limits of current OS design
Lets examine this claim:
- how do we know all the ideas are good?
- lots of things are adopted into operating systems
3. Current design leads to:
- poor reliability, because complex systems needed (e.g. vm, COW,
- Poor adaptability: hard to add new policy, mechanism. Change not
localized because OS applies to all apps
- Poor performance: abstractions take time to execute, applications
that don't need them still pay. Example: GC or DB that interacts
poorly with VM and would do better managing memory itself
- Poor flexibility: apps can't implement their own abstractions, just
emulate on top of existing OS at high cost
1. QUESTION: What are benefits
2. Higher level of programming
3. Code reuse – donŐt have to rewrite low-level code
4. Higher-level policy; can enforce policies with more knowledge
a. E.g. more information about sharing & cooperation
5. More security: can group information for finer-level access control
6. Can share at a higher level – e.g. files instead of blocks, to get better consistency semantics
7. Easier to write programs – can deal with things you think about (contiguous memory, files), not hardware
1. QUESTION: What are drawbacks
2. Performance: wrong abstraction means what you want is very expensive
a. E.g. file systems when you want block storage, or have large files, or billions of small files
b. E.g. networking – canŐt get to inner access of how protocols work, hard to avoid copying
3. Parameterized policies only work if you fit in the parameters
a. E,g, standard schedulers hard to use for real-time, have to muck with ugly kernel code
4. Hard to change monolithic kernel – difficult, complex code
a. E.g. optimize to deliver network packets right off disk to web server
b. Issue isnŐt as much one of booting into another OS, as to modifying to do what you want
iii. End-to-end argument
1. Application knows best
2. application has to handle things anyway
3. might as well give power to the application to do things their way
i. Big idea:
1. Kernel provides minimum necessary to securely multiplex hardware, but no abstractions (if possible)
2. OS runs as a library attached to application, provides abstractions
a. should be easier to extend, better tuned to applications
b. provides same high level abstractions to make it easy to program
3. Idea is NOT:
a. have a trusted user-mode implementation of a service
4. Idea IS:
a. Run the OS code yourself in your process
b. NO trusted third party
ii. Minimal kernel, exokernel, that provides access to hardware if at all possible
1. QUESTION: How?
2. Expose hardware names – e.g. physical address
3. Expose hardware events – e.g. swapping memory out
a. libOS needs to change of HW changes
b. QUESTION: when do apps need to change?
i. A: only if depend on HW – should hide in LibOS
c. ISSUE: may have more libOS to port
5. Check capabilities on use
a. Use HW support – e.g. TLB, or tables for disk blocks. Very lightweight
6. Provide SW TLB – apps can fill in own mappings safely
7. Notify app that resource is being removed – e.g. page, cpu being removed
8. QUESTION: how do you handle resource sharing?
a. DonŐt have enough policy in the kernel to make accurate decisions
iii. Provide abstractions in libraries – programs can choose what libraries they want
1. FROM REVIEWS: Confusion over trust – kernel doesnŐt trust application, but libos trusts application
2. QUESTION: Who writes these?
a. Could be OS developer, but leaves open modules for extension.
b. Could be a port of an existing OS for compatibility
c. Can run multiple simultaneously
3. QUESTION: What are implications? Do app developers have to write their own OS? Do app developers have to write own window routines in X because in user space, compared to windows? But can X be changed more easily?
a. Can think of OS code being more modular – can replace parts of it, just for your app. Can leave interface alone, or subclass & add new features.
4. QUESTION: portability- how handled?
a. A: library has hardware abstraction layer. But need not, if want extra performance
b. E.g. routines to manipulate page tables, handle specific exception formats
5. QUESTION: What are assumptions
a. Not for timesharing; relatively trusted libOS because computer user chooses it,
b. Probably not for malicious applications
6. Each application links to a libOS, which provides OS-level APIs
7. Applications trusting each other can have a libOS that shares state
8. LibOS chooses what state to save/restore on context switch
9. LibOS can be customized to the application – different mechanisms and policices
10. QUESTION: Impact on complexity?
a. Previously OS / device drivers hide complexity – only do once
b. Now: all libOS have to do it, or share code
c. Now: can have libOS that only has the features you need – leave out everything else
i. makes libOS simpler
d. When you need new functionality:
i. need to change libOS
ii. maybe copy code, maybe reimplement
iii. e.g. static web server -> dynamic web server
iv. Kernel handles dependencies between services, but in libOS, when add services, need to manage this separately
iv. Provide some functionality via safe downloaded code into kernel
1. Code has guaranteed completion
2. Limited access to memory
3. Used for packet filters, event notification (wake up), file system block translation
v. Big issues are sharing
1. If kernel doesnŐt handle it, how does it happen?
a. Between libOS – but can optimize for trust relationship
i. Mutual trust (e.g. unix processes of same user)
1. Allow sharing w/o OS intervention
ii. Unidirectional trust (e.g. process/kernel, microkernel server)
1. Trusted sides retains ownership of shared resources, e.g. page tables
iii. Mutual suspicion
1. Provide kernel support via. Downloaded code
2. LibOS must treat all data suspiciously
iv. QUESTION: should you trust? How does that impact reliability / security?
2. Handling stable storage
a. Goal : determine access rights to a block
b. Export free block list, disk map from kernel
c. use untrusted deterministic functions to compute on metadata
i. translates fs-block type to a native kernel format
ii. returns a list of blocks owned by this block (e.g. inode references other blocks)
3. NOTE: lose some flexibility of implementation if you want to share, as you must use same abstractions/mechanisms as other sharers (but may have some flexibility of policy)
i. APPLICATIONS: really fast web server, XCP zero-touch copy
ii. Benchmarks – standard Linux cp, gunzip, diff, gcc, rm, tar
iii. Sharing: how handle conflicting global policies (e.g. scheduling)
1. E.g. malicious libOS wants more resources?
iv. Applicability: best used for appliances, e.g. search appliance, tivo, web server, etc.
v. How handle different apps with different needs?
vi. LibOS may end up suffering same problem as kernel – still hard to modify, hard to test
vii. Relevance: Ideas can be abstracted into other OS, e.g. downloading code into kernel for event dispatch, exposing soft tlb for sharing.