A Comparison of Two Network-Based File Servers

James G. Mitchell @ Xerox & Jeremy Dion @ Cambridge Univ

Communications of the ACM, April 1981, pages 233-245

Introduction

This paper compares two file servers: CFS (Cambridge File Server) & XDFS (Xerox Distributed File System)

General Design Choices

Background and Goals

CFS Background and Goals

Intended as a backing store of virtual memory system with small segments

Required characteristics

rapid access to files

high transfer rate

For efficiency, simplicity is favored

File level locking

XDFS Background and Goals

Intended as a file server for database

Atomicity of transaction, including multiple files, is important

More parallelism - byte level locking

Flexible configuration

Access Control

CFS Access Control: Capability-based

Any client having the capability of a file can access the file. Server does not care who the client is

Capability: 32-bit disk address of the file + 32-bit random number

Server generates the capability when a file is created

Capability can be passed to other clients

Clients participating a transaction just need to share the same capability

XDFS Access Control: Identity-based

Access control is based on the identity of the client

Authentication is done at the connection establishment

When multiple clients participate in a single transaction, server need to keep track of whom she is talking to and enforce appropriate policy based on who the client is

Storage Reclamation

Issues about

how to organize files in the file server

garbage collection

CFS

CFS server organizes files in hierarchy with files and indices (similar to directory but no pathname-to-id mapping)

Each client is given a slot within root index

Client can organize its files and index (= subdirectory) arbitrarily

Objects (file or index) are shared by multiple client by each client having its entry for the shared object in its own file-tree

Server maintains reference count

When the reference count becomes zero, periodic garbage collector deletes the object

Garbage collector also deletes files or indices which are not reachable from the root index

XDFS

Essentially no organization

Files are automatically deleted when the transaction is aborted

If client loses the file reference which it created, the file never gets deleted

Transactions

Issues about how to maintain consistency of files -> transaction

CFS Transactions

A series of update operations of a single file can be atomic

Session semantics of sharing

Transaction: open -> read/write -> close

open: start transaction & get a lock for the file or index

read/write: multiple reader and single writer

close: commit or abort & release the lock

Clients can creates either:

special files: transactional file

normal files: ordinary files for which transactional requirements need not be met

Timeout mechanism is used to detect client crash: client infinitely looping cannot be detected

XDFS Transactions

Update operations over multiple files on multiple servers by multiple clients can be atomic

Transaction

start transaction: transaction UID returned

add-host to make a server get involved in the transaction

read/write

end transaction

Locking

Implicit locking as a result of read/write in byte granularity

Kind of complex mechanism to guarantee serializability

Network Protocols

CFS Protocol Issues

Error control

Due to the inherent reliability of underlying ring network, CFS uses big (semantic) packet: error checking per a complete write operation

Flow control

Explicit (hardware level) notification of packet lose from receiver -> Sender backs off and retransmits the packet

Stateful server: server maintains file offset

Partly idempotent client request: sends absolute file offset with read/write to cope with possible retransmission of packets

Packet duplication and out of order delivery are not addressed

XDFS Protocol Issues

Error control

send -> response -> ack for the response

Timeout for response and ack is used: rather than explicit notification

End-to-end error detection is used and error packets get dropped on the floor silently

Basically no flow-control. Pup handles it: who knows?

Comparison of Implementations

File Representation

CFS File Representation

File is represented as a tree

Data pages are leaf nodes

Starting from a single data page, the tree is built up as the file expands

XDFS File Representation

Single B-tree per partition

Pages from different files are put together in a B-tree

4.2 Disk Redundancy

4.2.1 CFS Disk Redundancy

4.2.2 XDFS Disk Redundancy

4.3 Dynamic Storage of a Transaction

4.3.1 CFS Dynamic Storage of a Transaction

4.3.2 XDFS Dynamic Storage of a Transaction

5. Evaluation

5.1 XDFS Successes

5.2 XDFS Shortfalls

5.3 XDFS B-tree

5.4 CFS Successes

5.5 CFS Shortcomings

5.6 Performance Analysis

5.6.1 Basic Communication Overhead

5.6.2 Open Transaction/Close Transaction

5.6.3 Reading 256 Bytes

5.6.4 Writing 256 Bytes

5.6.5 Writing 262,144 Bytes in an Existing File

6. Suggestions for Future File Servers