Past approaches log all
state-changing operations on the driver/kernel interface and restart
the driver and replay the entire log following a failure. This
approach poses two problems: first, restarting a driver can be slow
(multiple seconds) due to complex initialize and device probing code;
and second, drivers with proprietary commands cannot rely on generic
logging/replay, as the semantics of the commands are unknown outside
the driver. \fixme{What about internal device state not exposed
through the driver? We don't test this case so should probably leave
it out}.

The system
relies on checkpoint to snapshot driver state and executes driver code
in a software-isolated sandbox. Unlike past SFI-based driver isolation
mechanisms, ODFT executes driver code on a copy of data, allowing
recovery to discard changed state following a faiulre and restore
device state from a checkpoint.


-----
 As described next, Isolator also generates
code to invoke the SFI module with a copy of the driver and kernel
state Hence, the driver entry points in SFI module run with a {\it
  copy} of device and driver state and execute with {\it physical type
  safety} (provided by SFI), with full support for device and driver
rollback. Meanwhile, the entry points in the original module, run
natively without any isolation or recovery support and associated
overheads. In the following sub-sections we describe the various
components that provide isolation and recovery support and the
resource access management during


\ignore {
  File system library influence the file system semantics provided 
  to applications, whereas the server is concerned with maintaining 
  consistency of the file system internal data structures. 
  
  The file system semantics provided to applications are implementation 
  specific. They depend on the concurrency mechanism and how the 
  mechanism is used to isolate accesses. 
}



------
The ACPI specification, specifies device power management states
D0 through D3, where D3\subscript{\it \footnotesize off}, represents a device that's 
powered off, D3\subscript{\it \footnotesize hot} and  D3\subscript{\it \footnotesize cold} represent standby states
and D0\subscript{\it \footnotesize initialized}  and  
D0\subscript{\it \footnotesize uninitialized} represent fully powered states.

For example, in modern PCI drivers when a device enters standby state, it stores the 
device state to RAM and triggers a transition from D0\subscript{\it \footnotesize initialized} 
to D3\subscript{\it \footnotesize hot}. Since, this state is {\it always}, software 
triggered~\cite{pcipm:spec}, it is the responsibility of the OS(driver and the bus) to
save the device state correctly~\cite{pcipm:spec} and use it to restore the device after resume. 
When returning from standby, the device enters D0\subscript{\it \footnotesize initialized} 
state from D3\subscript{\it \footnotesize hot}, following which the bus and the driver 
restore the device state correctly in a manner that is transparent to the host application (
as much possible). Furthermore, one needs to 
save and restore the device state only during D0\subscript{\it \footnotesize initialized} 
state, which is an running, quiesced driver. Hence, if we can quiesce the driver,
for example by using device locks, and follow the driver protocol for obtaining
and resetting the state, we should be able to save/restore device state without
transitioning to the D3\subscript{\it \footnotesize hot state}. The specifications 
also enforce device reinitialization which is performed after return from standby in 
the driver callback. This re-initialization is lightweight compared to a full device
initialization. Few  drivers may return to the D0\subscript{\it \footnotesize uninitialized}
from D3\subscript{\it \footnotesize hot state} as  indicated by special bits,  
which require additional restoration  of the configuration space mappings by the bus, 
as performed during driver initialization. However, since we actually never transition 
to D3\subscript{\it \footnotesize hot state}, these mappings never get unmapped.

--------



Furthermore, the state of the device checkpoint must be
consistent with the state of the driver.

Thus, device state checkpointing ensures consistency by holding locks
while capturing device state. Rather than introducing new locks, we
observe that drivers must already protect against concurrnet
invocations on multiple CPUs either through internal locks or through
locks acquired by the kernel. Thus, device state checkpointing ensures
that one of these existing locks is held when capturing device
state. For example, in the code in Figure~\ref{fig:checkpoint-code}, the
checkpoint code holds \codesm{tp->lock} spinlock.

-------
In addition to restoring device function, these experiments also
demonstrate that ODFT correctly cleans up thread state and returns an
error to the claling thread. For example, when we call
into the sound card using {\tt proc} interface, the calling thread acquires mutexes in
user space before calling into the kernel. We find that these the thread is correctly 
cleaned up during failure and since an error is correctly returned, the {\tt proc} interface
for the sound card remains available. 

-------

\paragraph{Hardware Interface specification}
\noindent Several projects, such as Devil~\cite{merillon00},
Dingo~\cite{ryzhyk:dingo:eurosys:2009},
HAIL~\cite{sun:hail:emsoft:2005}, Nexus~\cite{williams08},
Laddie~\cite{wittie:check-driver-io:apges:2007} and others, 
improve reliability by ensuring driver follows correct protocol
for the device. These projects can be extended to track device 
states but will require specifications on a per device basis.
ODFT uses the generic driver-bus interface to track device 
configuration state and work for devices where full hardware
specification is not available.

