Transactions, Locking and Logging
[SHORE Storage Manager Application Programming Interface (SSM API)]

Collaboration diagram for Transactions, Locking and Logging:

---

Detailed Description

All storage manager operations on data must be done within the scope of a transaction (ss_m::begin_xct, ss_m::commit_xct, ss_m::abort_xct, ss_m::chain_xct).

A very few storage manager operations, such as formatting a volume, are called outside the scope of a transaction and the storage manager begins its own transaction to do the work.

Operations that fail return an error indication and the storage manager assumes that the server will thereafter abort the transaction in which the error occurred, when abort is indicated. Abort is indicated when eUSERABORT or eDEADLOCK is returned and when the erver chooses to abort rather than to work around the problem (whatever it might be, such as eRETRY).

The storage manager does not enforce the aborting of any erroneous transactions except, possibly, those that are in danger of running out of log space. (This is done with the destructor of the prologue used on each call to the storage manager, see next paragraph).

It is always the server's responsibility to abort. When the storage manager encounters a eLOGSPACEWARN condition (the log hasn't enough space at this moment to abort the running transaction, assuming a 1:1 ration of rollback-logging overhead to forward-processing logging overhead), it does one of two things:

• passes the error code eLOGSPACEWARN up the call stack back to the server if the storage manager was constructed with no log-space-warning callback argument (see LOG_WARN_CALLBACK_FUNC, ss_m::ss_m).
• tries to abort a transaction before passing an error code back up the call stack to the server. Choosing a victim transaction to abort is done by the server in its log-space-warning callback function (passed in on ss_m::ss_m, q.v. Only if that callback function returns a non-null victim transaction and returns eUSERABORT does the storage manager abort that victim before returning eUSERABORT up the call stack. Any other error code returned by the callback function is just returned up the call stack.

Locks

The storage manager automatically acquires the necessary locks when the data are read or written. The locks thus acquired are normally released at the end of a transaction, thus, by default, transactions are two-phase and well-formed (degree 3).

Lock Granularity

The fine-grained locks are normally used for records in files, but provision is made for using coarser-grained locks. The transaction has a default lock level associated with it, which governs the granularity of locks acquired by the storage manager on behalf of the transaction. The lock manager provides for lock escalation to coarser locks to reduce the locking costs. See Locking and smlevel_0::concurrency_t.

Key-value locking is normally used for B+-Trees. (See [MOH1].) R*-Trees normally use coarse-granularity locking. The locking protocol used with an index is determined when the index is created. A transaction may acquire coarse (index-level) locks with explicit calls to the lock manager, but by default, the granularity/level/protocol associated with the index is used. See smlevel_0::concurrency_t.

Distributed Transactions

Storage manager transactions may be used as "threads" (to overload this term) of distributed transactions. Coordination of 2-phase commit must be done externally, but the storage manager supports preparing the (local) transaction "thread" for two-phase commit, and it will log the necessary data for recovering in-doubt transactions.

Threads and Transactions

Transactions are not tied to storage manager threads (smthread_t, not to be confused with a local "thread" of a distributed transaction) in any way other than that a transaction must be attached to a thread while any storage manager work is being done on behalf of that transaction. This is how the storage manager knows which transaction is to acquire the locks and latches, etc. But a thread can attach and detach from transactions at will, so work may be performed by different threads each time the storage manager is called on behalf of a given transaction; this allows the server to keep a pool of threads to perform work and allows them to perform work on behalf of any active transaction.

Warning:

While there are limited circumstances in which multiple threads can be attached to the same transaction concurrently and perform storage manager operations on behalf of that transaction concurrently, which is a hold-over from the original storage manager, this functionality will be deprecated soon. The reason for this being removed is that it is extremely difficult to handle errors internally when multiple threads are attached to a transaction because partial rollback is impossible in the absence of multiple log streams for a transaction.

Under no circumstances may a thread attach to more than one transaction at a time.

Exotica

The storage manager also provides

• partial rollback (ss_m::save_work and ss_m::rollback_work), which undoes actions but does not release locks,
• transaction chaining (ss_m::chain_xct), which commits, but retains locks and gives them to a new transaction,
• lock release (sm_quark_t, ss_m::unlock), allowing less-than-3-degree transactions.

To reduce the cost (particularly in logging) of loading databases, the storage manager provides for unlogged loading of stores. See Stores.

All work performed on behalf of a transaction must occur while that transaction is "attached" to the thread that performs the work. Creating a transaction attaches it to the thread that creates the transaction. The thread may detach from the transaction and attach to another. Multiple threads may attach to a single transaction and do work in certain circumstances. See Multi-threaded Transactions

Modules
	Locking
	Partial Rollback: Savepoints
	Early Lock Release: Quarks
	Distributed Transactions: Two-Phase Commit
	Multi-threaded Transactions
	Running Out of Log Space
	How Log Sequence Numbers are Used
Functions
static rc_t	ss_m::begin_xct (timeout_in_ms timeout=WAIT_SPECIFIED_BY_THREAD)
	Begin a transaction.
static rc_t	ss_m::begin_xct (sm_stats_info_t *stats, timeout_in_ms timeout=WAIT_SPECIFIED_BY_THREAD)
	Begin an instrumented transaction.
static rc_t	ss_m::begin_xct (tid_t &tid, timeout_in_ms timeout=WAIT_SPECIFIED_BY_THREAD)
	Begin a transaction and return the transaction id.
static rc_t	ss_m::commit_xct (bool lazy=false, lsn_t *plastlsn=NULL)
	Commit a transaction.
static rc_t	ss_m::commit_xct (sm_stats_info_t *&stats, bool lazy=false, lsn_t *plastlsn=NULL)
	Commit an instrumented transaction and get its statistics.
static rc_t	ss_m::chain_xct (sm_stats_info_t *&stats, bool lazy=false)
	Commit an instrumented transaction and start a new one.
static rc_t	ss_m::chain_xct (bool lazy=false)
	Commit a transaction and start a new one, inheriting locks.
static rc_t	ss_m::abort_xct (sm_stats_info_t *&stats)
	Abort an instrumented transaction and get its statistics.
static rc_t	ss_m::abort_xct ()
	Abort a transaction.
static w_base_t::uint4_t	ss_m::num_active_xcts ()
	Return the number of transactions in active state.
static void	ss_m::attach_xct (xct_t *x)
	Attach the given transaction to the currently-running smthread_t.
static void	ss_m::detach_xct ()
	Detach any attached from the currently-running smthread_t.
static xct_t *	ss_m::tid_to_xct (const tid_t &tid)
	Get the transaction structure for a given a transaction id.
static tid_t	ss_m::xct_to_tid (const xct_t *x)
	Get the transaction ID for a given a transaction structure.
static xct_state_t	ss_m::state_xct (const xct_t *x)
	Get the transaction state for a given transaction (structure).
static smlevel_0::fileoff_t	ss_m::xct_log_space_needed ()
	Return the amount of log this transaction would consume if it rolled back. If a transaction aborts with eOUTOFLOGSPACE this function can be used in conjunction with xct_reserve_log_space to pre-allocate the needed amount of log space before retrying.
static rc_t	ss_m::xct_reserve_log_space (fileoff_t amt)
	Require the specified amount of log space to be available for this transaction before continuing. If a transaction risks running out of log space it can pre-request some or all of the needed amount before starting in order to improve its chances of success. Other new transactions will be unable to acquire log space before this request is granted (existing ones will be able to commit, unless they also run out of space, because that tends to free up log space and avoids wasting work).
static rc_t	ss_m::gather_xct_stats (sm_stats_info_t &stats, bool reset=false)
	Get a copy of the statistics from an attached instrumented transaction.

---

Function Documentation

static rc_t ss_m::begin_xct

(

timeout_in_ms

timeout = WAIT_SPECIFIED_BY_THREAD

)

[static, inherited]

Begin a transaction.

Parameters:

[in]

timeout

Optional, controls blocking behavior.

Start a new transaction and "attach" it to this thread. No running transaction may be attached to this thread.

Storage manager methods that must block (e.g., to acquire a lock) will use the timeout given. The default timeout is the one associated with this thread.

See also:

timeout_in_ms

Examples:

create_rec.cpp, log_exceed.cpp, sort_stream.cpp, and vtable_example.cpp.

static rc_t ss_m::begin_xct	(	sm_stats_info_t *	stats,
		timeout_in_ms	timeout = WAIT_SPECIFIED_BY_THREAD
	)			[static, inherited]

Begin an instrumented transaction.

Parameters:

[in]	stats	Pointer to an allocated statistics-holding structure.
[in]	timeout	Optional, controls blocking behavior.

No running transaction may be already attached to this thread. A new transaction is started and attached to the running thread.

The transaction will be instrumented. This structure is updated by the storage manager whenever a thread detaches from this transaction. The activity recorded during the time the thread is attached to the transcation will be stored in the per-transaction statistics.

Attention:

It is the client's responsibility to delete the statistics-holding structure.

Storage manager methods that must block (e.g., to acquire a lock) will use the timeout given. The default timeout is the one associated with this thread.

See also:

timeout_in_ms

static rc_t ss_m::begin_xct	(	tid_t &	tid,
		timeout_in_ms	timeout = WAIT_SPECIFIED_BY_THREAD
	)			[static, inherited]

Begin a transaction and return the transaction id.

Parameters:

[out]	tid	Transaction id of new transaction.
[in]	timeout	Optional, controls blocking behavior.

No running transaction may be attached to this thread.

Storage manager methods that must block (e.g., to acquire a lock) will use the timeout given. The default timeout is the one associated with this thread.

See also:

timeout_in_ms

static rc_t ss_m::commit_xct	(	bool	lazy = false,
		lsn_t *	plastlsn = NULL
	)			[static, inherited]

Commit a transaction.

Parameters:

[in]	lazy	Optional, controls flushing of log.
[out]	plastlsn	If non-null, this is a pointer to a log sequence number into which the storage manager writes the that of the last log record inserted for this transaction.

Commit the attached transaction and detach it, destroy it. If lazy is true, the log is not synced. This means that recovery of this transaction might not be possible.

Examples:

create_rec.cpp, log_exceed.cpp, sort_stream.cpp, and vtable_example.cpp.

static rc_t ss_m::commit_xct	(	sm_stats_info_t *&	stats,
		bool	lazy = false,
		lsn_t *	plastlsn = NULL
	)			[static, inherited]

Commit an instrumented transaction and get its statistics.

Parameters:

[out]	stats	Get a copy of the statistics for this transaction.
[in]	lazy	Optional, controls flushing of log.
[out]	plastlsn	If non-null, this is a pointer to a log sequence number into which the storage manager writes the that of the last log record inserted for this transaction.

Commit the attached transaction and detach it, destroy it. If lazy is true, the log is not synced. This means that recovery of this transaction might not be possible.

static rc_t ss_m::chain_xct	(	sm_stats_info_t *&	stats,
		bool	lazy = false
	)			[static, inherited]

Commit an instrumented transaction and start a new one.

Parameters:

[out]	stats	Get a copy of the statistics for the first transaction.
[in]	lazy	Optional, controls flushing of log.

Commit the attached transaction and detach it, destroy it. Start a new transaction and attach it to this thread.

Note:

The new transaction inherits the locks of the old transaction.

If lazy is true, the log is not synced. This means that recovery of this transaction might not be possible.

static rc_t ss_m::chain_xct

(

bool

lazy = false

)

[static, inherited]

Commit a transaction and start a new one, inheriting locks.

Parameters:

[in]

lazy

Optional, controls flushing of log.

Commit the attached transaction and detach it, destroy it. Start a new transaction and attach it to this thread.

Note:

The new transaction inherits the locks of the old transaction.

If lazy is true, the log is not synced. This means that recovery of the committed transaction might not be possible.

static rc_t ss_m::abort_xct

(

sm_stats_info_t *&

stats

)

[static, inherited]

Abort an instrumented transaction and get its statistics.

Parameters:

[out]

stats

Get a copy of the statistics for this transaction.

Abort the attached transaction and detach it, destroy it.

static rc_t ss_m::abort_xct

(

)

[static, inherited]

Abort a transaction.

Abort the attached transaction and detach it, destroy it.

static w_base_t::uint4_t ss_m::num_active_xcts

(

)

[static, inherited]

Return the number of transactions in active state.

While this is thread-safe, the moment a value is returned, it could be out of date. Useful only for debugging.

static void ss_m::attach_xct

(

xct_t *

x

)

[inline, static, inherited]

Attach the given transaction to the currently-running smthread_t.

It is assumed that the currently running thread is an smthread_t.

Definition at line 1173 of file sm.h.

static void ss_m::detach_xct

(

)

[inline, static, inherited]

Detach any attached from the currently-running smthread_t.

Sever the connection between the running thread and the transaction. This allow the running thread to attach a different transaction and to perform work in its behalf.

Definition at line 1238 of file sm.h.

static xct_t* ss_m::tid_to_xct

(

const tid_t &

tid

)

[static, inherited]

Get the transaction structure for a given a transaction id.

Parameters:

[in]

tid

Transaction ID.

Return a pointer to the storage manager's transaction structure. Can be used with detach_xct and attach_xct.

static tid_t ss_m::xct_to_tid

(

const xct_t *

x

)

[static, inherited]

Get the transaction ID for a given a transaction structure.

Parameters:

[in]

x

Pointer to transaction structure.

Return the transaction ID for the given transaction.

static xct_state_t ss_m::state_xct

(

const xct_t *

x

)

[static, inherited]

Get the transaction state for a given transaction (structure).

Parameters:

[in]

x

Pointer to transaction structure.

Returns the state of the transaction (active, prepared). It is hard to get the state of an aborted or committed transaction, since their structures no longer exist.

static rc_t ss_m::gather_xct_stats	(	sm_stats_info_t &	stats,
		bool	reset = false
	)			[static, inherited]

Get a copy of the statistics from an attached instrumented transaction.

Parameters:

[out]	stats	Returns a copy of the statistics for this transaction.
[in]	reset	If true, the statistics for this transaction will be zeroed.

---