by Jim Gast, published in Computer Technology Review, March, 1991.
The new high speed search 8mm tape transports add a new dimension to these devices which significantly widens the gap between backup systems and archiving systems. Backup systems are focused on taking a snapshot of the contents of a disk. Archiving systems preserve a large number of unique versions of important files across time.
When the first 8mm tape transports were introduced, they substantially reduced the amount of manual intervention required to perform a full snapshot of a large amount of data. Because the media holds 2.2 GBytes, system managers could run backups of large fileservers or small local area networks overnight, unattended.
But these drives had to use ingenious sleight-of-hand to make themselves appear to be start-stop devices. Mathematics and buffering overcame the uncertainties of very high bit densities. These drives have a density of 35 Mbits/in2. In order to operate on imperfect media, the drive uses a 256KByte buffer and both read-after-write and error correcting technology. Figure 1 shows a typical helical scan environment with stripes of data as they would be recorded on perfect media. (For more information about helical scan techniques, see the May 1987 issue of Computer Technology Review.)
Figure 1: Stripes of data recorded on a perfect media contain blocks in perfect sequence (here, the first four stripes from right to left).
Unfortunately, the real world is not so kind to us. The most common defects in magnetic media are longitudinal (parallel to the edge) and dropouts (small blotches), and either of these can occur either before or after data is written. Figure 2 shows what happens when the read-after-write head is unable to read a block.
Figure 2: An example of what happens when the read-after-write head is unable to read a block on imperfect media: block 5 was rewritten after block 15 and a logitudinal scratch deestroyed the first two copies of blocks 33, 41, and 55.
In this example, a bar is shown across the first attempt to write block 5. The read-after-write head is much less tolerant than the normal read head and was unable to read the original block 5. Since all data is maintained in the RAM buffer until it is verified as correctly written to tape, the data is still available for rewriting on a different area of the tape. This can be repeated, if necessary, several times. But error recovery would be tricky if we let the 'good' copy of a block be more than 9 tracks away from the first attempt to write that block. If there are so many bad blocks that the tape transport can't write a good copy within 9 tracks of the first attempt, the tape transport reports an error.
The example shows one of the fundamental differences between 4mm and 8mm technology. DAT (4mm) chose the simpler approach of rewriting whole tracks at a time, whereas the 8mm technique intentionally writes all rewritten copies at different 'height' on the stripe than the prior copy. This is illustrated by the handling of blocks 33, 41, and 55 in Figure 2.
These techniques protect against defects that were already on the tape before the data was written, but they do nothing to protect against defects that occur as the tape ages and wears. To correct those errors, the 8mm format includes a substantial amount of error correcting code (ECC) (See the March 1990 issue of Computer Technology Review for a more in-depth treatment of ECC.) The Reed-Solomon ECC adds almost 28% overhead to the data, but completely corrects error bursts up to 240 bytes.
ECC technology highlights the differences between logical position and physical position of the tape. Just because you find a readable copy of the block you want, you may not be positioned before a logically subsequent block, and this copy might not be either the first or the last readable (or correctable) copy of that block. Backspacing to the prior logical block may require moving as far as 10 tracks backward or 9 tracks forward! Imagine "positioning" to block 33 in Figure 2 and then "spacing" forward one block to be logically positioned at block 34. Similarly, moving forward from block 41 in Figure 2 requires physically going backward 2 tracks.
Because of the complexities, the original version of the 8mm tape transport (8200) concentrated on reliability and streaming rather than sophisticated searching. Those transports could only search for file marks (a relatively big, easy to find pattern) at 10X nominal speed. The new high speed search 8mm drives (8200SX at 2.5 GBytes, or 8500 at 5 GBytes) record extra information on the tape in SEARCH FIELDS allowing the transport to physically position the tape at 75X nominal speed. The search fields can be read while flying past. True reading starts before the first possible place for the specific desired block and continues until a readable (or correctable) copy is found. An error is not reported unless the transport reads 9 tracks past the first occurance of a block with a higher number than the desired block.
Most software had to be modified to take advantage of the benefits of fast search. Two additional SCSI commands were implemented: READ POSITION and LOCATE. The READ POSITION command can be used after writing any block: save the address, then use the LOCATE command to return to the logical point on the tape at very high speed.
Backup vendors are not in a hurry to rewrite their host software because they don't gain as much from the increase. After all, the READ and WRITE speeds are still the same, as is REWIND (which was always 75X nominal speed.)
An archivist carefully and constantly tends the tape library. In the course of maintaining a well-managed perpetual archive, the archivist moves some tapes offsite and rotates others back onsite. Some tapes are used to safeguard old copies of dormant files that have been intentionally removed from the disk. And several tapes are used very frequently, and rotated to maximize the likelihood that any recent version of any file is on as many tapes as practical.
Before a tape is taken offsite it is freshened up, adding stable files for long-term storage and separately adding checkpoints of any current files not yet taken offsite. Much later, when these tapes are brought back onsite, those long-term files are still valuable, but the checkpoints are redundant and obsolete. Archiving software knows how to clean these obsolete checkpoints off of tapes that were otherwise used for long-term purposes, freeing up the space for more permanent storage. Over time, these tapes accumulate filesets containing the stable files from many cycles, spread across time. And all along, the archivist keeps careful records and logs of tapes used and files saved.
Archiving software seldom starts at the beginning of a new tape. We need to bring back old tapes often enough that we seldom need new tapes -- and when an old tape is used we need to perform a variety of cleaning, housekeeping, and archiving functions before we can archive the files that changed recently. The bad news is that this requires a lot more searching and repositioning of the tape. Not only is the data added to the tape, but the directory on the tape must also be updated to reflect the new filesets added. The good news is that archiving builds a library that contains a much richer history of the files without using an unmanagably large number of tapes.
Once a particular version of a file has been permanently saved to three different tapes, the archivist knows that the file is "fully protected." Once a file is fully protected, it is no longer copied week after week to tape after tape. After the first three "save" filesets, later filesets are much smaller.
Over time, an archiving system will create tapes with many small filesets. In contrast, a backup system will tend to start at the start of the tape -- overwriting all of the old data without considering it -- and write one copy of each file regardless of how many other copies are already in the tape library. Ironically, the backup system is (without even looking at the old contents of the tape) often writing a file that is identical to a file that was already on the tape and being overwritten. Worse yet, if the file is not identical, then the old copy that is being overwritten might be the last extant copy of that version of that file. Backup systems seldom need (and do not readily profit from) high speed search.
The real pressure for responsiveness is when an end user needs to restore a single file from a few days, weeks, or months ago. An archiving system uses its on-disk catalog to find exactly where each version of the file is recorded on tape. In fact, it also knows where the other identical copies of that version are too. Without such a catalog, the end user would have to play a guessing game about which tape to mount. The archiving system displays the list of tapes that contain the desired version. After the end user selects one of the tapes, the tape transport verifies its identity and then uses high speed search to position to the right fileset. A search from the start of the tape to any point on the tape takes less than 2 minutes.
Tapes written with the new transports are readable in a first generation transport (but cannot make use of the search fields). Tapes written with the old transport are readable in the new transport, but, again, without the ability to search at high speed. Tapes written by one with later data appended by the other can only use high speed search in areas that have search fields recorded.
As more Local Area Networks use fault-tolerant techniques like duplexing and mirroring of disks, backup is still needed but full disk restores are less often exercised and are no longer the primary focus of the tape system. Archiving is the next stage in the logical evolution to Hierarchical Storage Mangement: tracking prior versions of valuable disk files and ensuring a managed number of redundant copies of files. High speed search in 8mm tape transports makes them significantly more convenient and responsive in these increasingly automated environments.
This page is maintained by Jim Gast, jgast@cs.wisc.edu. Suggestions and corrections are very welcome.
This page was last modified
Copyright 1991 by Jim Gast. All rights reserved.