Managing System Administration Tasks Using GNATS


Dave Plonka

Joining a system administration team can be a daunting task. Often problem-solving documentation is non-existent. Also, since system administrators spend much of their working life being interrupted, experienced coworkers are often unavailable to mentor. The challenge is for these junior SAs to perform the tasks assigned and inform their "customers" while juggling tasks in their work queue. Without a tracking and communication system in place, tasks could be neglected, which perpetuates a cycle of interruptions as complaintants find the need to reintroduce their issue. All SAs, regardless of experience, would probably welcome a more predictable introduction to their tasks at hand.

Assessing the Situation

SA work queues contain many "bite-sized" tasks. As such, these tasks are of a scope that they can often be described in a short synopsis by the complaintant. Junior system administrator tasks are often assigned to them based upon that administrators areas of expertise. These simple observations imply that if a sufficient synopsis of a task is supplied, a system could be devised by which the task could be routed automatically to an appropriately talented administrator. In addition, such a system could provide such features as e-mail notification of task milestones and journaling of the life-cycle of that task. An SA team member can query the database of journaled information regarding the resolution of problems to make use of past solutions logged by their peers when similar problems arise. It is just such a system that I propose.

Introducing GNATS

At the core of this system is the GNU Problem Report Management System (PRMS), known as GNATS. This package, intended as a system to track software bugs, was originally authored and is currently maintained by Cygnus Support. The name "PRMS" was the original package name and is used internally by Cygnus Support and appears in some of the documentation. Cygnus, an acronym for "Cover Your GNUS", provides support and development services for free software. According to the authors, GNATS was created in response to their inability to find a suitably configurable, distributed problem report management system for their own use. Like all GNU software, GNATS is freely available and it benefits from being maintained in a robust and portable fashion. According to Cygnus Support, in their document "Keeping Track: Managing Messages With GNATS, The GNU Problem Report Management System",

[GNATS] consists of several utilities which, when used in concert, formulate and administer a database of Problem Reports grouped by site-defined problem categories. It allows a support organization to keep track of problems (hence the term Problem Report) in an organized fashion. Essentially, GNATS acts as an active archive for field-separated textual data submitted through electronic mail.
GNATS makes this easy by automatically filing incoming problem reports into appropriate places, by notifying responsible parties of the existence of the problem and (optionally) sending an acknowledgment to the submitter that the report was received, and by making these Problem Reports accessible to queries and easily editable. GNATS is a database specialized for a specific task.
I would describe GNATS as a problem tracking tool of a design which is deeply rooted in the Unix paradigm: a database consisting of directories of flat ASCII files, colon-delimited-line-oriented configuration files, a sendmail-based distributed interface, and a regular-expression-fluent reporting facility. These features help to make GNATS feels like it's the Right Thing for Unix system administrators (SAs).

Solutions

Automated Task Assignment

With GNATS, a task for the system administrator exists as a Problem Report or "PR". Junior system administrators are immediately notified via e-mail when a new task enters their work queue. Because new tasks are routed and assigned automatically, they are not left unattended in an "in-box" or incoming mail folder. Even when a manager or senior system administrator is overwhelmed or unavailable, the work is not neglected. Tasks are routed to the system administrators that will ultimately do the work according to the relationship between the category in which the problem report was submitted and the responsible party as defined by the GNATS administrator. Amongst system administrators, the obvious candidate for GNATS administrator is the senior administrator that currently assigns tasks.

e-mail Notification

GNATS keeps all interested parties informed via e-mail:

The Details

The GNATS Database

With GNATS, problem reports reside in a directory and file-based database, similar to that of InterNetNews. An audit trail is maintained within each PR. When updating PRs, the system will prompt system administrators for the reason for a given change. This response, the change itself, the responsible party's name, and a timestamp, form the audit trail. The audit trail provides documentation on the sort of problems that are encountered, how such problems are resolved, and by whom. The database of raw information is everything a senior system administrator or manager may need to summarize the activity of his or her organization. More importantly, this database can be used as a learning tool and problem solving database for fellow administrators. For example, if a new PR arrives stating that the print queue foobar is dysfunctional, a fruitful first step might be for the responsible administrator to query the database of past PRs to learn if and how this problem has been resolved previously. GNATS provides the ability to query based on any PR fields or combination of fields (such as Responsible, Category, or State) using regular expressions for patterns that occur within PRs.

The GNATS User Interface

Obtaining GNATS

As of this writing, the latest official release is gnats-3.2. However, in anticipation of the upcoming gnats-4 release, a number of beta releases exist. Currently, gnats-3.100 ("three dot one hundred") beta is available. The recent beta releases contain important bug fixes to version 3.2, so I highly recommend starting there if gnats-4 is not yet available. The distribution may be obtained via anonymous ftp. See Table 1 for more information.

Installing GNATS

For my application of GNATS for SAs, the installation was straight forward. I suggest starting out with just one "site" on one Unix host, which is also the machine from which problem reports will be submitted. Excellent documentation is provided, in texinfo format, with the distribution. I found it more convenient to read the document reformatted in HTML, and was able to do so easily with the freely available texi2html translator. HTML and PostScript versions of the document are also available. See Table 1.

Configuring GNATS

Once GNATS is installed, the categories, responsible and submitters files are all that must be maintained. Our categories file contained these options: novell, windows, unix, network, printers, and misc. The responsible file contains the SAs names, against which the ">Responsible:" field is validated when a PR is modified, and their respective e-mail addresses. The submitters file contains names of organizations which submit problems, and an optional list of parties to be notified.

GNATS Enhancements

While GNATS is serviceable as-is, I've employed these additions to either provide convenient interfaces and to provide additional features.

wwwgnats
wwwgnats is a WWW front end for GNATS. It requires that the Unix host on which the GNATS installation is performed also be a WWW server. wwwgnats strong point is its reporting capabilities which display either "active bugs by status and person", or "all bugs by status and category" in a table format. It is freely available and is covered by the terms of the GNU General Public License. Further information about wwwgnats see see Table 1.

form-pr
form-pr.cgi is my own custom WWW front-end for GNATS send-pr. Like wwwgnats, it is written in perl using HTML forms and the Common Gateway Interface (CGI) and is strictly optional. My goal was to provide a stream-lined alternative to the command-line-based send-pr, for users who are not comfortable using a Unix shell or editor. See Figure 1.

remind-pr
remind-pr is my own custom utility to remind SAs about neglected PRs. By default, GNATS provides no "Deadline" or "Due-Date" field and therefore will not enforce that a task is completed on time. However, unlike software bugs, SA tasks often do have deadlines. What I've done is simply to add two optional fields to the template PR form: Due: and Remind:. remind-pr is a perl script that just provides some glue between GNATS' query-pr and the Unix calendar reminder facility.

Living with GNATS

With GNATS, tasks no longer fall through the cracks. While objectively this is an obvious advantage, subjectively it means that GNATS is a tireless nag. As with other automatically generated e-mail, recipients of problem reports tend to dismiss them out of hand. To address this potential problem the GNATS administrator should:

Also, to avoid initial confusion, be sure to make the documentation available up-front to end users. If it's difficult for your "customers" to find how to introduce problem reports into your new problem tracking system, you'll find your complainant using the traditional method of interrupting the SAs once again with phone calls or face-to-face confrontations, eliminating some of the advantages of your automated tracking system.

Summary

GNATS can be an effective tool for Unix system administrators. While originally conceived to track software bugs, GNATS' assignment, notification, and journaling capabilities shine just as brightly when employed in the tracking of system administration tasks. Once configured, it routes tasks directly to a system administrator with the appropriate skill-set. Junior system administrators and their superiors can examine their work queue, and are automatically notified of any changes in priority or responsibility. This system returns to system administrators' the time they would have had to spend building, organizing, and reporting on their queue of work.

Dave is a benevolent hacker and Unix aficionado. He has a B.S. in Computer Science from Carroll College in Waukesha, Wisconsin (1991) and employed as Lead Systems Programmer at McHugh Freeman, (http://www.mfa.com) and can be reached at Dave.Plonka@mfa.com or via http://www.cc.edu/~plonka.