Network device Configuration Management System

This document describes the Network device Configuration Management System (NetCMS): a system for tracking Cisco router and switch configurations.

• Introduction

  NetCMS is a Network Configuration Management System for network devices. Currently it supports only Cisco routers and switches. The underlying router "getcnf" utility have been tested with approximately 200 Cisco routers (2500, 3000, 4500, 7000 and 7500 series) running IOS versions 9.x through 11.25. The switch "e-getcnf" utility currently supports only Cisco's Catalyst 5000, and optionally installed ATM modules (where the Catalyst 5000 is operating as an ATM "edge" device).

  NetCMS is implemented as an RCS and make(1)-based project. Essentially, NetCMS is a revision control system for network device configurations. Also, NetCMS generates summary reports (e-mailed as MIME attachments) containing such information as:

  • a list of the devices that are "locked" (via RCS), indicating that they are being currently being configured by a network engineer. (Engineers can use NetCMS to "lock-out" a device for their own use. NetCMS will indicate that it is locked by that person if anyone else tries to perform a NetCMS update on that device while it is locked.)
  • a list of devices that have been added or removed. (That is, new devices that have been introduced to the network or taken out of service.)
  • a list of differences between devices' running configuration(s) and "flash" (stored) configurations.
  • a list of differences between devices' "checked-in" (via RCS) configurations and "flash" configurations.
  • the hand-written explanations (RCS "log" messages), that have been supplied on "check-in" by network engineers, explaining the configurations changes that have been made during the current reporting period.

---

• NetCMS Overview

  The Network device Configuration Management System is a make and RCS-based project. The NetCMS working directory, specified using the --prefix= and --project= options to configure at install time, is where everything resides and procudures such as updating network device (ie. routers and switches) configurations are performed.

  • Process Flow

  This diagram summarizes the relationships amongst components of this system. For instance, it indicates that the network of routers is an input to everything. Also, the hosts file is input to all the normal user procedures (make new and make update) and the scheduled e-mail report generation (make clean mailout).

        _______
       (       )
  .---( Network )---> make ...hosts ---> ...hosts ---> make edit ---> hosts,v ---.
  |    (_______)                                                                 |
  |                                                                              |
  |    .---------------------------------------------------------------------<---'
  |    |         
  |    |         
  +--->+---> make new ---> device.conf,v ---------------.
  |    |                                                |
  |    |    .---------------------<---------------------+
  |    |    |                                           |
  +--->+--->+---> make update ---> device.conf,v --->---'
  |    |    |
  |    |    |
  |    |    |
  `--->+--->+---> make clean mailout ---> ...configurations, ...*.mailout --->---.
                                                                                 |
  							  [e-mail reports] <---+
  									       |
  							     make mailout <----+
                                                                                 |
  							     make rlog <-------+
  							                       |
  						             make somediff <---'
  

  ---

  • Directory Structure

  NetCMS/
   | Makefile, hosts, *.conf,v, ...*
   |
   `--...configurations
       |--RCS/
       |   *.conf
       |
       |--flash/
       |   *.conf
       |
       `--running/
           *.conf
  

  • NetCMS

    This is the primary working directory and is where makes are performed.

  • NetCMS/...configurations

    This is a temporary directory.

  • NetCMS/...configurations/RCS

    This is a temporary directory where the most recent RCS revision is written while generating reports.

  • NetCMS/...configurations/flash

    This is a temporary directory where the most recently downloaded flash configurations are written.

  • NetCMS/...configurations/running

    This is a temporary directory where the most recently downloaded running configurations are written.

  ---

  • Menu Options

  NetCMS can be run using an interactive menu. It is invoked by simply running make, with no arguments in the directory.

  update

  Use this option to update an existing device's configuration. This option can also be used at any time to check whether or not the current flash configuration has been stored in the RCS device.conf,v file.

  new

  Use this option to add a new device's configuration to RCS. Before this option is used, it must exist in the hand-maintained hosts file. (You can add an entry to the hosts file using "make edit".)

  mailout

  Use this option interactively if you wish to regenerate the e-mail. You will be given the option to change the To: and Subject: and the message body itself. (mutt, the MUA which is used to compose the e-mail, will require you to change the message content in some way, otherwise it will think that you don't want to send it.)

  done

  Hopefully this is self-explanatory.

  edit

  Edit the NetCMS hosts file. This would be used, for instance, to add a new device before running new or to modify the IP address or name of an existing device. (In the latter case, the RCS revision file (,v) for that device should be renamed as well.)

  show

  Show recent changes that have been logged using NetCMS. By recent we mean changes since the last time the NetCMS e-mail reports have been generated.

  ---

  • NetCMS make Targets

  • menu

  This is the default target. It is for interactive use only and produces a menu of choices.

  • new

  This target, for interactive use, is used to add a new device to NetCMS. You will be prompted for the device name.

  • edit

  This target, for interactive use, is used to edit the NetCMS hosts file. Because this file is stored sorted by IP address, you show use this target rather than editing the file manually so that unnecessary differences do not show up in the hosts-diff report.

  • locked.getcnf

  This target, for interactive use, is used to get all the configurations for the device configuration files that are currently locked out by you. It's primary purpose is to simplify the process of doing the same modification to a number of device at once. (It would done following a manual co -l and preceding the ci of the list of device.conf files on which you wish to operate.)

  • update

  This target, for interactive use, is used to update an existing device's configuration in NetCMS. You will be prompted for the device name.

  • .conf.update

  This default rule, for internal use only, is responsible for performing the actual update logic. The quickest way to update a device is to do this:

     $ make device.update
     

  • .conf,v.conf

  This is default rule, for internal use only. It tells make how to retrieve a configuration file from RCS.

  • hosts

  This target, for internal use only, tells make how to retrieve the hand-maintained hosts file from RCS.

  • ...hosts

  This target will cause make to rebuild a current impression of the hosts file. If the hosts file is out of date, you can use the content of ...hosts when you update the hosts file manually.

  • ...configurations

  This target will cause make to build a sub-directory (called ...configurations) containing the current flash and running configurations from the device defined in ...hosts.

  • rlog

  This target will cause make to produce an rlog-based report (to standard output) of all RCS log comments in the device configuration version files since the last time they were generated by the ...rlog.mailout targer.

  • show

  This target will make rlog and pipe it to $PAGER.

  • flash-diff

  This target, for both interactive and internal use, generates diff output showing differences between the configuration checked into RCS as the head revision vs. the configuration in flash.

  • running-diff

  This target, for both interactive and internal use, generates diff output showing differences between the device' configuration in flash vs. running configuration.

  • hosts-diff

  This target, for both interactive and internal use, generates diff output between the hand-maintained hosts file and the automatically generated hosts file: ...hosts.

  • check-locks

  This target, for both interactive and internal use, generates rlog ouput showing which NetCMS files are currently locked out.

  • ...check-locks.mailout
  • ...hosts-diff.mailout
  • ...running-diff.mailout
  • ...flash-diff.mailout
  • ...rlog.mailout

  These targets are for internal use by the mailout target. Primarily they simply create files to be used as e-mail attachments by recursively invoking make on the targets of the corresponding names.

  • mailout

  This target, for either interactive and batch use, is used to generate and e-mail a NetCMS status summary and set of reports as MIME attachments.

  • clean

  This target, primarily for batch use, is used to remove all ... targets. A typical use would be to clean things up before running make mailout in batch.

  ---

• NetCMS Installation

  • Prerequisites

    NetCMS requires the following packages:

    • ksh

      The NetCMS Makefile rules are written in ksh syntax. If you are using a system without ksh, use the freely-available pdksh (installed ask ksh). Possibly even bash (linked to ksh) will suffice, although I have not tested NetCMS with it.

    • RCS

      RCS is a Revision Control System for files. NetCMS uses RCS to maintain revisions of all its configuration files (such as the NetCMS hosts file) and the network device configuration (*.conf) files.

    • perl5

      perl is used both by the NetCMS Makefile and by component scripts.

    • mutt

      mutt, an MUA, is used to compose e-mail with MIME attachments both interactively and in "batch" mode, for NetCMS cron-scheduled reports.

    • IOSchat and getcnf

      getcnf, a perl script which uses IOSchat and is supplied with that distribution, is the utility which NetCMS uses to retrieve the configuration from Cisco routers.

    • e-getcnf

      e-getcnf, which requires Comm.pl, is the utility which NetCMS uses to retrieve the configuration from Cisco Catalyst 5000 switches, sometimes used as ATM edge devices (hence the prefix e-).

  ---

  • Step-by-step Installation

    This install should be performed as a user as the group to which all users of NetCMS will belong. (NetCMS is implemented as an RCS project, so the working-directory created by the install will be group writable.)

    1. Extract the distribution:

           $ gunzip -c NetCMSversion.tar.gz |tar xf -
           

    2. Configure:

       If for instance, you wished to install NetCMS in the current users home directory (in a sub-directory called NetCMS), you might configure it as follows:

           $ ./configure --help # note --with-... options
           $ ./configure --prefix=/opt/net --project=cms # for example
           

    3. Make and Install:

           $ make
           $ make -n install
           $ make install
           

    4. Create the initial NetCMS hosts file:

           $ cd /opt/net/cms
           $ make
           1) update an existing device
           2) new device
           3) mailout
           4) QUIT
           5) edit NetCMS hosts file
           6) show changes since last e-mail report
           #? 5
           

    5. Add your routers and/or switches:

           $ cd /opt/net/cms
           $ make
           1) update an existing device
           2) new device
           3) mailout
           4) QUIT
           5) edit NetCMS hosts file
           6) show changes since last e-mail report
           #? 2
           

    6. Test the reporting features by running the mailout target:

           $ cd /opt/net/cms
           $ make clean mailout
           

  ---

• NetCMS Usage FAQ

  1. How do I set up my environment for working with NetCMS?

     Since NetCMS is an RCS-based project, maintained in a shared directory, it is important that your shell environment is set up properly. For instance your umask should be 002 so that files you create, and are created by the NetCMS Makefile, have group read and write permission. If you use the Korn, POSIX, or Bourne Again, or Bourne shell this would be set up in your .profile:

        umask 02
        

     If you're login group is not the group which owns the NetCMS project directory, you can use the newgrp command to change into that group before doing NetCMS work:

        $ newgrp - wnstaff
        

     Something like this might be useful in your .profile to set up conditional behavior based upon which group you are in (not that this is Bourne, Bourne again, Korn, or POSIX shell syntax):

        case `id -gn` in
        wnstaff)
           cd /home/wiscnet
           PATH=$PATH:/home/wiscnet/bin
           umask 02
           ;;
        *)
           umask 022
           ;;
        esac
        

     Dave Plonka, Oct 15 1997

  ---

  2. What do I do after making a change on an existing router?

     While in the NetCMS working directory, there are three ways to update the revision-controlled router configuration:

     Using a menu:

        $ make
        

        $ make update
        

        $ make router.update
        

     The makefile will invoke the co, ci, getcnf, diff, and rcsdiff commands for you. If changes have been made you will be prompted to enter a meaningful log message.
     Dave Plonka, Oct 15 1997

  ---

  3. How do I add a new router?

     While in the NetCMS working directory... First, that router must exist in the NetCMS hosts file. If it is not already there the hosts file can be edited from the NetCMS menu or like this:

        $ make edit
        

     Then, do this:

        $ make new
        

     Dave Plonka, Oct 15 1997

  ---

  4. How do I update a whole slew of routers en masse?

     To do this in one fell swoop, you'll need to know a bit more about RCS. Before you start it would be a good idea to be sure that there aren't other changes pending to be checked in:

        $ make flash-diff |less
        

     Here's a step-by-step example to update all routers:

        $ co -l *.conf,v # lock out all router configurations
        

     Now that all routers are "reserved" for your exclusive use, perform your updates, using a tool of your choice (eg. expect, perl with chat2.pl, perl with IOSchat.pm, etc.).
     Next, you'll need to fetch the current configurations for all the locked configuration files. This will take a while:

        $ rm -f nohup.out && nohup make locked.getcnf &
        

     After that has completed, check that the differences are what you expect using rcsdiff. An invocation like this might be useful:

        $ egrep 'strict;$' *.conf,v | grep -v 'locks;' |grep ${LOGNAME?} |cut -f1 -d: |xargs rcsdiff 2>&1 |less
        

     If the changes are all part of your mass change, you could check all the files in with the same log message:

        $ egrep 'strict;$' *.conf,v | grep -v 'locks;' |grep ${LOGNAME?} |cut -f1 -d: |xargs ci -m'put a meaningful log message here!'
        

     That's it! At this point, the flash and running configuration files in the ...configuration sub-directory are out of date. These files are that used for report generation, under the mailout Makefile target. This will be regenerated by the next cron-scheduled make clean mailout.
     Dave Plonka, Oct 16 1997

  ---

  5. How do I see what changes been done recently? To view the log messages that have been entered since the last time they were reported (eg. via e-mail from a cron job), you can do this in the NetCMS working directory:

        $ make
        

     or

        $ make show
        

     Dave Plonka, Oct 16 1997

  ---

  6. How do I retrieve a specific revision of a configuration?

     First use the RCS rlog(1) command to see a list of revisions stored in a given router's RCS revision file (router.conf,v):

        $ rlog router.conf
        

     Then, use the RCS co(1) command to rerieve that version:

        $ co -rrevision_number router.conf
        

     If you'd like to retrieve the revision to a file other than the working file (ie. router.conf), use the -p option to send it to standard output. Eg.:

        $ co -prevision_number router.conf |less
        

     or

        $ co -prevision_number router.conf >/tmp/router.conf.revision_number
        

     Please don't leave these temporary files in the NetCMS directory.
     Dave Plonka, Oct 24 1997

  ---

  Please direct questions, comments, or suggestions to:
  Dave Plonka <plonka@doit.wisc.edu>

  $Id: NetCMS.html,v 1.1 1997/12/17 17:14:49 plonka Exp $