New Python test framework

[ghudson@MIT.EDU]

I've committed a new Python module k5test.py and some build system
logic to make use of it.  As always, it's easier to change stuff like
this early after it's introduced.  For convenience of review, here is
the --help output from a test script:

    Usage: t_anonpkinit.py [options]

    Options:
      -h, --help            show this help message and exit
      -v, --verbose         Display verbose output
      -p PASS, --pass=PASS  If a multi-pass test, run only PASS
      --debug=NUM           Debug numbered command (or "all")
      --debugger=COMMAND    Debugger command (default is gdb --args)
      --stop-before=NUM     Stop before numbered command (or "all")
      --stop-after=NUM      Stop after numbered command (or "all")
      --shell-before=NUM    Spawn shell before numbered command (or "all")
      --shell-after=NUM     Spawn shell after numbered command (or "all")

and here is the module docstring:

"""A module for krb5 test scripts

Put test script names in the PYTESTS make variable to get them run
with the appropriate PYTHONPATH during "make check".  Sample test
script usage:

    from k5test import *

    # Run a test program under a variety of configurations:
    for realm in multipass_realms():
        realm.run_as_client(['./testprog', 'arg'])

    # Run a test server and client under just the default configuration:
    realm = K5Realm()
    realm.start_server(['./serverprog'], 'starting...')
    realm.run_as_client(['./clientprog', realm.host_princ])

    # Inform framework that tests completed successfully.
    success()

By default, the realm will have:

* The name KRBTEST.COM
* Listener ports starting at 61000
* Four different krb5.conf files for the client, server, master KDC,
  and slave KDC, specifying only the variables necessary for
  self-contained test operation
* Two different kdc.conf files for the master and slave KDCs
* A fresh DB2 KDB
* Running krb5kdc and kadmind processes
* Principals named realm.user_princ and realm.admin_princ; call
  password('user') and password('admin') to get the password
* Credentials for realm.user_princ in realm.ccache
* Admin rights for realm.admin_princ in the kadmind acl file
* A host principal named realm.host_princ with a random key
* A keytab for the host principal in realm.keytab

The realm's behaviour can be modified with the following constructor
keyword arguments:

* realm='realmname': Override the realm name

* portbase=NNN: Override the listener port base; currently three ports are
  used

* testdir='dirname': Override the storage area for the realm's files
  (path may be specified relative to the current working dir)

* krb5_conf={ ... }: krb5.conf options, expressed as a nested
  dictionary, to be merged with the default krb5.conf settings.  The
  top level keys of the dictionary should be 'all' to apply to all
  four krb5.conf files, and/or 'client'/'server'/'master'/'slave' to
  apply to a particular one.  A key may be mapped to None to delete a
  setting from the defaults.  A key may be maped to a list in order to
  create multpile settings for the same variable name.  Keys and
  values undergo the following template substitutions:

    - $type:     The configuration type (client/server/master/slave)
    - $realm:    The realm name
    - $testdir:  The realm storage directory (absolute path)
    - $buildtop: The root of the build directory
    - $srctop:   The root of the source directory
    - $plugins:  The plugin directory under $buildtop/util/fakedest
    - $hostname: The FQDN of the host
    - $port0:    The first listener port (portbase)
    - ...
    - $port9:    The tenth listener port (portbase + 9)

  When choosing ports, note the following:

    - port0 is used in the default krb5.conf for the KDC
    - port1 is used in the default krb5.conf for kadmind
    - port2 is used in the default krb5.conf for kpasswd
    - port3 is the return value of realm.server_port()

* kdc_conf={...}: kdc.conf options, expressed as a nested dictionary,
  to be merged with the default kdc.conf settings.  The top level keys
  should be 'all' or 'master'/'slave'.  The same conventions and
  substitutions for krb5_conf apply.

* create_kdb=False: Don't create a KDB.  Implicitly disables all of
  the other options since they all require a KDB.

* krbtgt_keysalt='enctype:salttype': After creating the KDB,
  regenerate the krbtgt key using the specified key/salt combination,
  using a kadmin.local cpw query.

* create_user=False: Don't create the user principal.  Implies
  get_creds=False.

* create_host=False: Don't create the host principal or the associated
  keytab.

* start_kdc=False: Don't start the KDC.  Implies get_creds=False.

* start_kadmin=False: Don't start kadmind.

* get_creds=False: Don't get user credentials.

Scripts may use the following functions and variables:

* fail(message): Display message (plus leading marker and trailing
  newline) and explanatory messages about debugging.

* output(message, force_verbose=False): Place message (without any
  added newline) in testlog, and write it to stdout if running
  verbosely.

* password(name): Return a weakly random password based on name.  The
  password will be consistent across calls with the same name.

* stop_daemon(proc): Stop a daemon process started with
  realm.start_server() or realm.start_in_inetd().  Only necessary if
  the port needs to be reused; daemon processes will be stopped
  automatically when the script exits.

* multipass_realms(**keywords): This is an iterator function.  Yields
  a realm for each of the standard test passes, each of which alters
  the default configuration in some way to exercise different parts of
  the krb5 code base.  keywords may contain any K5Realm initializer
  keyword with the exception of krbtgt_keysalt, which will not be
  honored.  If keywords contains krb5_conf and/or kdc_conf fragments,
  they will be merged with the default and per-pass specifications.

* success(): Indicate that the test script has completed successfully.
  Suppresses the display of explanatory debugging messages in the
  on-exit handler.

* buildtop: The top of the build directory (absolute path).

* srctop: The top of the source directory (absolute path).

* plugins: The plugin directory under <buildtop>/util/fakedest.

* hostname: This machine's fully-qualified domain name.

* null_input: A file opened to read /dev/null.

* args: Positional arguments left over after flags are processed.

* verbose: Whether the script is running verbosely.

* testpass: The command-line test pass argument.  The script does not
  need to examine this argument in most cases; it will be honored in
  multipass_realms().

* Pathname variables for programs within the build directory:
  - krb5kdc
  - kadmind
  - kadmin
  - kadmin_local
  - kdb5
  - ktutil
  - kinit
  - klist
  - kdestroy
  - kpasswd
  - t_inetd
  - kproplog
  - kpropd
  - kprop

Scripts may use the following realm methods and attributes:

* realm.run_as_client(args, **keywords): Run a command with an
  environment pointing at the client krb5.conf, obeying the
  command-line debugging options.  Fail if the command does not return
  0.  Log the command output appropriately, and return it as a single
  multi-line string.  Keyword arguments can contain input='string' to
  send an input string to the command, and expected_code=N to expect a
  return code other than 0.

* Similar methods for the server, master KDC, and slave KDC
  environments:
  - realm.run_as_server
  - realm.run_as_master
  - realm.run_as_slave

* realm.server_port(): Returns a port number based on realm.portbase
  intended for use by server processes.

* realm.start_server(args, sentinel): Start a process in the server
  environment.  Wait until sentinel appears as a substring of a line
  in the server process's stdout or stderr (which are folded
  together).  Returns a subprocess.Popen object which can be passed to
  stop_daemon() to stop the server, or used to read from the server's
  output.

* realm.start_in_inetd(args, port=None): Begin a t_inetd process which
  will spawn a server process within the server environment after
  accepting a client connection.  If port is not specified,
  realm.server_port() will be used.  Returns a process object which
  can be passed to stop_daemon() to stop the server.

* realm.create_kdb(): Create a new master KDB.

* realm.start_kdc(): Start a krb5kdc with the realm's master KDC
  environment.  Errors if a KDC is already running.

* realm.stop_kdc(): Stop the krb5kdc process.  Errors if no KDC is
  running.

* realm.start_kadmind(): Start a kadmind with the realm's master KDC
  environment.  Errors if a kadmind is already running.

* realm.stop_kadmind(): Stop the kadmind process.  Errors if no
  kadmind is running.

* realm.stop(): Stop any KDC and kadmind processes running on behalf
  of the realm.

* realm.addprinc(princname, password=None): Using kadmin.local, create
  a principle in the KDB named princname, with either a random or
  specified key.

* realm.extract_keytab(princname, keytab): Using kadmin.local, create
  a keytab for princname in the filename keytab.  Uses the -norandkey
  option to avoid re-randomizing princname's key.

* realm.kinit(princname, password=None, flags=[]): Acquire credentials
  for princname using kinit, with additional flags [].  If password is
  specified, it will be used as input to the kinit process; otherwise
  flags must cause kinit not to need a password (e.g. by specifying a
  keytab).

* realm.klist(client_princ, service_princ=None, ccache=None): Using
  klist, list the credentials cache ccache (must be a filename;
  self.ccache if not specified) and verify that the output shows
  credentials for client_princ and service_princ (self.krbtgt_princ if
  not specified).

* realm.klist_keytab(princ, keytab=None): Using klist, list keytab
  (must be a filename; self.keytab if not specified) and verify that
  the output shows the keytab name and principal name.

* realm.run_kadminl(query): Run the specified query in kadmin.local.

* realm.realm: The realm's name.

* realm.testdir: The realm's storage directory (absolute path).

* realm.portbase: The realm's first listener port.

* realm.user_princ: The principal name user@<realmname>.

* realm.admin_princ: The principal name user/admin@<realmname>.

* realm.host_princ: The name of the host principal for this machine,
  with realm.

* realm.krbtgt_princ: The name of the krbtgt principal for the realm.

* realm.keytab: A keytab file in realm.testdir.  Initially contains a
  host keytab unless disabled by the realm construction options.

* realm.ccache: A ccache file in realm.testdir.  Initially contains
  credentials for user unless disabled by the realm construction
  options.

When the test script is run, its behavior can be modified with
command-line flags.  These are documented in the --help output.