Bug Tracking Plans

[Sam Hartman]

In a meeting last Wednesday, we decided to start down  the path of
adopting RT for our bug tracking needs to replace Gnats.  Eventual
goals of the project include:

* A public bug database accessible over the web

* Better MIME handling for bug reports and attachments 

* Integration with CVS to allow for closing out/updating bugs in
  commit messages

* Better understanding of how our bug database fits into our release
  process

* Better knowledge of what changes between Kerberos releases.

                            Current Scope

Over the past few weeks, those of us here at MIT have been playing
with an RT schema I've set up on my workstation.

I'm now working on packaging RT so it can be installed on one of our
servers and on automating the schema creation  so that as we try
modifying it we can consistently reproduce   the environment we are
discussing approving.

This mail is a heads up  for people with commit access letting them
know what changes are planned.  It's also a FYI  for the development
community at large.  You don't need to change how you're doing things
now; more mail will explain changes as we get to appropriate points in
the process.
I plan to  move things forward in the following steps:

1) Get RT up and running on our server with a copy of the Gnats
   database.

2)  Make some UI changes to RT that we've agreed internally.

3) Test out mail handling and mail related schema configuration

4)   Write a program to parse send-pr output and import into the RT
     schema.  This is partially done since we already have code to
     convert the Gnats database.

5) Investigate what will be required for the CVS integration.  RT has
   a cvs integration script, but  it will require  a bit of work for
   our environment.

6) Confirm everyone is happy.  Confirm current people who actively
   commit to the tree  can use the bug database, have the right
   permissions, etc.

7) Commit to using RT; pull gnats down, one final import of gnats
   database, switch incoming mail to go to RT.  Past this point,
   changes made to bugs in the RT schema will be preserved.

8)  Start following new bug policies (see below)

9) Open the system to the public.

10) Make sure people who have commit access but who commit
    infrequently are familiar with the new bug tracking system and can
    use it well enough that they can commit.


                     Public Access During Testing

We don't plan to support public access  to the bug system  during
testing.    Of course you can still report bugs and they will be
handled as they are now, but  until we have things reliably set up, we
won't expose a public search interface.  You can of course subscribe
to our bugs mailing list and you will receive bug reports as they
come in.

The reason for doing this is that we don't want to support public
queries/usage of a system we're still testing and debugging.  If you
are on krbcore and want to give someone a public access password,
because you believe they might find it useful or give valuable
feedback, that's fine, but please make it clear that the service is
unsupported and that people you give a password to should come to you,
not the mailing list with questions.


                           Process Changes


First, we'd like to get back to a procedure we followed during the
Kerberos 1.0 days regarding release branches.  All changes to the
release branch were tracked using explicit pull-up requests in the
form of bug entries.  A very small number of developers were allowed
to commit to the branch; back then Ted or Ted and Tom.  Pull-up
requests (or in the rare cases when appropriate, requests to introduce
a change to the branch not on the trunk) were sent to one of these
developers for integration.  It made it much easier to  track changes
and to know the stability of the release branch.  Especially if we are
doing much more frequent releases, this should be desirable and should
increase code quality.

Secondly, we'd like to try and have better integration between  CVS
and the bug tracking system.  In particular, I assume that by
including keyword value pairs, you can   either create a new bug or
attach  a commit log to an existing bug.    For example you might
commit with one of the following messages:

 ticket: 1097

 Accept patch to ftpd to work on multihomed hosts.   Don't call
 gethostname, instead call gethostbyaddr on the the interface the
 client connected to.

Or alternatively :

 ticket: new
 Subject: Add AES support to Kerberos
 component: krb5-libs

 Commit an AES crypto system along with enabling code. 


For those familiar with the existing RT CVS integration, I'm using
slightly different syntax.  I know; I'm trying to be simple not
pedantic.  We may adopt the RT syntax, although I suspect that people
will rightly complain it is too verbose and we'll just change the
script to have more readable syntax.

                            Bug Work Flow


 each bug will have three version fields: the version the bug is
reported in, the version that the bug will be fixed in, and the
version in which the bug is fixed.  The version the bug will be fixed
in (here after target release) is used to track release goals.  The
version fixed in is used to track what changes have been made to a
release.  In addition, a resolution field is required; for bugs that
were fixed by making code changes, this is blank.  It can be set to
invalid, worksforme and various other values to indicate why the bug
was closed but no changes were made.  The intent of this field is to
avoid claiming that mistaken bug reports or user questions were fixed
in a particular release.


Committing fixes:To commit a fix to the mainline, first decide if  the fix needs to be
documented in release notes.  If so, then make sure there is a bug
open for the fix (or open one during the commit).  KFM currently opens
bugs after committing the fix.  By linking the bugs to the commit it's
a lot easier to track changes, and we believe that the inconvenience
is justified.  The bug can be opened resolved if that's desirable and
you don't care what release the change eventually makes it into.  You
must leave the version_fixed_in field blank at this stage.  Then
commit the fix indicating in the commit message what  bug you are
fixing.  

If the bug simply needs to go onto the mainline and get into the next
branched release, that's all you have to do.  If the bug needs to go
into an upcoming release that is already branched you should set the
target release to that release.  You may need to generate a pull-up
request for the change to the release branch, but that depends on how
we decide to handle pull ups, which we decided was out of scope
Wednesday.  From a theoretical standpoint, marking the bug as targeted
for the release actually gives the release engineer enough
information.  



Tracking changes in a release: When it is time to branch a release,
the release engineer should identify all resolved bugs that neither
have a resolution field indicating no code was changed (invalid,
worksforme, etc) or a non-null version_fixed_in field.  These are all
the bugs that were fixed on the mainline but have not yet been in a
release.  These bugs should all be tagged with a version_fixed_in of
the release being branched.  Later as changes are pulled up to the
release branch,  the version_fixed_in should be set to the version
fixed for this release.  Both of these steps can benefit 
from automation to some degree.  

To search for the bugs fixed in a particular release, search for bugs
with a version_fixed_in of that release.

Tracking Release goals:  Whenever through appropriate procedure a
particular change/bug becomes a release goal, the target release field
of that ticket should be set to the release in question.

To check what bugs are release goals, search for bugs with the release
in question as the target release.

To check which release goals are not met yet, it depends on  whether
the branch has been cut.  Before the branch cut, check for release
goals that are not resolved.  After the branch cut check for release
goals that  have an empty version_fixed_in field.