Recovering from a removed master key

[Charles Adams]

Hello Kerberos community,

I have created a problem with my Kerberos installation -- the master key K/
M at MY.REALM.ORG was accidentally removed while cleaning out some really
crufty stuff, and I'd like to try and recover from this as gracefully as
possible. Here are some details:

I have a kerberos realm with a master server and 2 slaves. The slaves run
kpropd and the master kprops them hourly with a database dump. The
propagation has been failing (well, the database dump step of the script I
use to propagate has been failing) since about 26-Jan-2015, which is when I
believe the master key principal was removed. I do still have what I
believe is the matching stash file. The slave KDCs still have a k/m
principal. They have stash files as well, but those files are larger than
the stash file on the master. I was able to dump the K/M at MY.REALM.COM
principal on one of the slaves out using kdb5_util:

slave1# kdb5_util dump -ov -verbose ~/kerbmaster-ov K/M at MY.REALM.ORG
slave1# kdb5_util dump -verbose ~/kerbmaster K/M at MY.REALM.ORG

I do also still have a full database dump from the master from prior to the
believed key deletion event.

Since the K/M key was removed, authentication, kinit, user creation, and
password changes have occurred without visible error (although users
attempting to authenticate to the slave servers with a
new-since-26-Jan-2015 password fail, obviously). The master KDC process has
been running continuously since prior to the K/M deletion event.

Is there a recommended recovery procedure that I can follow that would
cause a minimum of disruption in my organization? Any hidden gotchas? My
first reaction is to take the dumped /tmp/kerbmaster file above and import
it into the master server like this:

master# kdb5_util load -verbose -update ~/kerbmaster

But given the high stakes and my relative lack of Kerberos knowledge, I
wanted to see if anyone was willing to comment prior to heading down this
path.

Thanks!

Charles