krb5-1.12 - New iprop features - tree propagation & slave notify

[Richard Basch]

Now that I have stress tested this feature, I have discovered it is possible
under high load that slave notification events may be triggered but either
are lost or due to resource constraints cannot be processed (perhaps the
kprop is failing), and that it must fall back to the polling.

 

Overall, as long as the rate of change is not too great, I have not observed
latency issues, nor do I see any obvious bugs, except that the notification
is not synchronous and any failure there may result in the notification not
being processed and the client not being queued to be notified for another
update.

 

In other words, what I provided really is an immense improvement over the
prior replication (latency is generally reduced), but there is still an edge
case under load where it might fall back to prior behavior. I might change
my implementation slightly so that instead of dropping the client after the
first notification to implement a countdown for number of notifications
before the client is dropped from the list. This will likely reduce the
likelihood of the bug. I do not think slave notification should be
synchronous.

 

 

From: Richard Basch [mailto:basch at alum.mit.edu] 
Sent: Thursday, December 12, 2013 12:01 AM
To: 'krbdev at mit.edu'
Cc: 'ghudson at mit.edu'; 'tlyu at mit.edu'; 'Nico Williams';
'richard.basch at gs.com'; 'Richard Basch'
Subject: RE: krb5-1.12 - New iprop features - tree propagation & slave
notify

 

Resent message (reformatted). Outlook previously munged the text wrapping &
URLs.


I cleaned up the commits (especially since I found an error in one of my man
page updates).

 

Wiki:

 <https://github.com/rbasch/krb5/wiki/Replication-enhancements>
https://github.com/rbasch/krb5/wiki/Replication-enhancements

 

Within the wiki, it lists the commits:

 
<https://github.com/rbasch/krb5/commit/0f4b84669c345cb81e60b26372fa32bb9834c
093>
https://github.com/rbasch/krb5/commit/0f4b84669c345cb81e60b26372fa32bb9834c0
93 (ulog tracking on slaves)
<https://github.com/rbasch/krb5/commit/023602cc7934c96ab7130aa19481228958473
749>
https://github.com/rbasch/krb5/commit/023602cc7934c96ab7130aa194812289584737
49 (tree propagation feature)

 
<https://github.com/rbasch/krb5/commit/de078a1f49a8d40ba5a5cb7fdbe22c593f305
09a>
https://github.com/rbasch/krb5/commit/de078a1f49a8d40ba5a5cb7fdbe22c593f3050
9a (slave notification)

 

A summary view is available via:

 <https://github.com/rbasch/krb5/compare/krb5-1.12>
https://github.com/rbasch/krb5/compare/krb5-1.12

 

For anyone just joining in the discussion, I enhanced the replication
strategy so a tree-based hierarchy can be setup (useful for improving
scale-out and for sites where multiple servers may be across slow WAN
links). In addition, when updates are registered in the database, notify
events are sent to iprop slave servers so they know there are pending
updates (which reduces replication lag). The exact methodology is described
in the wiki.

 

The patches have already been tested with the final 1.12 production release
(they were previously tested against the 1.12 beta releases).