special ccache performance issue

[Wang Jian]

Hi list,

When using ansible with kerberos for thousands of targets, there is a
serious ccache performance issue.

Using file ccache (DIR:)
- from a cold ccache, running simple script on servers is fast, at 500-700
hosts/min with 2 or 4 concurrent ansible instance. But things change when
ccache has over 5000 host tickets. The speed drops to 10-30/min and sys CPU
keeps very high.
- High file lock intesion which consumes nearly all CPU

Using kernel keyring ccahe
- fast from start, but eventually, continuous failure, and high sys CPU
- from klist -a, the output is empty now and then, which indicates that
keyring has kneed down under pressure

Using Heimdal KCM
- didn't try. Heimdal KCM uses sequential algorithm and single lock

That is, it's nearly impractical to for thousands hosts, kerberos and
ansible.

I know this is a special case, but perhaps it should be addressed.

--
Regards