issue with k5start

[Kristen Webb]

Thanks for the feedback, Russ
On Tue, Sep 25, 2018 at 3:11 PM Russ Allbery <eagle at eyrie.org> wrote:

> Kristen Webb <kwebb at teradactyl.com> writes:
>
> > When I use the -k ccache option it appears that each job simply
> > overwrites the cchache file.
>
> It should only do this if the ticket is going to expire sooner than two
> minutes before the next wake-up period, though, I think?  I would have
> expected this to work with all jobs sharing the same cache file, as long
> as they're at least a little staggered.  That said, I don't think I've
> really tested for this sort of parallelism, and it's entirely possible
> that the separate k5start processes don't manage coordination between each
> other on the same ticket cache properly.
>
The only control I can find for the CC is the -a flag, which always renews
tickets when k5start wakes up.
It looks like k5start always inits the CC on startup.

>
> > Is there a way to use k5start to achieve what I am after
> >      - shared ccache for many jobs to keep kerberos server traffic down
> >      - allow long running jobs to continue beyond their initial aklog
> > renewal date
>
> > If I ran k5start as a daemon and managed periodic aklog's within my
> > application, would that work?
>
> Yes, that's what I was going to suggest.  If each application is running
> in a separate PAG, each application needs to run aklog periodically
> independently of the others.  If you also want to share a single ticket
> cache among the applications, you probably want to split those two
> operations.

This works for most cases.  However, when a new job starts near the
time a TGT is ready to be renewed, it's afs token lifetime can be quite
short and can expire before a planned re-aklog within that PAG.

It seems that the only way around this is for each job to know when the
shared TGT is really
going to expire so it can plan/init accordingly at least the first aklog.
I cannot seem to find
an easy command line function for determining the remaining lifetime.  The
closest I have found is:

    kinit /ccache -s

But by the time it tells you that the CC is expired it is already too late!

In my current solution I am testing:
1. A cron job with kinit to check and the remaining lifetime on the TGT and
renew if  needed.
(eventually replace with a k5start daemon)
2. Use the same code on each new vos job's watchdog to check the lifetime
and kinit if necessary.
This helps to ensure that the jobs sleep time for aklog will not cause the
afs token to expire.
3. The watchdog process for the job then only runs aklog going forward.

>


> Unfortunately, k5start doesn't currently have a mode of operation in which
> it only runs the aklog command but doesn't try to renew tickets if they
> aren't about to expire.
>
This is similar to what I am trying to do manually.  It would appear that
k5start might be capable of doing this, and I would be interested in trying
to
help implement this feature.

>
> --
> Russ Allbery (eagle at eyrie.org)              <http://www.eyrie.org/~eagle/>
>


-- 
This message is NOT encrypted
--------------------------------
Mr. Kristen J. Webb
Chief Technology Officer
Teradactyl LLC.
2450 Baylor Dr. S.E.
Albuquerque, New Mexico 87106
Phone: 1-505-338-6000
Email: kwebb at teradactyl.com
Web: http://www.teradactyl.com



Providers of Scalable Backup Solutions
   for Unique Data Environments

--------------------------------
NOTICE TO RECIPIENTS: Any information contained in or attached to this
message is intended solely for the use of the intended recipient(s). If
you are not the intended recipient of this transmittal, you are hereby
notified that you received this transmittal in error, and we request
that you please delete and destroy all copies and attachments in your
possession, notify the sender that you have received this communication
in error, and note that any review or dissemination of, or the taking of
any action in reliance on, this communication is expressly prohibited.


Regular internet e-mail transmission cannot be guaranteed to be secure
or error-free. Therefore, we do not represent that this information is
complete or accurate, and it should not be relied upon as such. If you
prefer to communicate with Teradactyl LLC. using secure (i.e., encrypted
and/or digitally signed) e-mail transmission, please notify the sender.
Otherwise, you will be deemed to have consented to communicate with
Teradactyl via regular internet e-mail transmission. Please note that
Teradactyl reserves the right to intercept, monitor, and retain all
e-mail messages (including secure e-mail messages) sent to or from its
systems as permitted by applicable law