[krbdev.mit.edu #9249] git commit

[Greg Hudson via RT]

<URL: https://krbdev.mit.edu/rt/Ticket/Display.html?id=9249 >


Improve libdb2 data validation

In the btree module, make __bt_pgin() bounds-check h->lower and reset
the page to empty if it is not within the allowable range.  Change the
definition of NEXTINDEX to clamp the result lower bound to 0, so that
it can't return an enormous offset due to unsigned underflow.  In
__bt_search() and __bt_seqset(), detect when we fill the parent page
stack (because the database is circular or the tree is unexpectedly
imbalanced) and return an error.

In the hash module, make __kdb2_hash_open() reject a block size of
zero to prevent division by zero, and reject an invalid overflow
pointer to prevent an out-of-bounds memory access.  Prevent unbounded
recursion in collect_key() and collect_data() by adding a depth
counter and limiting it to max_bucket.  Prevent out-of-bounds access
in __pgin_routine() by bounds-checking num_ent for non-bitmap pages.

[ghudson at mit.edu: combined several commits and rewrote commit message;
simplified __bt_pgin() bounds check and made it leave h->lower == 0
alone for nroot(); added checks for full parent page stack; made minor
edits for code style]

(cherry picked from commit 64470b21a544f1619673373c74efd96c99778028)

https://github.com/krb5/krb5/commit/de22049d7732621571f3c4e53ccb01346fd4c11d
Author: Bogdan Boguslavskij <bogdanb at altlinux.org>
Committer: Greg Hudson <ghudson at mit.edu>
Commit: de22049d7732621571f3c4e53ccb01346fd4c11d
Branch: krb5-1.22
 src/plugins/kdb/db2/libdb2/btree/bt_conv.c    | 10 +++++++++-
 src/plugins/kdb/db2/libdb2/btree/bt_search.c  |  2 ++
 src/plugins/kdb/db2/libdb2/btree/bt_seq.c     |  8 ++++++++
 src/plugins/kdb/db2/libdb2/btree/btree.h      |  8 ++++++--
 src/plugins/kdb/db2/libdb2/hash/hash.c        |  8 +++++++-
 src/plugins/kdb/db2/libdb2/hash/hash_bigkey.c | 23 ++++++++++++++---------
 src/plugins/kdb/db2/libdb2/hash/hash_page.c   | 13 +++++++++++++
 7 files changed, 59 insertions(+), 13 deletions(-)