[krbdev.mit.edu #6732] checks for openpty() aren't made using -lutil

[The RT System itself via RT]

>From krb5-bugs-incoming-bounces at PCH.mit.edu  Fri May 21 16:21:03 2010
Return-Path: <krb5-bugs-incoming-bounces at PCH.mit.edu>
Received: from pch.mit.edu (PCH.MIT.EDU [18.7.21.90])
	by krbdev.mit.edu (Postfix) with ESMTP id A24D73E715;
	Fri, 21 May 2010 16:21:03 -0400 (EDT)
Received: from pch.mit.edu (pch.mit.edu [127.0.0.1])
	by pch.mit.edu (8.13.6/8.12.8) with ESMTP id o4LKL3IX008316;
	Fri, 21 May 2010 16:21:03 -0400
Received: from mailhub-dmz-3.mit.edu (MAILHUB-DMZ-3.MIT.EDU [18.9.21.42])
	by pch.mit.edu (8.13.6/8.12.8) with ESMTP id o4LKHaPq007578
	for <krb5-bugs-incoming at PCH.mit.edu>; Fri, 21 May 2010 16:17:36 -0400
Received: from dmz-mailsec-scanner-6.mit.edu (DMZ-MAILSEC-SCANNER-6.MIT.EDU
	[18.7.68.35])
	by mailhub-dmz-3.mit.edu (8.13.8/8.9.2) with ESMTP id o4LKHPOK013809
	for <krb5-bugs at mit.edu>; Fri, 21 May 2010 16:17:36 -0400
X-AuditID: 12074423-b7c0bae0000030f0-d4-4bf6ea5f8391
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28])
	by dmz-mailsec-scanner-6.mit.edu (Symantec Brightmail Gateway) with
	SMTP id 64.93.12528.F5AE6FB4; Fri, 21 May 2010 16:17:36 -0400 (EDT)
Received: from int-mx01.intmail.prod.int.phx2.redhat.com
	(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o4LKHY1g004828
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
	for <krb5-bugs at mit.edu>; Fri, 21 May 2010 16:17:34 -0400
Received: from blade.bos.redhat.com (blade.bos.redhat.com [10.16.0.23])
	by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
	id o4LKHXHk004166
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <krb5-bugs at mit.edu>; Fri, 21 May 2010 16:17:34 -0400
Received: from blade.bos.redhat.com (blade.bos.redhat.com [127.0.0.1])
	by blade.bos.redhat.com (8.14.4/8.14.3) with ESMTP id o4LKHXCq015135
	for <krb5-bugs at mit.edu>; Fri, 21 May 2010 16:17:33 -0400
Received: (from nalin at localhost)
	by blade.bos.redhat.com (8.14.4/8.14.4/Submit) id o4LKHXiB015133;
	Fri, 21 May 2010 16:17:33 -0400
Date: Fri, 21 May 2010 16:17:33 -0400
Message-Id: <201005212017.o4LKHXiB015133 at blade.bos.redhat.com>
To: krb5-bugs at mit.edu
Subject: checks for openpty() aren't made using -lutil
From: nalin at redhat.com
X-send-pr-version: 3.99
X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11
X-Brightmail-Tracker: AAAAAgAFNSIURuK5
X-Mailman-Approved-At: Fri, 21 May 2010 16:21:01 -0400
X-BeenThere: krb5-bugs-incoming at mailman.mit.edu
X-Mailman-Version: 2.1.6
Precedence: list
Sender: krb5-bugs-incoming-bounces at PCH.mit.edu
Errors-To: krb5-bugs-incoming-bounces at PCH.mit.edu


>Submitter-Id:	net
>Originator:	Nalin Dahyabhai
>Organization:
>Confidential:	no
>Synopsis:	checks for openpty() aren't made using -lutil
>Severity:	non-critical
>Priority:	low
>Category:	krb5-appl
>Class:		sw-bug
>Release:	1.0
>Environment:
	
System: Linux blade.bos.redhat.com 2.6.33.3-85.fc13.x86_64 #1 SMP Thu May 6 18:09:49 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux
Architecture: x86_64

>Description:
	The krb5-appl-1.0 configure script checks for libutil in case it
	is where needed functions such openpty() are defined, but when it
	subsequently checks for the availability of those functions, it
	doesn't find them if they're in libutil.
>How-To-Repeat:
	
>Fix:
	It's not pretty, but this causes openpty() to be found on my system
	when it otherwise wouldn't be:
--- krb5-appl-1.0/configure.ac
+++ krb5-appl-1.0/configure.ac
@@ -107,6 +107,8 @@ AC_SYS_LARGEFILE
 AC_FUNC_FSEEKO
 AC_TYPE_LONG_LONG_INT
 AC_TYPE_UNSIGNED_LONG_LONG_INT
+old_LIBS="$LIBS"
+LIBS="$UTIL_LIB $LIBS"
 AC_CHECK_FUNCS(_getpty cgetent getcwd getenv gettosbyname getusershell getutmp)
 AC_CHECK_FUNCS(getutmpx grantpt inet_aton initgroups isatty killpg killpg)
 AC_CHECK_FUNCS(line_push logwtmp openpty ptsname revoke rmufile rresvport_af)
@@ -114,6 +116,7 @@ AC_CHECK_FUNCS(seteuid setlogin setpgid 
 AC_CHECK_FUNCS(setutent setutsent setutxent strsave tcgetpgrp tcsetpgrp)
 AC_CHECK_FUNCS(ttyname unsetenv updwtmp updwtmpx utimes utmpname utmpxname)
 AC_CHECK_FUNCS(vhangup vsnprintf waitpid)
+LIBS="$old_LIBS"
 
 # Determine which functions to define in libmissing.
 AC_REPLACE_FUNCS(daemon getdtablesize getopt herror parsetos setenv setsid)