Thanks for the reply. I checked the key exchange issue, and it didn't seem to be that. I tried confirming that su worked to prove it was an OpneSSH issue, but su failed too. I uninstalled Centrify completely, added the "PasswordAuthentication yes" to /etc/ssh/sshd_config and confirmed that /usr/sbin/sshd works without problems for local users. I reinstalled Centrify Express and am getting a similar error using /usr/sbin/sshd.
Here's the auth.log from my first connection attempt using AD credentials (present in getent passwd) initiated from a fresh Putty session using Putty defaults. I was prompted for a password and the session terminated on me typing in the domain password:
Jan 16 09:59:17 osm sshd[573]: Accepted keyboard-interactive/pam for i87000 from 192.168.0.10 port 55006 ssh2 Jan 16 09:59:17 osm adclient[364]: INFO <fd:10 PAMCreateKrb5Creds > daemon.ipcclient2 Problem storing credentials into credentials cache file for user 'i87000': Problem setting the ownership of FILE:/tmp/krb5cc_851444974: error = -1765328188, error message = krb5_cc_chown: Internal credentials cache error Jan 16 09:59:17 osm adclient[364]: WARN <fd:22 sshd(573)> Set credentials for user 'i87000': Problem storing credentials into credentials cache Jan 16 09:59:17 osm adclient[364]: INFO AUDIT_TRAIL|Centrify Suite|PAM|1.0|201|PAM set credentials denied|5|user=i87000(type:ad,i87810@MYDOMAIN.COM) pid=573 utc=1484560757106 centrifyEventID=24201 status=DENIED service=sshd tty=ssh client=192.168.0.10 reason=Failed to set user credentials Jan 16 09:59:17 osm sshd[573]: fatal: PAM: pam_setcred(): Authentication failure Jan 16 09:59:32 osm adclient[364]: INFO AUDIT_TRAIL|Centrify Suite|Trusted Path|1.0|2700|Trusted path granted|5|user=osm$@MYDOMAIN.COM pid=364 utc=1484560772490 centrifyEventID=23700 status=GRANTED server=ldap/pdc.mydomain.com@MYDOMAIN.COM
Further connection attempts fail on providing the username and don't prompt for a password.
I can successfully SSH in as a local user. When I try to su from that local user to a domain user I get the following from auth.log:
Jan 16 10:26:19 osm adclient[364]: INFO AUDIT_TRAIL|Centrify Suite|PAM|1.0|100|PAM authentication granted|5|user=i87000(type:ad,i87000@MYDOMAIN.COM) pid=709 utc=1484562379611 centrifyEventID=24100 status=GRANTED service=su tty=/dev/pts/0 client=(none) Jan 16 10:26:19 osm su[709]: Successful su for i87000 by localuser Jan 16 10:26:19 osm adclient[364]: INFO AUDIT_TRAIL|Centrify Suite|PAM|1.0|300|PAM account management granted|5|user=i87000(type:ad,i87000@MYDOMAIN.COM) pid=709 utc=1484562379614 centrifyEventID=24300 status=GRANTED service=su tty=/dev/pts/0 client=(none) Jan 16 10:26:19 osm su[709]: + /dev/pts/0 localuser:i87000 Jan 16 10:26:19 osm su[709]: bad group ID `851444974' for user `i87000': Invalid argument
I've tried running sshd in debug on a fresh port with '/usr/sbin/sshd -ddde -p 2222' then running SSH from another Ubuntu server using the domain ID. I've truncated the sshd debug output (it's error free above this point):
debug1: PAM: initializing for "i87000" debug1: PAM: setting PAM_RHOST to "192.168.0.11" debug1: PAM: setting PAM_TTY to "ssh" debug2: monitor_read: 100 used once, disabling now debug3: receive packet: type 50 [preauth] debug1: userauth-request for user i87810 service ssh-connection method publickey [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method publickey [preauth] debug1: userauth_pubkey: test whether pkalg/pkblob are acceptable for RSA SHA256:l4Ntxwcz+iWnxeaY1XjVvyfo18E/Xev4VUgaj5a/uWg [preauth] debug3: mm_key_allowed entering [preauth] debug3: mm_request_send entering: type 22 [preauth] debug3: mm_key_allowed: waiting for MONITOR_ANS_KEYALLOWED [preauth] debug3: mm_request_receive_expect entering: type 23 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 4 debug3: mm_answer_authserv: service=ssh-connection, style=, role= debug2: monitor_read: 4 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 22 debug3: mm_answer_keyallowed entering debug3: mm_answer_keyallowed: key_from_blob: 0x55bcb38df3c0 debug1: temporarily_use_uid: 851444974/851444974 (e=0/0) initgroups: i87000: Invalid argument debug1: do_cleanup debug1: PAM: cleanup debug3: PAM: sshpam_thread_cleanup entering debug1: Killing privsep child 712 debug1: audit_event: unhandled event 12
Thank you for the link to the troubleshooting tips for OpenSSH - it was very helpful and informative, but I'm not sure this is an OpenSSH issue. Neither the problem adclient has in storing credentials or the su error with invalid groups relate to OpenSSH. I'm not sure which of these is the underlying issue - is adclient failing to store the cache credentials because it's trying to use an invalid group, or is there an invalid group because adclient is unable to cache credentials?