Winbind in SmartOS, Part III (Polishing)

You now have a base-32 or base-64 zone running Winbind, what do you need to take it the rest of the way? We will cover a number of topics in this post, pick the ones that are relevant to your environment:

A github repository has been made available for this series, including all relevant configuration files and a setup script that summarizes commands issued (aside from a few points in this post). The script is not intended for production use, but is good for a simple test and for those that read code more readily than blog posts.

Creating the Winbind Service

We have manually started Winbind with the “winbindd” command up to this point, let’s remedy that by creating an SMF service for the Winbind process. First, create a manifest file for the service (we can save it as ~/winbind.xml):

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
  <service name='network/winbind' type='service' version='0'>
    <create_default_instance enabled='true'/>
    <dependency name='fs' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/system/filesystem/local'/>
    <dependency name='network' grouping='optional_all' restart_on='none' type='service'>
      <service_fmri value='svc:/milestone/network'/>
    <exec_method name='refresh' type='method' exec=':kill -HUP' timeout_seconds='60'/>
    <exec_method name='start' type='method' exec='/opt/local/sbin/winbindd' timeout_seconds='60'/>
    <exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'/>
        <loctext xml:lang='C'>Winbind</loctext>

Stop our current winbind process, then import the manifest using svccfg:

# pkill winbind
# svccfg import ~/winbind.xml

We now have Winbind running as an SMF service:

# svcs winbind
STATE          STIME    FMRI
online         19:04:24 svc:/network/winbind:default

Reducing the Authentication Delay

If you are testing Winbind on a larger domain, you may notice that the getent commands and login take a decent amount of patience. We had the Winbind user and group enumeration options enabled for testing purposes, but they are not strictly necessary for the majority of operations (we haven’t yet encountered a case where they were needed). Remove these lines from /opt/local/etc/samba/smb.conf:

        winbind enum groups = yes
        winbind enum users = yes

Restart the winbind service that we just created to apply the new configuration:

# svcadm refresh winbind

Note that you can still use getent to query for particular users or groups by specifying the user or group name in the command, even if the general getent commands will no longer display all domain users or groups.

Creating Home Directories

If you look back at the end result of the previous posts, you will notice this error on login:

Could not chdir to home directory /home/[user]: No such file or directory

We have the home directory configured, but have nothing set up to create individual user directories.


Linux has a PAM module called “pam_mkhomedir”, and it seems like a straight-forward solution but is not provided or supported in SmartOS. Compiling the module in Solaris has been done, but we don’t want to support that kind of work through our configuration management solution (puppet) due to the amount of hack required.

Winbind PAM has a “mkhomedir” option, but it has a fatal flaw. Here is what login looks like with the option set:

$ ssh [user]@[host]
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ; Instance (base-64-lts 15.4.0)

-bash-4.1$ ls -al
total 2
drwx------   2 [user] [group]       2 Feb 26 19:19 .
drwxr-xr-x   4 root     root           4 Feb 26 19:19 ..
-bash-4.1$ echo $PATH

The home directory is created, but it is completely empty and we are left with bash defaults for everything. The PATH variable is especially unhelpful, as we will be missing out on around 400 commands with the default value (including sudo).

The autofs service was originally intended for auto-mounting of network shares, but it can be tweaked for the purposes of auto-generation of user home directories. Using a modified version of Znogger’s auto_homedir script, we will create a home directory for each user (as needed) based on a copy of the /etc/skel directory (very important, it includes good shell defaults including a good PATH value). Save this script to /etc/auto_home (overwrite the file that is there):



# The folder being requested is first argument, we assume at
# this point that it is a user
group=$(id -gn "$user")
home_dir=$(getent passwd "$user" | cut -d: -f6)

# Only create a directory if it doesn't exist
if [ ! -d "$physical_dir" ]; then
  # Sanity check, ensure that this is actually a user and that
  # his home directory is in the expected location
  if [[ "$home_dir" != $HOMEDIRPATH/* ]]; then

  # Use the shell defaults in /etc/skel to populate the initial
  # home directory
  cp -r /etc/skel "$physical_dir"
  chown -R "$user":"$group" "$physical_dir"

echo "localhost:$physical_dir"

Make the script executable:

# chmod +x /etc/auto_home

Enable the bind and autofs services (autofs needs bind to be running):

# svcadm enable bind
# svcadm enable autofs

Now try logging in:

$ ssh [user]@[host]
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ; Instance (base-64-lts 15.4.0)

[user@host ~]$ ll -a
total 12
drwxr-xr-x 3 26467 27831  11 Feb 26 19:47 ./
dr-xr-xr-x 2 root  root    2 Feb 26 19:45 ../
-rw-r--r-- 1 26467 27831  76 Feb 26 19:47 .bash_profile
-rw-r--r-- 1 26467 27831 304 Feb 26 19:47 .bashrc
-rw-r--r-- 1 26467 27831 151 Feb 26 19:47 .cshrc
-rw-r--r-- 1 26467 27831  38 Feb 26 19:47 .curlrc
-rw-r--r-- 1 26467 27831 240 Feb 26 19:47 .irbrc
-rw-r--r-- 1 26467 27831  40 Feb 26 19:47 .login
-rw-r--r-- 1 26467 27831 690 Feb 26 19:47 .profile
drwxr-xr-x 2 26467 27831   3 Feb 26 19:47 .ssh/
-rw-r--r-- 1 26467 27831 846 Feb 26 19:47 .vimrc
[user@host ~]$ echo $PATH
[user@host ~]$ 

Much better!

How did that work?

You can see Znogger’s post for a more detailed explanation of the autofs process. We don’t need step 1 (modifying the /etc/auto_master file) in his instructions, since /etc/auto_home is configured by default in SmartOS to be called for /home. If you wanted to have several auto-mounted sources for /home, or use a different home directory, you will need to store the script in a different location and modify /etc/auto_master or the existing /etc/auto_home file instead.

Limiting Access

For the needs of the Faithlife environment, we want to limit SSH access based on Active Directory group membership. Setting that up in Winbind is straightforward, but let’s lay out some groundwork by moving pam_winbind settings to a separate settings file before it gets too long. Remove “use_first_pass” from pam.conf:

# sed -i 's/ use_first_pass//g' /etc/pam.conf

Then create our initial pam_winbind configuration at /etc/security/pam_winbind.conf with this as the content:


try_first_pass = yes

We can now add a membership limitation in that file (get the group name or guid from getent groups list, and comma-separate if permitting multiple groups):

# echo 'require_membership_of = [group]' >> /etc/security/pam_winbind.conf

Granting Sudo Access

Winbind access does not allow us to use the more granular features provided by the Solaris RBAC system, so we will need to go to the old admin standby: sudo. Nothing out of the ordinary for Winbind, we just need to add any groups or users into the sudoers file so that they can take administrative actions.

We’ll just append to the sudoers file here, but using the visudo command is recommended when manually editing sudoers. For a group (escape spaces and backslashes with backslashes):

# echo '%[group] ALL=(ALL) ALL' >> /opt/local/etc/sudoers

For a user (also escaped if necessary):

# echo '[user] ALL=(ALL) ALL' >> /opt/local/etc/sudoers

With that change, we now have sudo access:

[user@host ~]$ sudo su -
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ; Instance (base-32-lts 15.4.0)

[root@host ~]# 

Working with More than 16 Group Memberships

If your Active Directory domain users tend to have a significant number of security-group memberships, you may have already encountered this issue when granting sudo access: SmartOS is based on a version of Solaris that (by default) has a group membership limit of 16, and any AD security group memberships beyond 16 are truncated. The limitation does not affect basic SSH access, since the Winbind PAM doesn’t consult NSS for the membership list, but sudo will be blocked if the relevant group was not included in the 16.

The truncation isn’t realistically controllable, and although increasing the group limit is possible it requires grub-level changes (and a reboot) to make that change. That change becomes even more difficult in an SDC or Triton environment, and impossible in the JPC environment.

Our workaround hinges on one predictable aspect of the truncation: the primary user group is never cut. We ensure that the primary AD group for each AD user is the one with the most significant for SSH access.

Eliminating the SSH Key Loophole for Disabled Accounts

Using RSA keys to access SmartOS zones based on domain accounts is possible and a great boost to productivity while working with batches of servers. Unfortunately, use of RSA keys reveals a surprising limitation between PAM and SSH: if an RSA key is used for access, some of the PAM sections are not consulted. As a result, a disabled account will not be denied access by Winbind if an RSA key is utilized that was previously added to ~/.ssh/authorized_keys for that user.

Our workaround for that security hole is fairly simple: we utilized an Active Directory group for “Disabled Users” and block members of that group through the SSH service configuration. Do that by setting the DenyGroups value in sshd_config:

# echo 'DenyGroups "disabled users"' >> /etc/ssh/sshd_config
# svcadm refresh ssh

Note that the 16-group-membership limit applies here, so if you leave group memberships applied to the disabled account you should set the “Disabled Users” group as the primary security group.

Updating Passwords

Winbind PAM does support password changes via passwd and during login, but under SmartOS we don’t have that luxury:

[user@host ~]$ passwd
passwd: Changing password for [user]
passwd: Unsupported nsswitch entry for "passwd:". Use "-r repository ".
Unexpected failure. Password file/table unchanged.

Solaris, and by lineage SmartOS, only supports a few configurations for password changes (a hard-coded limitation):

Only five passwd configurations are permitted:

  • passwd: files
  • passwd: files nis
  • passwd: files nisplus
  • passwd: compat
  • passwd: compat passwd_compat: nisplus

There is no real fix for this limitation, but kpasswd (the Kerberos equivalent to passwd command) can be used to change the domain password for a user. The kpasswd command does not work for password changes on login.

Working with Long Hostnames

Since we are joining an Active Directory domain, a familiar limitation comes into play for hosts with longer hostnames:

# net join -k -U [user]
Our netbios name can be at most 15 chars long, "FAIRLY-LONG-HOSTNAME" is 20 chars long
Invalid configuration.  Exiting....
Failed to join domain: The format of the specified computer name is invalid.
ADS join did not work, falling back to RPC...
Our netbios name can be at most 15 chars long, "FAIRLY-LONG-HOSTNAME" is 20 chars long

We can’t go changing Microsoft now, so we need to change the registration name of the host we are trying to join to the network. We can do that via the “netbios name” setting in /opt/local/etc/smb.conf:

# echo '        netbios name = shorter-name' >> /opt/local/etc/samba/smb.conf
# net join -k -U [user]
Enter [user]'s password:
Using short domain name -- LRSCORP
Joined 'SHORTER-NAME' to dns domain ''

The kind of truncation that you can do on the hostname depends on your environment, in ours we removed dashes to get all our zone hostnames under the limit.

Allow Normal Users to Query Users and Groups

After testing things out as a normal user, you may notice that the user can’t see his own user or group names:

$ whoami
whoami: cannot find name for user ID 23456
$ id
uid=23456 gid=65432 groups=65432,76543

Truss is once again a key to figuring out why things aren’t working. Let’s use our truss-and-grep method from part 1:

$ truss whoami 2>&1 >/dev/null | grep winbind
stat("/lib/64/", 0xFFFFFD7FFFDFF090) Err#2 ENOENT
stat("/usr/lib/64/", 0xFFFFFD7FFFDFF090) Err#2 ENOENT
stat("/opt/local/lib/", 0xFFFFFD7FFFDFF090) = 0
resolvepath("/opt/local/lib/", "/opt/local/lib/", 1023) = 32
open("/opt/local/lib/", O_RDONLY) = 4
stat("/opt/local/lib//", 0xFFFFFD7FFFDFEF60) Err#2 ENOENT
stat("/opt/local/gcc49/x86_64-sun-solaris2.11/lib/amd64/", 0xFFFFFD7FFFDFEF60) Err#2 ENOENT
stat("/opt/local/gcc49/lib/amd64/", 0xFFFFFD7FFFDFEF60) Err#2 ENOENT
stat("/opt/local/lib/samba/private/", 0xFFFFFD7FFFDFEF60) Err#13 EACCES [file_dac_search]

The EACCES error seems fairly self-explanatory, let’s check the permissions on that library:

$ ll -d /opt/local/lib/samba/private/
ls: cannot access /opt/local/lib/samba/private/ Permission denied
$ ll -d /opt/local/lib/samba/private
drwx------ 3 root root 104 Mar 10 20:47 /opt/local/lib/samba/private/

Allow regular users to read and execute the “private” winbind libraries (from the root user):

# chmod 755 /opt/local/lib/samba/private
# chmod 755 /opt/i386/opt/local/lib/samba/private

User info is now available:

$ whoami
$ id
uid=23456([user]) gid=65432([primary-group]) groups=65432([primary-group]),76543([other-group])