Ultimate NAS How-To

Step 1: Installing Software and Performing Basic Configuration

Lets start with Samba and VSFTP and NFS. For Samba, I used the rpms from EnterpriseSamba.org because CentOS AFAIK only ships 3.5. I recommend that you actually download the yum repo file so this way, you can keep Samba and Winbind up to date.
Install winbind3, samba3, krb5-libs, nfs-utils and vsftpd via yum then run the commands below to enable winbind authentication, the automatic creation of home directories and to disable ocfs2, samba, nfs and vsftpd related runlevel scripts since they will be controlled via Pacemaker.

[root@nas1-1 ~]# authconfig  --enablewinbindauth --update
[root@nas1-1 ~]# authconfig  --enablewinbind --update
[root@nas1-1 ~]# authconfig  --enablemkhomedir --update
[root@nas1-1 ~]# for s in o2cb ocfs2 winbind vsftpd samba nfs nfslock; do chkconfig $s off; done

At this point, we need to do some additional configuration since we are using AD authentication. Please do the following:

• Ensure DNS is correct and your AD DC can be resolved.
• Ensure the FQDN of your NAS cluster is specified in /etc/hosts.
• Ensure your local server time is in sync with your AD DC.
• Ensure your domain line matches your AD domain in /etc/resolv.conf.

You’ll also need suitable /etc/krb5.conf, /etc/samba/smb.conf and /etc/vsftpd/vsftpd.conf configuration files. Below are my respective configurations. Feel free to change them to suit your needs:

[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 24h
forwardable = yes

[appdefaults]
pam = {
   debug = false
   ticket_lifetime = 36000
   renew_lifetime = 36000
   forwardable = true
   krb4_convert = false
}

[global]
passdb backend = tdbsam
clustering = yes
idmap backend = tdb2
ctdbd socket = /var/lib/ctdb/ctdb.socket
# Do something sensible when Samba crashes: mail the admin a backtrace
panic action = /usr/share/samba/panic-action %d
aio write size = 1
aio read size = 1
max protocol = SMB2
# Winbind 3.4.x plus implementation
# For Windows 2008 R2 domain (2003 functional level)
winbind separator = "\"
# Yes backslash is a valid character  for separation
# here you just have to use double quotes in the config
workgroup = FOO
password server = 192.168.1.10
realm = FOO.LOCAL
security = ads
winbind nss info = rfc2307
################################################
# Required for Samba/Winbind 3.4+
# Note that local tdb idmap backend
# required now for Samba/Winbind 3.4+
#idmap uid = 9000-9999
#idmap gid = 9000-9999
#################################################
idmap config * : cache time = 1800
idmap config * : backend  = tdb
idmap config * : range = 10000-5000000
idmap config * : schema_mode = rfc2307
winbind use default domain = true
winbind offline logon = false
log file = /var/log/samba/smbd.log
local master = no
domain master = no
preferred master = no
dns proxy = no
client use spnego = yes
winbind enum users = yes
winbind enum groups = yes
client ntlmv2 auth = yes
netbios name = NAS-CLUSTER1
force unknown acl user = Yes
strict locking = Yes
#kernel oplocks = no
server signing = auto
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=262144 SO_RCVBUF=262144 SO_KEEPALIVE
#socket options = IPTOS_LOWDELAY TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 SO_KEEPALIVE
use sendfile = Yes
min receivefile size=16384
use sendfile=true
aio read size = 16384
aio write size = 16384
aio write behind = true
dns proxy=no
use mmap = No
fileid:mapping = fsname
vfs objects = fileid
#write cache size = 262144
template shell = /bin/bash
template homedir = /srv/samba/shares/data/user/shares/homes/%U

[data]
   read only = no
    path = /srv/samba/shares/data/user
   browsable = yes
   guest ok = no
   valid users = @"domain users"

[homes]
    browsable = no
    map archive = yes

anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_file=/var/log/xferlog
xferlog_std_format=YES
idle_session_timeout=600
data_connection_timeout=120
#nopriv_user=ftpsecure
listen=YES
pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
user_sub_token=$USER
local_root=/srv/samba/shares/data/user/shares/homes/$USER
pasv_min_port=30000
pasv_max_port=30099
chroot_local_user=YES
## Must be yes if we want directories to be created upon login.
session_support=Yes

Note: Notice that the value of local_root in vsftpd.conf matches the value of template homedir in smb.conf. If you would rather not make your user directories available via FTP, then feel free to set it to something else. Also note that I’m not using SSL, which would be a big no-no for any production level service. Be sure to enable SSL if you decide to run this in production!

Lastly, edit /etc/sysconfig/nfs to reflect the text below:

NFS_HOSTNAME=
STATD_PORT=595
STATD_OUTGOING_PORT=596
MOUNTD_PORT=597
RQUOTAD_PORT=598
LOCKD_TCPPORT=599
LOCKD_UDPPORT=599
RPCNFSDARGS="-N 4"
NFS_TICKLE_SHARED_DIRECTORY=
STATD_SHARED_DIRECTORY=
STATD_HOSTNAME="$NFS_HOSTNAME -P "$STATD_SHARED_DIRECTORY/$PUBLIC_IP" -H /etc/ctdb/statd-callout -p 97"

Obviously, edit and use values specific for your environment.

All your cluster nodes must run your NFS service using the same configuration. Otherwise during failover, clients may have problems reconnecting.

Next lets install CTDB. This how-to uses the latest code from git.samba.org. Again, as with Samba, CentOS provides an older version (v1.0.114). In this case, you are welcome to try that version if you’d like. Either way, install ctdb then disable it’s initialization script via chkconfig.

Note: If you decide to use the the git repository, all you need to do is clone the repo then run the packaging/RPM/makerpms.sh then install the resulting rpms.

After the installation, disable the runlevel scripts via chkconfig. As with samba, winbind and the other services, Pacemaker will control starting and stopping. Also, create /etc/ctdb/nodes. This file should simply contain a new line separated list of the IPs of all the nodes in your cluster.

Now we need to install csync2 and it’s dependency, sqlite2. Csync2 (you can read about it here) is a synchronization tool. We are going to use it to keep our cluster config files and tickle directory (more on this later) in sync.
It’s not available AFAIK in any of the CentOS repositories so I built an RPM and put it up on Dropbox as a short cut for this how-to. There is one caveat though. I built csync2 without ssl because for some reason, the configure script wasn’t properly detecting gnutls on my system. With that said, below are the links to download csync2 and sqlite2.

• csync2-1.34-1.x86_64.rpm
• sqlite2-2.8.17-8.el6.x86_64.rpm
• sqlite2-devel-2.8.17-8.el6.x86_64.rpm

It requires that you have rsync installed, so install rsync via yum and then install the RPMS from above. After installation, run the command csync2 -k /etc/csync2.key to generate the key needed to sync between nodes. Eventually, that key will need to go to all your cluster nodes.
Csync2 is both a client and server can run stand-alone daemon or through xinetd. I choose to go the route of xinetd. Edit the /etc/xinetd.d/csync and change disable = yes to disable = no then reload xinetd. If it’s not installed, install it and make sure it starts up at boot. That’s it for csync2 for now. We’ll need to come back to it later though.

So to recap, we’ve installed Samba, VSFTP, CTDB, NFS and csync2. The NFS, Samba, CTDB and VSFTP configurations are complete. We will need to configure csync2 and we will get to that in shortly. Lets move on to configuring our cluster stack.