glusterfs.git/extras/ganesha/scripts, branch v3.9dev

common-ha: continuous grace_mon log messages in /var/log/messages

2016-04-28T10:19:50+00:00

messages are seen on RHEL6.x and RHEL7.1 and earlier versions of
pacemaker. (And RHEL7.2 with RHEL7.1 pacemaker packages.)

It's not possible to query attrd attributes in the older version,
only set/update/clear them. The messages come from invalid attempts
to query the attributes.

However it is possible to query crm attributes. The fix here is to
create a "shadow" crm attribute for the attrd attribute. Changes are
made to both, queries are made on the crm attribute.

(Resource Agents "follow" the attrd attribute using constraint locations,
so we must keep the attrd attribute.)

Change-Id: I84ac1a80673e528d98b67b7d5062e21dcf744d4a
BUG: 1324509
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13919
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: soumya k

ganesha: Include a script to generate epoch value

2016-03-21T08:15:04+00:00

In a NFS-Ganesha HA cluster setup, for NFS clients to recover state
succesfully post failover, the NFS-servers should start with a unique
epoch value.

With NFS-Ganesha 2.3, the service accepts an option "EPOCH_EXEC" which
takes path of the script, generating epoch value. This script is executed before
starting nfs-ganesha service so that the generated epoch value is used
while bringing up the service.

This patch includes the script to be used by nfs-ganesha+gluster setup.

The epoch value is computed as follows -
       - first 32-bit contains the now() time
       - rest 32-bit value contains the local glusterd node uuid

Change-Id: I876ea5a3730d7c6b40503e0fec16a4a142c54a36
BUG: 1317902
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/13744
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY

ganesha-ha: merge release-3.7 to master for release-3.8

2016-03-15T07:26:01+00:00

cherry-pick changes from release-3.7 branch to master in preparation
for 3.8

changes merged/cherry-picked from release-3.7 consist of:

> common-ha: delete virt-IP entry of deleted node
> commit aee983ff43655771a9102592a284d0b0a29ad89f
> BUG: 1250601
>
> common-ha: concise output for HA status
> commit d804b17f2fe92b1516f85f03978072c42ddc6f19
> BUG: 1250628
>
> common-ha : refresh-config should print sensible output
> commit d7adcca24fb9638df2806c01d8ea7e73eec46928
> BUG: 1254494
>
> CommonHA: Avoid scp of the config state to the same host
> commit 07b31a008b59d6c0b06bd17994de85fc56560b38
> BUG: 1259225
>
> CommonHA: Fix the path of 'systemctl' cmd
> commit 8e7ca068720fa6fa50e4746e5b17c305f38171e8
> BUG: 1259225
>
> common-ha: refresh-config output includes dbus "method return" msg
> commit 045cb34238e341d68288893b8f040056e138c04e
> BUG: 1262881
>
> common-ha: distribution neutral location of config files
> commit 6667478cdba920d2658bba2edc99c8f8cc33e271
> BUG: 1251821
>
> common-ha: Corrected refresh-config output parsing
> commit 2892bab69cac2d991509fca1279a659bce1ae793
> BUG: 1254494
>
> common-ha: reliable grace using pacemaker notify actions
> commit e8121c4afb3680f532b450872b5a3ffcb3766a97
> BUG: 1290865
>
> ganesha: Read export_id on each node while performing refresh-config
> commit e0e633cdce7586af92490730257ed7f0cffcff61
> BUG: 1309238
>
> nfs-ganesha: pcs cluster setup needs '--name' option
> commit 812e7321d3c56d329526628fe96a8f6fabea75ca
> BUG: 1314204

Change-Id: I41c19bd369bd9da1092b0677590634a4e9f72813
BUG: 1317424
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13728
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

nfs-ganesha: pcs cluster setup needs '--name' option

2016-03-15T07:25:38+00:00

Even on fedora machines (on f22, f23), pcs cluster setup CLI
expects '--name' to be provided. This patch addresses the same.

BUG: 1314204
Change-Id: If23851bfd7fe2f1c200731d66dcb3e744390ff83
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/13590
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
NetBSD-regression: NetBSD Build System 
Reviewed-on: http://review.gluster.org/13727
Tested-by: Kaleb KEITHLEY

ganesha: Read export_id on each node while performing refresh-config

2016-03-15T07:25:19+00:00

As mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1309238#c1,
there could be cases which shall result in having different ExportIDs
for the same volume on each node forming the ganesha cluster.

Hence during refresh-config, it is necessary to read the ExportID on
each of those nodes and re-export that volume with the same ID.

BUG: 1309238
Change-Id: Id39b3a0ce2614ee611282ff2bee04cede1fc129d
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/13459
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.org/13726
Tested-by: Kaleb KEITHLEY

common-ha: reliable grace using pacemaker notify actions

2016-03-15T04:34:27+00:00

Using *-dead_ip-1 resources to track on which nodes the ganesha.nfsd
had died was found to be unreliable.

Running `pcs status` in the ganesha_grace monitor action was seen to
time out during failover; the HA devs opined that it was, generally,
not a good idea to run `pcs status` in a monitor action in any event.
They suggested using the notify feature, where the resources on all
the nodes are notified when a clone resource agent dies.

This change adds a notify action to the ganesha_grace RA. The ganesha_mon
RA monitors its ganesha.nfsd daemon. While the daemon is running, it
creates two attributes: ganesha-active and grace-active. When the daemon
stops for any reason, the attributes are deleted. Deleting the
ganesha-active attribute triggers the failover of the virtual IP (the
IPaddr RA) to another node where ganesha.nfsd is still running. The
ganesha_grace RA monitors the grace-active attribute. When the
grace-active attibute is deleted, the ganesha_grace RA stops, and will
not restart. This triggers pacemaker to trigger the notify action in
the ganesha_grace RAs on the other nodes in the cluster; which send a
DBUS message to their ganesha.nfsd.

(N.B. grace-active is a bit of a misnomer. while the grace-active
attribute exists, everything is normal and healthy. Deleting the
attribute triggers putting the surviving ganesha.nfsds into GRACE.)

To ensure that the remaining/surviving ganesha.nfsds are put into
NFS-GRACE before the IPaddr (virtual IP) fails over there is a short
delay (sleep) between deleting the grace-active attribute and the
ganesha-active attribute. To summarize:
  1. on node 2 ganesha_mon:monitor notices that ganesha.nfsd has died
  2. on node 2 ganesha_mon:monitor deletes its grace-active attribute
  3. on node 2 ganesha_grace:monitor notices that grace-active is gone
     and returns OCF_ERR_GENERIC, a.k.a. new error. When pacemaker
     tries to (re)start ganesha_grace, its start action will return
     OCF_NOT_RUNNING, a.k.a. known error, don't attempt further
     restarts.
  4. on nodes 1, 3, etc., ganesha_grace:notify receives a post-stop
     notification indicating that node 2 is gone, and sends a DBUS
     message to its ganesha.nfsd putting it into NFS-GRACE.
  5. on node 2 ganesha_mon:monitor waits a short period, then deletes
     its ganesha-active attribute. This triggers the IPaddr (virt IP)
     failover according to constraint location rules.

ganesha_nfsd modified to run for the duration, start action is invoked
to setup the /var/lib/nfs symlink, stop action is invoked to restore it.
ganesha-ha.sh modified accordingly to create it as a clone resource.

BUG: 1290865
Change-Id: I1ba24f38fa4338b3aeb17c65645e9f439387ff57
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/12964
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-on: http://review.gluster.org/13725

common-ha: Corrected refresh-config output parsing

2016-03-15T04:34:01+00:00

>>>> Sample program with the earlier changes -
output=$(dbus-send --print-reply --system \
--dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr \
org.ganesha.nfsd.exportmgr.RemoveExport uint16:5 2>&1\
  | grep -v "^method return")
ret=$?
echo "${output}"
echo $ret
sleep 1
output=$(dbus-send --system --dest=org.ganesha.nfsd \
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport \
string:/usr/etc/ganesha/exports/export.vol3.conf \
string:"EXPORT(Path=/vol3)" 2>&1  | grep -v "^method return")
ret=$?
echo "${output}"
echo $ret

Output:

1

1

Even if the command was successfully executed, 'grep -v' has
filtered out the output.

>>>> Sample program with the current changes -

output=$(dbus-send --print-reply --system --dest=org.ganesha.nfsd \
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport\
 uint16:5 2>&1)
ret=$?
echo "${output}"
echo $ret
sleep 1
output=$(dbus-send --print-reply --system --dest=org.ganesha.nfsd \
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport \
string:/usr/etc/ganesha/exports/export.vol3.conf \
string:"EXPORT(Path=/vol3)" 2>&1)
ret=$?
echo "${output}"
echo $ret

Output:
method return sender=:1.155 -> dest=:1.174 reply_serial=2
0
method return sender=:1.155 -> dest=:1.175 reply_serial=2
   string "1 exports added"
0

BUG: 1254494
Change-Id: I44fbe32588ec11f087c8b99b2d55ed55ba73727c
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/12439
Tested-by: NetBSD Build System 
Reviewed-by: jiffin tony Thottan 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.org/13724
Tested-by: Kaleb KEITHLEY 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

common-ha: distribution neutral location of config files

2016-03-15T04:33:37+00:00

work on systems with config files in directories other than
/etc/sysconfig.

BUG: 1251821
Change-Id: I009946623b508ba422a4881913455218d8846055
Signed-off-by: Joe Julian 
Reviewed-on: http://review.gluster.org/12132
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.org/13723
Tested-by: Kaleb KEITHLEY 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

common-ha: refresh-config output includes dbus "method return" msg

2016-03-15T04:30:14+00:00

filter out extraneous text from dbus-send

BUG: 1262881
Change-Id: I2fec5c0cc8d1db0a12a42ca2d727a4320fd57b30
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/12174
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: soumya k 
Reviewed-on: http://review.gluster.org/13722
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

CommonHA: Fix the path of 'systemctl' cmd

2016-03-15T04:27:04+00:00

'systemctl' command path should be '/usr/bin/sytemctl'.
Fixed the typo in the HA-script.

BUG: 1259225
Change-Id: I0e7fcc1eb74419679d78d4cc61253c6edf23b57b
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/12089
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-on: http://review.gluster.org/13721
Tested-by: Kaleb KEITHLEY 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System