glusterfs.git/heal, branch v3.9.1 glfsheal: Explicitly enable self-heal xlator options 2016-12-16T01:34:04+00:00 Ravishankar N ravishankar@redhat.com 2016-12-14T17:18:20+00:00 a29e54ecba610b0339bf58466db4d843b6b759a5 Enable data, metadata and entry self-heal as xlator-options so that glfs-heal.c can heal split-brain files even if they are disabled on the volume via volume set commands. > Reviewed-on: http://review.gluster.org/11333 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> > Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> (cherry picked from commit 209c2d447be874047cb98d86492b03fa807d1832) Change-Id: Ic191a1017131db1ded94d97c932079d7bfd79457 BUG: 1405126 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/16143 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> 
Enable data, metadata and entry self-heal as xlator-options so that glfs-heal.c
can heal split-brain files even if they are disabled on the volume via volume
set commands.

> Reviewed-on: http://review.gluster.org/11333
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
(cherry picked from commit 209c2d447be874047cb98d86492b03fa807d1832)

Change-Id: Ic191a1017131db1ded94d97c932079d7bfd79457
BUG: 1405126
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/16143
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
cli: Print to screen frequently 2016-12-07T16:18:30+00:00 Pranith Kumar K pkarampu@redhat.com 2016-11-17T07:41:16+00:00 4543a68e6eb36355e37988247947494e7bcecb97 Problem: CLI appears to be hung because XML document is not flushed periodically Fix: Flush the buffer as soon as we print something >BUG: 1395993 >Change-Id: Ic5f61d4c7d29ee162a124a049e60ceb810d6da6d >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/15863 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> BUG: 1396779 Change-Id: Ie7b4093d54afc7dafc4d1d7c151a9b383cfe3830 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/15885 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> 
Problem:
CLI appears to be hung because XML document is not flushed periodically

Fix:
Flush the buffer as soon as we print something

 >BUG: 1395993
 >Change-Id: Ic5f61d4c7d29ee162a124a049e60ceb810d6da6d
 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
 >Reviewed-on: http://review.gluster.org/15863
 >Smoke: Gluster Build System <jenkins@build.gluster.org>
 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
 >CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
 >Reviewed-by: Atin Mukherjee <amukherj@redhat.com>

BUG: 1396779
Change-Id: Ie7b4093d54afc7dafc4d1d7c151a9b383cfe3830
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/15885
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
cluster/afr: CLI for granular entry heal enablement/disablement 2016-11-29T10:52:15+00:00 Krutika Dhananjay kdhananj@redhat.com 2016-09-22T11:18:54+00:00 eaeef4b80cd9e19f8ecda5f842732470712d476b Backport of: http://review.gluster.org/15747 When there are already existing non-granular indices created that are yet to be healed, if granular-entry-heal option is toggled from 'off' to 'on', AFR self-heal whenever it kicks in, will try to look for granular indices in 'entry-changes'. Because of the absence of name indices, granular entry healing logic will fail to heal these directories, and worse yet unset pending extended attributes with the assumption that are no entries that need heal. To get around this, a new CLI is introduced which will invoke glfsheal program to figure whether at the time an attempt is made to enable granular entry heal, there are pending heals on the volume OR there are one or more bricks that are down. If either of them is true, the command will be failed with the appropriate error. New CLI: gluster volume heal <VOL> granular-entry-heal {enable,disable} Change-Id: Ic79519468a087cd337df664b968188c4adcba43a BUG: 1398500 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/15941 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
        Backport of: http://review.gluster.org/15747

When there are already existing non-granular indices created that are
yet to be healed, if granular-entry-heal option is toggled from 'off' to
'on', AFR self-heal whenever it kicks in, will try to look for granular
indices in 'entry-changes'. Because of the absence of name indices,
granular entry healing logic will fail to heal these directories, and
worse yet unset pending extended attributes with the assumption that
are no entries that need heal.

To get around this, a new CLI is introduced which will invoke glfsheal
program to figure whether at the time an attempt is made to enable
granular entry heal, there are pending heals on the volume OR there
are one or more bricks that are down. If either of them is true, the
command will be failed with the appropriate error.

New CLI: gluster volume heal <VOL> granular-entry-heal {enable,disable}

Change-Id: Ic79519468a087cd337df664b968188c4adcba43a
BUG: 1398500
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/15941
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
glusterd : Introduce reset brick 2016-08-30T02:55:53+00:00 Anuradha Talur atalur@redhat.com 2016-08-22T17:22:03+00:00 936f8aeac3252951e7fa0cdaa5d260fad3bd5ea0 The command basically allows replace brick with src and dst bricks as same. Usage: gluster v reset-brick <volname> <hostname:brick-path> start This command kills the brick to be reset. Once this command is run, admin can do other manual operations that they need to do, like configuring some options for the brick. Once this is done, resetting the brick can be continued with the following options. gluster v reset-brick <vname> <hostname:brick> <hostname:brick> commit {force} Does the job of resetting the brick. 'force' option should be used when the brick already contains volinfo id. Problem: On doing a disk-replacement of a brick in a replicate volume the following 2 scenarios may occur : a) there is a chance that reads are served from this replaced-disk brick, which leads to empty reads. b) potential data loss if next writes succeed only on replaced brick, and heal is done to other bricks from this one. Solution: After disk-replacement, make sure that reset-brick command is run for that brick so that pending markers are set for the brick and it is not chosen as source for reads and heal. But, as of now replace-brick for the same brick-path is not allowed. In order to fix the above mentioned problem, same brick-path replace-brick is needed. With this patch reset-brick commit {force} will be allowed even when source and destination <hostname:brickpath> are identical as long as 1) destination brick is not alive 2) source and destination brick have the same brick uuid and path. Also, the destination brick after replace-brick will use the same port as the source brick. Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de BUG: 1266876 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/12250 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Ashish Pandey <aspandey@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
The command basically allows replace brick with src and
dst bricks as same.

Usage:
gluster v reset-brick <volname> <hostname:brick-path> start
This command kills the brick to be reset. Once this command is run,
admin can do other manual operations that they need to do,
like configuring some options for the brick. Once this is done,
resetting the brick can be continued with the following options.

gluster v reset-brick <vname> <hostname:brick> <hostname:brick> commit {force}

Does the job of resetting the brick. 'force' option should be used
when the brick already contains volinfo id.

Problem: On doing a disk-replacement of a brick in a replicate volume
the following 2 scenarios may occur :

a) there is a chance that reads are served from this replaced-disk brick,
which leads to empty reads. b) potential data loss if next writes succeed
only on replaced brick, and heal is done to other bricks from this one.

Solution: After disk-replacement, make sure that reset-brick command is
run for that brick so that pending markers are set for the brick and it
is not chosen as source for reads and heal. But, as of now replace-brick
for the same brick-path is not allowed. In order to fix the above
mentioned problem, same brick-path replace-brick is needed.
With this patch reset-brick commit {force} will be allowed even when
source and destination <hostname:brickpath> are identical as long as
1) destination brick is not alive
2) source and destination brick have the same brick uuid and path.
Also, the destination brick after replace-brick will use the same port
as the source brick.

Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de
BUG: 1266876
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/12250
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
glfsheal: print brick name and path even when brick is down 2016-08-12T06:25:30+00:00 Ravishankar N ravishankar@redhat.com 2016-08-11T10:10:25+00:00 5ef32c57f327e1dd4e9d227b9c8fd4b6f6fb4970 The xml variant of heal info command does not display brick name when the brick is down due to a failure to fetch the hostUUID. But the non xml variant does. So fixed the xml variant to print the remote_host and remote_subvol even when the brick is down. Change-Id: I16347eb4455b9bcc7a9b0127f8783140b6016578 BUG: 1366222 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/15146 Reviewed-by: Anuradha Talur <atalur@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
The xml variant of heal info command does not display brick name when
the brick is down due to a failure to fetch the hostUUID. But the non
xml variant does. So fixed the xml variant to print the remote_host
and remote_subvol even when the brick is down.

Change-Id: I16347eb4455b9bcc7a9b0127f8783140b6016578
BUG: 1366222
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/15146
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
heal/xml : xml implementation of heal info and splitbrain info 2016-05-08T12:19:50+00:00 hari gowtham hgowtham@redhat.com 2016-04-28T06:18:23+00:00 f0ade919006b2581ae192f997a8ae5bacc2892af Current output: heal didn't have xml output. expected output with patch: gluster v heal v1 info --xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <cliOutput> <healInfo> <bricks> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b1</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b2</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> </bricks> </healInfo> <opRet>0</opRet> <opErrno>0</opErrno> <opErrstr/> </cliOutput> gluster v heal v1 info split-brain --xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <cliOutput> <healInfo> <bricks> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b1</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c"> <name>10.70.42.183:/home/bricks/b2</name> <status>Connected</status> <numberOfEntries>0</numberOfEntries> </brick> </bricks> </healInfo> <opRet>0</opRet> <opErrno>0</opErrno> <opErrstr/> </cliOutput> Change-Id: I20ad5eb7950d4721364ab46089e3af515162db82 BUG: 1331287 Signed-off-by: hari gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/14100 Reviewed-by: Ravishankar N <ravishankar@redhat.com> Tested-by: Ravishankar N <ravishankar@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> 
Current output:
heal didn't have xml output.

expected output with patch:
gluster v heal v1 info --xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <healInfo>
    <bricks>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b1</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b2</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
    </bricks>
  </healInfo>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
</cliOutput>

gluster v heal v1 info split-brain --xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cliOutput>
  <healInfo>
    <bricks>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b1</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
      <brick hostUuid="3677edbb-0a6a-415e-b0ee-b846ff93259c">
        <name>10.70.42.183:/home/bricks/b2</name>
        <status>Connected</status>
        <numberOfEntries>0</numberOfEntries>
      </brick>
    </bricks>
  </healInfo>
  <opRet>0</opRet>
  <opErrno>0</opErrno>
  <opErrstr/>
</cliOutput>

Change-Id: I20ad5eb7950d4721364ab46089e3af515162db82
BUG: 1331287
Signed-off-by: hari gowtham <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/14100
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Ravishankar N <ravishankar@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
heal: Fix incorrect heal info output 2016-05-05T01:46:23+00:00 Anuradha Talur atalur@redhat.com 2016-05-04T06:25:43+00:00 2326f8c2a775f6536d95e2e4676866f1b5dfc368 Problem: In heterogenous clusters, heal info gives incorrect outout as "Failed to process entries completely. Number of entries so far: 0". This happens when the getxattr on virtual xattr for <brickpath>/.glusterfs/indices/dirty fails on older bricks as they do not recognize the xattr. Fix: Ignore that error so that heal info doesn't incorrectly report failure. Change-Id: I0d3541ed0594d67c3c3b8568a89a975a100bf6dd BUG: 1332798 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/14199 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> 
Problem:
In heterogenous clusters, heal info gives incorrect
outout as "Failed to process entries completely.
Number of entries so far: 0".

This happens when the getxattr on virtual xattr
for <brickpath>/.glusterfs/indices/dirty fails
on older bricks as they do not recognize the xattr.

Fix:
Ignore that error so that heal info doesn't
incorrectly report failure.

Change-Id: I0d3541ed0594d67c3c3b8568a89a975a100bf6dd
BUG: 1332798
Signed-off-by: Anuradha Talur <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/14199
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
afr: add mtime based split-brain resolution to CLI 2016-03-29T19:08:35+00:00 Ravishankar N ravishankar@redhat.com 2016-03-25T13:18:30+00:00 08d18ef9257067fac510af408665360019566000 Extended the CLI to include support for split-brain resolution based on mtime. The command syntax is: $:gluster volume heal <VOLNAME> split-brain latest-mtime <FILE> where <FILE> can be either the full file name as seen from the root of the volume (or) the gfid-string representation of the file. Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f BUG: 1321322 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/13828 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> 
Extended the CLI to include support for split-brain resolution based on
mtime. The command syntax is:

$:gluster volume heal <VOLNAME> split-brain latest-mtime <FILE>

where <FILE> can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file.

Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f
BUG: 1321322
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/13828
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
glfs/heal: Use encrypted connection in shd 2016-03-24T23:55:00+00:00 Ashish Pandey aspandey@redhat.com 2016-03-23T05:56:49+00:00 351ec36e3146b7605334cb658927b447b1dbc796 When management encryption is enabled, GlusterD only allows encrypted connections for port 24007. SHD is trying to fetch it's volfile using an unencrypted connection. If /var/lib/glusterd/secure-access is present , i.e. if management ssl is enabled, use encrypted connection fecth info from glusterd. Change-Id: Ia41dcde1a69375379409889ec5b3a70cb95a4fd0 BUG: 1320388 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/13815 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
When management encryption is enabled, GlusterD
only allows encrypted connections for port 24007.
SHD is trying to fetch it's volfile using an
unencrypted connection.

If /var/lib/glusterd/secure-access is present , i.e.
if management ssl is enabled, use encrypted connection
fecth info from glusterd.

Change-Id: Ia41dcde1a69375379409889ec5b3a70cb95a4fd0
BUG: 1320388
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/13815
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
heal: Remove sleep() 2016-02-11T17:05:46+00:00 Pranith Kumar K pkarampu@redhat.com 2016-02-10T09:58:39+00:00 320779d53ae013147d5e2556d2946c73e45734ab I wrote this program from a sample gfapi program which had sleep. I am not sure why this sleep was needed. So removing it now. Changed tests/bugs/replicate/bug-1190069-afr-stale-index-entries.t to execute count_sh_entries every second, instead of comparing same value over and over. Change-Id: I7b89d6cab3e50bb7bf4d40a6064f2d8734155bea BUG: 1306199 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/13421 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> 
I wrote this program from a sample gfapi program which had sleep.
I am not sure why this sleep was needed. So removing it now.

Changed tests/bugs/replicate/bug-1190069-afr-stale-index-entries.t
to execute count_sh_entries every second, instead of comparing
same value over and over.

Change-Id: I7b89d6cab3e50bb7bf4d40a6064f2d8734155bea
BUG: 1306199
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13421
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>