summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorCsaba Henk <csaba@redhat.com>2012-04-04 03:12:26 +0200
committerVijay Bellur <vijay@gluster.com>2012-05-23 01:59:47 -0700
commit80f4d9a0561f7b263c2d9f0e1cc941208b58a468 (patch)
treed6f1ea6485f48728e7ae634b4e5726e823f193d8
parent8e7f28fa1ebc6d7efdc892856884a92191e47040 (diff)
geo-rep / gsyncd: recognize ECONNABORTED as termination of aux glusterfs
Don't dump stack, rather log the "glusterfs session went down" message. If the aux glusterfs is already dead when we try to do some file operation, we get a failure with ENOTCONN, which is already handled as above. However, it's also possible that glusterfs dies while we are in a syscall into it -- in that case we get ECONNABORTED, and so far then we end up with an ugly stack strace. From now on we take ECONNABORTAD as well into consideration. Nb. wrt. testing: it's not easy to synthetically force the aux glusterfs to end this way; for that we have to provoke gsyncd into intensive synchronization. I succeeded in that with the following ruby oneliner: ruby -rcgi -e ' Dir.chdir($*[0]) a=[] Thread.new { loop { while a.size >= 100; File.delete a.shift; end; sleep 1 }} loop { a<<CGI.escape(STDIN.read 10); open(a[-1], "w") {}}' MTPT < /dev/urandom where the geo-rep master is mounted at MTPT. With this going on, deliver a SIGKILL to the geo-rep session's aux glusterfs. (It is giving ECONNABORTED non-deterministically, actually in the minority of cases.) Change-Id: I24fd8d0295cdba91d8b994057a1255ca8e2d1a67 BUG: 764510 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3078 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com> Reviewed-on: http://review.gluster.com/3408
-rw-r--r--xlators/features/marker/utils/syncdaemon/syncdutils.py6
1 files changed, 3 insertions, 3 deletions
diff --git a/xlators/features/marker/utils/syncdaemon/syncdutils.py b/xlators/features/marker/utils/syncdaemon/syncdutils.py
index c0f0ddd481a..b29936d596d 100644
--- a/xlators/features/marker/utils/syncdaemon/syncdutils.py
+++ b/xlators/features/marker/utils/syncdaemon/syncdutils.py
@@ -6,7 +6,7 @@ import fcntl
import shutil
import logging
from threading import Lock, Thread as baseThread
-from errno import EACCES, EAGAIN, EPIPE, ENOTCONN, EINTR
+from errno import EACCES, EAGAIN, EPIPE, ENOTCONN, ECONNABORTED, EINTR, errorcode
from signal import SIGTERM, SIGKILL
from time import sleep
import select as oselect
@@ -174,8 +174,8 @@ def log_raise_exception(excont):
if hasattr(gconf, 'transport'):
gconf.transport.wait()
gconf.transport.terminate_geterr()
- elif isinstance(exc, OSError) and exc.errno == ENOTCONN:
- logging.error('glusterfs session went down')
+ elif isinstance(exc, OSError) and exc.errno in (ENOTCONN, ECONNABORTED):
+ logging.error('glusterfs session went down [%s]', errorcode[exc.errno])
else:
logtag = "FAIL"
if not logtag and logging.getLogger().isEnabledFor(logging.DEBUG):