glusterd: reject peer probe from a reinstalled node

Backport of http://review.gluster.org/13519

In a cluster if a node (say N1) goes through a OS reinstallation then probing
some other node in the cluster from N1 doesn't fail as in
gd_validate_mgmt_hndsk_req () uuid & hostname checks are done separately but
there should be one more check where both the conditions should meet.

Steps to create the problem
- N1 probes N2
- bring down glusterd instance on N2
- remove /var/lib/glusterd/* from N2
- restart glusterd instance on N2
- execute gluster peer probe N1 from N2

Validations in gd_validate_mgmt_hndsk_req () has been improved to handle this
special case

Change-Id: I3ba5d8e243bae07a7a6743d01b019e7014d39171
BUG: 1315147
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/13519
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/13619

2 files changed, 20 insertions, 2 deletions

diff --git a/xlators/mgmt/glusterd/src/glusterd-handler.c b/xlators/mgmt/glusterd/src/glusterd-handler.c
index e2bf485e4e2..43fa98e1f6e 100644
--- a/xlators/mgmt/glusterd/src/glusterd-handler.c
+++ b/xlators/mgmt/glusterd/src/glusterd-handler.c
@@ -3839,7 +3839,8 @@ glusterd_xfer_friend_add_resp (rpcsvc_request_t *req, char *myhostname,
 
         gf_msg ("glusterd", GF_LOG_INFO, 0,
                 GD_MSG_RESPONSE_INFO,
-                "Responded to %s (%d), ret: %d", remote_hostname, port, ret);
+                "Responded to %s (%d), ret: %d, op_ret: %d", remote_hostname,
+                port, ret, op_ret);
         GF_FREE (rsp.hostname);
         return ret;
 }
diff --git a/xlators/mgmt/glusterd/src/glusterd-handshake.c b/xlators/mgmt/glusterd/src/glusterd-handshake.c
index 804e06ed01a..d3e5c22225c 100644
--- a/xlators/mgmt/glusterd/src/glusterd-handshake.c
+++ b/xlators/mgmt/glusterd/src/glusterd-handshake.c
@@ -1032,8 +1032,25 @@ gd_validate_mgmt_hndsk_req (rpcsvc_request_t *req, dict_t *dict)
         if (ret)
                 return _gf_false;
 
+        /* If peer object is not found it indicates that request is from an
+         * unknown peer, if its found, validate whether its uuid is also
+         * available in the peerinfo list. There could be a case where hostname
+         * is available in the peerinfo list but the uuid has changed of the
+         * node due to a reinstall, in that case the validation should fail!
+         */
         rcu_read_lock ();
-        ret = (glusterd_peerinfo_find (NULL, hostname) == NULL);
+        peer = glusterd_peerinfo_find (NULL, hostname);
+        if (!peer) {
+                ret = -1;
+        } else if (peer && glusterd_peerinfo_find (peer_uuid, NULL) != NULL) {
+                ret = 0;
+        } else {
+                gf_msg (this->name, GF_LOG_ERROR, 0,
+                        GD_MSG_HANDSHAKE_REQ_REJECTED, "Request from peer %s "
+                        "has an entry in peerinfo, but uuid does not match",
+                        req->trans->peerinfo.identifier);
+                ret = -1;
+        }
         rcu_read_unlock ();
 
         if (ret) {