summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* cluster/dht: Fix dht_access treating directory like filesShyam2014-09-092-4/+105
| | | | | | | | | | | | | | | | | | | | | | | | | When the cluster topology changes due to add-brick, all sub volumes of DHT will not contain the directories till a rebalance is completed. Till the rebalance is run, if a caller bypasses lookup and calls access due to saved/cached inode information (like NFS server does) then, dht_access misreads the error (ESTALE/ENOENT) from the new subvolumes and incorrectly tries to handle the inode as a file. This results in the directories in memory state in DHT to be corrupted and not heal even post a rebalance. This commit fixes the problem in dht_access thereby preventing DHT from misrepresenting a directory as a file in the case presented above. Change-Id: Idcdaa3837db71c8fe0a40ec0084a6c3dbe27e772 BUG: 1138393 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on-master: http://review.gluster.org/8462 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8608 Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* cluster/dht: Added keys in dht_lookup_everywhere_doneVenkatesh Somyajulu2014-09-091-4/+72
| | | | | | | | | | | | | | | | | | | | | Case where both cached (C1) and hashed file are found, but hash does not point to above cached node (C1), then dont unlink if either fd-is-open on hashed or linkto-xattr is not found. Change-Id: I7ef49b88d2c88bf9d25d3aa7893714e6c0766c67 BUG: 1138385 Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com> Change-Id: I86d0a21d4c0501c45d837101ced4f96d6fedc5b9 Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com> Reviewed-on-master: http://review.gluster.org/8429 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: susant palai <spalai@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8607 Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* dht: fix rename raceNithya Balachandran2014-09-091-2/+5
| | | | | | | | | | | | | | | Additional check to check if we created the linkto file before deleting it in the rename cleanup function Change-Id: I919cd7cb24f948ba4917eb9cf50d5169bb730a67 BUG: 1138387 Signed-off-by: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on-master: http://review.gluster.org/8338 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8605 Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* cluster/dht: Fix races to avoid deletion of linkto fileVenkatesh Somyajulu2014-09-095-41/+401
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Explanation of Race between rebalance processes: https://bugzilla.redhat.com/show_bug.cgi?id=1110694#c4 STATE 1: BRICK-1 only one brick Cached File in the system STATE 2: Add brick-2 BRICK-1 BRICK-2 STATE 3: Lookup of File on brick-2 by this node's rebalance will fail because hashed file is not created yet. So dht_lookup_everywhere is about to get called. STATE 4: As part of lookup link file at brick-2 will be created. STATE 5: getxattr to check that cached file belongs to this node is done STATE 6: dht_lookup_everywhere_cbk detects the link created by rebalance-1. It will unlink it. STATE 7: getxattr at the link file with "pathinfo" key will be called will fail as the link file is deleted by rebalance on node-2 Fix: So in the STATE 6, we should avoid the deletion of link file. Every time dht_lookup_everywhere gets called, lookup will be performed on all the nodes. So to avoid STATE 6, if linkto file is found, it is not deleted until valid case is found in dht_lookup_everywhere_done. Case 1: if linkto file points to cached node, and cached file exists, uwind with success. Case 2: if linkto does not point to current cached node, and cached file exists: a) Unlink stale link file b) Create new link file Case 3: Only linkto file exists: Delete linkto file Case 4: Only cached file Create link file (Handled event without patch) Case 5: Neither cached nor hashed file is present Return with ENOENT (handled even without patch) Change-Id: Ibf53671410d8d613b8e2e7e5d0ec30fc7dcc0298 BUG: 1138385 Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com> Reviewed-on-master: http://review.gluster.org/8231 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8603 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/changelog: Removal of redundant fop color count while draining.Ajeet Jha2014-09-091-5/+0
| | | | | | | | | | BUG: 1138952 Change-Id: I594be0d09c6af2e4a34da3e819d1ab6fd85e34c4 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/8542 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-on: http://review.gluster.org/8647
* features/changelog: barrier all entry creation fopsVijay Bellur2014-09-091-27/+308
| | | | | | | | | | | | | | | | | | | | when a snapshot is taken, there are chances of entry creation fops not being recorded either in changelog or through the recursive ancestry xtime updation by marker. This causes consumers of changelog (primarily geo-replication as of today) to not be aware of these entries after a snapshot is restored. This can lead to inconsistencies. This patch is an interim workaround to barrier creates till changelog becomes completely crash consistent. BUG: 1138952 Change-Id: Idd5e690a05fe2c7c5d32d1541a0d9b5132881ea7 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8517 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: ajeet jha <ajha@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8646
* geo-rep/glusterd: API to check active geo-rep session for the volumeKotresh H R2014-09-084-67/+183
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Requirement: Snapshot needs an API to fail the CLI if any geo-rep session is active for that volume. Solution: A function "gd_vol_is_geo_rep_active" is provided to check if any geo-rep session is active for that volume. An in memory dict called 'gsync_running_slaves' is maintained in 'volinfo' structure to keep track of active geo-rep session for the volume. The key 'slavenode::slavevol' with value 'running' is added whenever geo-rep is started/resumed into the dict and the same is removed if stopped/paused. So the 'count' in dict is used to decide whether the geo-rep is active or not for that volume. Also added "this->name" in gf_log in routines which this patch is touched. BUG: 1138952 Change-Id: Ib13aeb509a56edf510651b77e20bf3cc43a3e763 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8459 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/8645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fixing issue with xsync upper limitAravinda VK2014-09-082-59/+50
| | | | | | | | | | | | | | | | | | | | | While identifying the file/dir to sync, xtime of the file was compared with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync, xtime of parent directory is updated as stime. With the upper limit condition, stime is updated as MIN(xtime_parent, xsync_upper_limit) With this files will get missed if `xtime_of_file == xsync_upper_limit` With this patch xtime_of_file is compared as xtime_of_file <= xsync_upper_limit BUG: 1138952 Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/8439 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8644
* geo-rep: Handle RMDIR recursivelyAravinda VK2014-09-081-0/+12
| | | | | | | | | | | | | | | | | | If RMDIR is recorded in brick changelog which is due to self heal traffic then it will not have UNLINK entries for child files. Geo-rep hangs with ENOTEMPTY error on slave. Now geo-rep recursively deletes the dir if it gets ENOTEMPTY. BUG: 1138952 Change-Id: Ie79db90c52103b39fa795bb8a096b363d450b427 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/8477 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/8643 Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* feature/geo-rep: Keep marker.tstamp's mtime unchangeable during snapshot.Kotresh H R2014-09-085-13/+169
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Geo-replicatoin does a full xsync crawl after snapshot restoration of slave and master. It does not do history crawl. Analysis: Marker creates 'marker.tstamp' file when geo-rep is started for the first time. The virtual extended attribute 'trusted.glusterfs.volume-mark' is maintained and whenever it is queried on gluster mount point, marker fills it on the fly and returns the combination of uuid, ctime of marker.tstamp and others. So ctime of marker.tstamp, in other sense 'volume-mark' marks the geo-rep start time when the session is freshly created. From the above, after the first filesystem crawl(xsync) is done during first geo-rep start, stime should always be less than 'volume-mark'. So whenever stime is less than volume-mark, it does full filesystem crawl (xsync). Root Cause: When snapshot is restored, marker.tstamp file is freshly created losing the timestamps, it was originally created with. Solution: 1. Change is made to depend on mtime instead of ctime. 2. mtime and atime of marker.tstamp is restored back when snapshot is created and restored. BUG: 1138952 Change-Id: I0e19e1cb2593171b9a2b41d0d303330feb7fd2b3 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8401 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8642
* geo-rep/libgfchangelog: Support of symlinks while creation of working dir.Kotresh H R2014-09-081-1/+1
| | | | | | | | | | | | | | | In gf_changelog_register, enable symlink support while creating working directory if its not already created. BUG: 1138952 Change-Id: I8fec52a5768fae46ce30a2331f30f1d8d5e2e173 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8409 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/8641 Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep/libgfchangelog: Create working dir during changelog_register if not ↵Kotresh H R2014-09-081-6/+15
| | | | | | | | | | | | | | | | | | | | present. Earlier, xysnc's register was being called first, which was creating working directory before calling changelog_register. Now it is history crawl first. Hence working directory would not have been created. Create it in gf_changelog_register itself if it is not already created. BUG: 1138952 Change-Id: Ie39b9fd8c1ef7385f76a9b67d0acc3c1c2fd2bb2 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8399 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8640
* geo-rep: minimize xsync crawl usage and set upper limit to xsync crawlAravinda VK2014-09-082-38/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For effective handling of deletes and renames use history crawl as much as possible. History crawl will run in loop till it syncs all data before live changelog time. When it uses xsync crawl(fallback when changelog not available, or very first crawl) it sets upper limit to crawl. After completing History crawl, it checks actual end time returned by history api to compare with register time, if actual end is less than register time then run history crawl one more time. If first turn history processing time is less than the CHANGELOG ROLLOVER TIME then sleep for the difference, After sleep if it is guaranteed that rollover will happen and switches to live changelog consumption without switching to xsync. This sleep is only when history processing completed < CHANGELOG_ROLLOVER_TIME and sleep only after the first turn, So will not affect the performance. BUG: 1138952 Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/8151 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/8639 Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: Capture "correct" internal FOPsVenky Shankar2014-09-084-11/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes changelog capturing internal FOPs in a cascaded setup, where the intermediate master would record internal FOPs (generated by DHT on link()/rename()). This is due to I/O happening on the intermediate slave on geo-replication's auxillary mount with client-pid -1. Currently, the internal FOP capturing logic depends on client pid being non-negative and the presence of a special key in dictionary. Due to this, internal FOPs on an inter-mediate master would be recorded in the changelog. Checking client-pid being non-negative was introduced to capture AFR self-heal traffic in changelog, thereby breaking cascading setups. By coincidence, AFR self-heal daemon uses -1 as frame->root->pid thereby making is hard to differentiate b/w geo-rep's auxillary mount and self-heal daemon. BUG: 1138952 Change-Id: Ia08a2cfa3b02bb785f343794f5b2695d44398c4c Original-Author: Venky Shankar <vshankar@redhat.com> Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8347 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8638
* DHT/Create : Failing to identify a linkto file in lookup_everywhere_cbk pathSusant Palai2014-09-081-7/+47
| | | | | | | | | | | | | | | | | In case a file is not found in its cached subvol we proceed with dht_lookup_everywhere. But as we dont add the linkto xattr to the dictionary, we fail to identify any linkto file encountered.The implication being we end up thinking the linkto file as a regular file and proceed with the fop. Change-Id: Iab02dc60e84bb1aeab49182f680c0631c33947e2 BUG: 1138389 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on-master: http://review.gluster.org/8277 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-on: http://review.gluster.org/8602 Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* features/changelog: Crash consistency of changelog wrt snapshotAjeet Jha2014-09-084-0/+232
| | | | | | | | | | | | | | This patch introduces call-path fop details logging for data operations in CHANGELOG.SNAP. This feature is enabled with barrier-enable notification and disabled with barrier-disable notification. BUG: 1138952 Change-Id: Ic418dd70b0a0b369202c5b79a6f7f96512821065 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/8533 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8648
* gluster: Fix the recursive goto outs in the source code.Avra Sengupta2014-09-075-17/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added a script check_goto.pl, that when run from the source code root, will scan all .c files to match the following pattern: label: if (condition) goto label; On finding such a pattern the script will print the file name and the line number. There are certain cases where the above recursive pattern is intended. Hence adding those labels to ignore-labels. Thanks Vijaikumar Mallikarjuna for the perl script. Also fixed all such existing errors BUG: 1138952 Change-Id: Ie6b75621711736e7e30f2f9d25e50435d58fc1e2 Signed-off-by: Vijaikumar Mallikarjuna <vmallika@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/8307 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/8637 Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Fix quotad RPC options use-after-free bug in glusterEmmanuel Dreyfus2014-09-071-1/+4
| | | | | | | | | | | | | | | | | | | | In cli/src/cli.c:cli_quotad_clnt_rpc_init(), dict_unref (rpc_opts) causes the options to be freed while code in rpc/rpc-transport/socket still relies on it. The options are corrupted when memory is reallocated, which sometimes leads to a crashes on NetBSD when socket_connect() attempted to read options. Fix the problem by not doing the dict_unref(). Make sure the rpc_opts are freed on error, though. This is a backport of If1d6ea50cc3e1599e9e369863c8db0c0694d3671 BUG: 1138897 Change-Id: Idcd46c3ee9a7daa64dfeb8950e14d6b99f177bb3 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8635 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Always check for ENODATA with ENOATTREmmanuel Dreyfus2014-09-077-14/+27
| | | | | | | | | | | | | | | | | | | | Linux defines ENODATA and ENOATTR with the same value, which means that code can miss on on the two without breaking. FreeBSD does not have ENODATA and GlusterFS defines it as ENOATTR just like Linux does. On NetBSD, ENODATA != ENOATTR, hence we need to check for both values to get portable behavior. This is a backport of I003a3af055fdad285d235f2a0c192c9cce56fab8 BUG: 1138897 Change-Id: I272cd53e637993c7fd2ac74bd607001d3581ced7 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8634 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* NetBSD /dev/fuse detectionEmmanuel Dreyfus2014-09-071-0/+4
| | | | | | | | | | | | | | | NetBSD's FUSE being pure userland implementation, there is no /dev/fuse to open. Test /dev/puffs (kernel fs-in-userland subsystem supporting FUSE) insead. This is a backport of Ia65e95c246dc31ea2839cf64d7c851430828542e BUG: 1138897 Change-Id: I9beb673cff08d429c8ae66a819266f6037086b3e Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8633 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Enable aux-gfid-mount option on non Linux systemsEmmanuel Dreyfus2014-09-071-3/+0
| | | | | | | | | | | | | | The aux-gfid-mount works on non Linux systems, and it is required to pass tests/basic/gfid-access.t This is a backport of Ic6c8ef425e091440a139bbd25fadbf4f82e378cb BUG: 1138897 Change-Id: Iab5487437fde65a66cff9cfeb7d209db7eb12108 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8632 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Fix glustershd detection on volume restartEmmanuel Dreyfus2014-09-072-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | On NetBSD and FreeBSD, doing a 'gluster volume start $volume force' causes NFS server, quotad, snapd and glustershd to be undetected by glusterd once the volume has restarted. 'gluster volume status' shows the three processes as 'N' in the online column, while they have been launched successfully. This happens because glusterd attempts to connect to its child processes just between the child does a unlink() on the socket in __socket_server_bind() and the time it calls bind() and listen(). Different scheduling policy may explain why the problem does not happen on Linux, but it may pop up some day since we make no guaranteed assumptions here. This patchet works this around by introducing a boolean transport.socket.ignore-enoent option, set by nfs and glustershd, which prevents ENOENT to be fatal and cause glusterd to retry and suceed later. Behavior of other clients is unaffected. This is a backport of Ifdc4d45b2513743ed42ee235a5c61a086321644c BUG: 1138897 Change-Id: I04472f045249c99a9492218ceebfab847474db2d Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8630 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Search zlib without pkg-configEmmanuel Dreyfus2014-09-072-8/+13
| | | | | | | | | | | | | | | | NetBSD has zlib installed in base system and no .pc file for it. In order to discover it, fallback to AC_CHECK_LIB is PKG_CHECK_MODULES fail to find it. This is a backport of Ide84402aa38edc2709d12e2530401c6b8c722529 and I34ded175f056d1a0898804fe602e3d2d2cba27f5 BUG: 1138897 Change-Id: I14a3ab987cf2bedd3d1d00145466647fb646795b Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8628 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/afr: Set pending changelog based on filetype for new entriesPranith Kumar K2014-09-072-4/+40
| | | | | | | | | | | Backport of http://review.gluster.org/8506 BUG: 1136822 Change-Id: Ia864040306405acf9ebddabf63e87dc2016372dd Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/8588 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ec: Test volume mount point in a better wayXavier Hernandez2014-09-071-1/+1
| | | | | | | | | | | | | | | | | | An 'ls -a1' on an empty volume seems to return 3 entries instead of the expected 2 ('.' and '..') in the build servers. I changed the test to a simple 'stat', which is enough and more reliable. This is a backport of http://review.gluster.org/8313. Change-Id: I12d0f47394ad378b40fc9b86507cdb3543f99970 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8313 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8417
* build: make GLUSTERD_WORKDIR rely on localstatedirHarshavardhana2014-09-0371-270/+345
| | | | | | | | | | | | | | | | | | | | | | | Backport from master branch - http://review.gluster.org/#/c/8246/ - Break-way from '/var/lib/glusterd' hard-coded previously, instead rely on 'configure' value from 'localstatedir' - Provide 's/lib/db' as default working directory for gluster management daemon for BSD and Darwin based installations - loff_t is really off_t on Darwin - fix-off the warnings generated by clang on FreeBSD/Darwin - Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all platforms. - Define proper environment for running tests, define correct PATH and LD_LIBRARY_PATH when running tests, so that the desired version of glusterfs is used, regardless where it is installed. (Thanks to manu@netbsd.org for this additional work) Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276 BUG: 1130308 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8486 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cli: Xml output for geo-replication status commandndarshan2014-08-285-55/+237
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds xml output for geo-replication status and status detail command. sample: -------------------------------------------------------------- <geoRep> <volume> <name>master</name> <sessions> <session> <session_slave>:2a301d66-b9d2-44b4-b827-d680d67123eb:ssh://XXXXXXXXXX::slave</session_slave> <pair> <master_node>localhost.localdomain</master_node> <master_node_uuid>2a301d66-b9d2-44b4-b827-d680d67123eb</master_node_uuid> <master_brick>/root/master_b1</master_brick> <slave>ssh://XXXXXXXXXXX::slave</slave> <status>faulty</status> <checkpoint_status>N/A</checkpoint_status> <crawl_status>N/A</crawl_status> </pair> </session> </sessions> </volume> </geoRep> ------------------------------------------------------------- Change-Id: Ia19dbe751c3ab1ec7cb8923cdd6c8b99c374072f BUG: 1133464 Signed-off-by: ndarshan <dnarayan@redhat.com> Reviewed-on: http://review.gluster.org/8089 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushal M <kaushal@redhat.com> Signed-off-by: ndarshan <dnarayan@redhat.com> Reviewed-on: http://review.gluster.org/8532 Reviewed-by: Sachin Pandit <spandit@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cli: Xml output for geo-replication config command.ndarshan2014-08-281-4/+167
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds xml output for geo-replication config command. sample: --------------------------------------------------------------------- <cliOutput> <opRet>0</opRet> <opErrno>0</opErrno> <opErrstr/> <geoRep> <config> <parameter1_name>value</parameter1_name> <parameter2_name>value</parameter2_name> ... ... ... </config> </geoRep> </cliOutput> --------------------------------------------------------------------- Change-Id: Iac0451983ae5d0e65b95604eb1c29b968e1ee22f BUG: 1133464 Signed-off-by: ndarshan <dnarayan@redhat.com> Reviewed-on: http://review.gluster.org/8270 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushal M <kaushal@redhat.com> Signed-off-by: ndarshan <dnarayan@redhat.com> Reviewed-on: http://review.gluster.org/8527 Reviewed-by: Sachin Pandit <spandit@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* NFS: stripe-xlator should pass EOF at end of READDIRNiels de Vos2014-08-251-11/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | NFS READDIR replies are made of a header, a sequence of entries, and a EOF flag. When GlusterFS's NFS server is used along with stripe xlator, it fails to set the EOF flag, which violates NFS RFC and confuses some clients. The bug is caused because nfs xlator sets EOF if it gets op_errno set to ENOENT. That value is produced in storage xlator and propagated through server, client, and other xlators until stripe xlator handles it. stripe only passed op_errno if op_ret < 0, which is not the case here. This change set adds a special case for that situation to fix the problem. Cherry picked from commit 9b5231e5c98b8cfa116838287c7a14042702795f: > Change-Id: Ie6db94b0515292387cfb04c1e4a9363f34fcd19a > BUG: 1130969 > Reported-by: Emmanuel Dreyfus <manu@netbsd.org> > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/8493 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Santosh Pradhan <spradhan@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org> > Tested-by: Emmanuel Dreyfus <manu@netbsd.org> Change-Id: Ie6db94b0515292387cfb04c1e4a9363f34fcd19a BUG: 1132390 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/8508 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
* cluster/afr: Fix mem-leakPranith Kumar K2014-08-191-2/+5
| | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/8457 Problem: local->xattr_req is already reffed with xattr_req that comes in lookup fop. But when afr_lookup_xattr_req_prepare is called local->xattr_req is over-written with dict_new() which leads to ref leak on the dict which came in lookup fop Fix: Create local->xattr_req only when it is NULL BUG: 1128801 Change-Id: I4a1065add7700317e0cd3d0dda0a91e12d77e340 Reviewed-on: http://review.gluster.org/8460 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* porting: OSX/Darwin 10.9 porting issuesHarshavardhana2014-08-152-4/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xdrproc_t() arguments are variadic and non-variadic On OSX > 10.9 ------------- typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int); On OSX < 10.9 ------------ typedef bool_t (*xdrproc_t)(XDR *, ...); FreeBSD all versions ------------ typedef bool_t (*xdrproc_t)(XDR *, ...); NetBSD 6.1.4 ----------- typedef bool_t (*xdrproc_t)(XDR *, const void *); Linux all versions ----------- typedef bool_t (*xdrproc_t)(XDR *, void *,...); This weird and odd implementations across various platforms should be handled properly. Change-Id: I49fab73cba0d965c78c71da1beba1ffb2d58b8f8 BUG: 1130307 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8488 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* osx: snapview-server fails to build on OSX use 'syscall.h'Harshavardhana2014-08-151-0/+1
| | | | | | | | | Change-Id: Ib9443913abb89c6a04a8723bced4c77128047bd0 BUG: 1130307 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8487 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* rdma: glusterfsd SEGV at volume startKaleb S. KEITHLEY2014-08-131-0/+1
| | | | | | | | | | | | | | | | | glusterfsd NULL ptr deref in proto/server: get_frame_from_request() with 'transport rdma' volume no test case, our regression test framework doesn't have Infiniband. If it did, the test case would be to create a 'transport rdma' volume, start it, and create/write/read/delete files on the volume. Change-Id: I8dd4bea08bdecbbdf0115d3badccb1594fa69a27 BUG: 1129710 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/8480 Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/ec: Fix incorrect management of NFS requestsXavier Hernandez2014-08-114-89/+69
| | | | | | | | | | | | | | | | | | | | | | Some operations, specially those comming from NFS, do not use a regular fd and use an anonymous fd (i.e. a previous open call has not been sent). Any context information created during open or create will not be present on these fd's, so we simply return NULL for contexts of those fd. Also it seems that NFS can send write requests with a very big buffer (higher that the default value of 128 KB). Some changes have been made to correctly handle these large buffers. This is a backport of http://review.gluster.org/8367. Change-Id: I281476bd0d2cbaad231822248d6a616fcf5d4003 BUG: 1126734 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8367 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8416
* fuse: fuse_readlink_cbk() - linkname NULL termination unnecessaryHarshavardhana2014-08-031-6/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | op_ret incremented to compensate for NULL terminating character leads to self referential loop where OSXFUSE which would reply on a same READLINK() over and over again ~~~ [2014-07-12 08:41:29.815473] T [fuse-bridge.c:1372:fuse_readlink_cbk] 0-glusterfs-fuse: 1: /a/b/1 => ../../1 [2014-07-12 08:41:29.815820] T [fuse-bridge.c:1372:fuse_readlink_cbk] 0-glusterfs-fuse: 0: /a/b/1 => ../../1 [2014-07-12 08:41:29.816165] T [fuse-bridge.c:1372:fuse_readlink_cbk] 0-glusterfs-fuse: 1: /a/b/1 => ../../1 ~~~ It happens due to the problem being (op_ret + 1) > strlen(linkname), for some odd reason this isn't an issue on Linux where there are odd safegaurds on these things - Example of following code ~~~ ((char *)linkname)[op_ret] = '\0'; send_fuse_data (this, finh, (void *)linkname, op_ret + 2048); <---- Here! ~~~ This behaves normally with no issue, the reasoning i have is that internally 'readlink()' is verified with strlen() again or perhaps the size is re-adjusted to the strlen() of `linkname` This isn't the case on OSX, one needs to make sure that ~~~ strlen(linkname) == op_ret ~~~ Otherwise you would get READLINK() loops as shown above. This patch fixes the problem. Many thanks to Anand Avati for helping me out on this. Change-Id: Ia35818de78a5e4d89bad03ab06e2c5ed6e6753a4 BUG: 1095525 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8336 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ec: Add -msse2 option to avoid fedora/rawhide compilation errorsXavier Hernandez2014-08-031-1/+1
| | | | | | | | | | | | | | | | | | | This solves a compilation error on some compiler versions. To avoid compilation issues on non-Intel architectures, patch http://review.gluster.org/8381/ has already been merged. It disables ec on those architectures. These modifications are temporary patches until a proper solution for bug #1125166 is ready. This is a backport of http://review.gluster.org/8395/ Change-Id: I74d0b79e84601cc4f86ad08ce0f8102b99a79a68 BUG: 1125277 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8396 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* ec: Fixed coverity scan issuesXavier Hernandez2014-08-039-39/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CID list: 1226163 Logically dead code 1226166 Missing break in switch 1226167 Missing break in switch 1226168 Missing break in switch 1226169 Missing break in switch 1226170 Missing break in switch 1226171 Missing break in switch 1226172 Missing break in switch 1226173 Missing break in switch 1226174 Missing break in switch 1226175 Missing break in switch 1226176 Missing break in switch 1226177 Missing break in switch 1226178 Data race condition 1226179 Data race condition 1226180 Data race condition 1226181 Thread deadlock 1226182 Uninitialized pointer read 1226183 Uninitialized pointer read 1226184 Read from pointer after free Change-Id: I4d33aa42289371927175c43bb29e018df64fb943 BUG: 1122834 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/8372 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Disable Erasure Code xlator if MMX is missingEmmanuel Dreyfus2014-07-282-1/+13
| | | | | | | | | | | | | | | Erasure Code xlator requires MMX instruction set. Disable building it if MMX is missing. This is a backport of Id9fe87db33e00c5630c1f3633ebd50fc4ebaec4d BUG: 764655 Change-Id: I880109e42838b5b35ce68508eeb0f8f005faf6f1 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8381 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
* porting: include compat-errno.h for errno'sHarshavardhana2014-07-241-0/+1
| | | | | | | | | | | | | | disperse module fails to compile since ENODATA is non-existent on FreeBSD/Darwin Use errno conversion in compat-errno.h to avoid build issues. Change-Id: I0126224f06365e0b8fac16c205f1294863dc2bc4 BUG: 1111774 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8350 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* porting: extattr_remove_fd() doesn't existHarshavardhana2014-07-211-1/+1
| | | | | | | | | | Fix typo extattr_remove_fd() --> extattr_delete_fd() Change-Id: I0dc3399722eb272d3c04836726e22b212ef15eba BUG: 1111774 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8335 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Update rfc.sh to release-3.6v3.6.0alpha1Vijay Bellur2014-07-181-1/+1
| | | | | | | Change-Id: I4421b2891309d797d8fb7c14d1eb78e483453660 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8328 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* dht: fix rename raceJeff Darcy2014-07-173-3/+112
| | | | | | | | | | | | | | | | | | | | | | | If two clients try to rename the same file at the same time, we sometimes end up with *no file at all* in either the old or new location. That's kind of bad. The culprit seems to be some overly aggressive cleanup code. AFAICT, based on today's study of the code, the intent of the changed section is to remove any linkfile we might have created before the actual rename. However, what we're removing might not be our extra link. If we're racing with another client that's also doing a rename, it might be the only remaining link to the user's data. The solution, which is good enough to pass this test but almost certainly still not complete, is to be more selective about when we do this unlink. Now, we only do it if we know that, at some point, we did in fact create the link without error (notably ENOENT on the source or EEXIST on the destination) ourselves. Change-Id: I8d8cce150b6f8b372c9fb813c90be58d69f8eb7b BUG: 1117851 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/8269 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glusterd: Correctly reset volinfo->caps during volume createKaushal M2014-07-171-14/+12
| | | | | | | | | | Change-Id: I012899be08a06d39ea5c9fb98a66acf833d7213f BUG: 1120589 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/8323 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* porting: use __builtin_ffsll() instead of ffsll()Harshavardhana2014-07-161-0/+4
| | | | | | | | | | | | | - NetBSD/Darwin doesn't implement ffsll() - use Compiler builtin Change-Id: Iee78b4b81747b0bd3877fd2fcb98746f642ce080 BUG: 764655 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8308 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-by: Justin Clift <justin@gluster.org>
* libgfapi: Fixed an issue with healing files during glfs_resolveSoumya Koduri2014-07-162-9/+27
| | | | | | | | | | | | | | | | | | | | | | | | While resolving any path during the first lookup, libgfapi should generate and send gfid as well along with the new inode created to the syncop_lookup(..) so that POSIX translator can heal the files with missing gfid using the new gfid passed. This wasn't happening correctly in the current "glfs_resolve_component(..)" implementation. Fixed the same. Also have added the changes from http://review.gluster.org/5337 in libgfapi, which is a fix to unlink the inode on revalidate if entry not found. In addition to the above, have cleaned up a redundant gfapi log mesage. Change-Id: I0757dda782d16ba6bdbe7ebdbde9c43381229b0a BUG: 1116854 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/7976 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* make snapview-server more compatible with NFS serverRaghavendra Bhat2014-07-167-107/+268
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * There was no handle based API for listxattr. With this change, glfs_h_getxattrs also handles the listxattr functionality by checking whether the name is NULL or not (like posix). But all the gfapi functions for listxattr (glfs_h_getxattrs AND glfs_listxattr AND glfs_flistxattr) returns the names of the xattrs in a buffer provided by the caller. But snapview-server has to return the list of xattrs in a dict itself (similar to posix xlator). But the buffer just contains the names of the xattrs. So for each xattr, a zero byte value is set (i.e. "") into the dict and sent back. Translators which do xattr caching (as of now md-cache which caches selinux and acl related xattrs) should not cache those xattrs whose value is a zero byte data (""). So made changes in md-cache to ignore zero byte values. * NFS server was not linking the inodes to inode table in readdirp. This was leading to applications getting errors. The below set of operations would lead to applications getting error 1) ls -l in one of the snaopshots (snapview-server would generate gfids for each entry on the fly and link the inodes associated with those entries) 2) NFS server upon getting readdirp reply would not link the inodes of the entries. But it used to generate filehandles for each entry and associate the gfid of that entry with the filehandle and send it as part of the reply to nfs client. 3) NFS client would send the filehandle of one of those entries when some activity is done on it. 4) NFS server would not be able to find the inode for the gfid present in the filehandle (as the inode was not linked) and would go for hard resolution by sending a lookup on the gfid by creating a new inode. 5) snapview-client will not able to identify whether the inode is a real inode existing in the main volume or a virtual inode existing in the snapshots as there would not be any inode context. 6) Since the gfid upon which lookup is sent is a virtual gfid which is not present in the disk, lookup would fail and the application would get an error. To handle above situation, now nfs server also does inode linking in readdirp. Change-Id: Ibb191408347b6b5f21cff72319ccee619ea77bcd BUG: 1115949 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/8230 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* mgmt/glusterd: do not check for snapd handle in restore if uss is disabledRaghavendra Bhat2014-07-152-0/+43
| | | | | | | | | Change-Id: I01afe64685a5794cce9265580c6c5de57a045201 BUG: 1119582 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/8310 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushal M <kaushal@redhat.com>
* tests: make dd less noisyJeff Darcy2014-07-152-1/+12
| | | | | | | | | | | | | Also fixed one case in quota.t where error output is expected. There are probably other similar cases which can be fixed separately. Change-Id: If80fad0d9fcff6f8ca91d00f4f7b2d5f3fef4256 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/8298 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Add possibility to run single testsAnders Blomdell2014-07-152-6/+83
| | | | | | | | | | | Change-Id: I9282b711c09611bd0fd4cc814f3ec34aa67d10c0 BUG: 1118453 Signed-off-by: Anders Blomdell <anders.blomdell@control.lth.se> Reviewed-on: http://review.gluster.org/8291 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glusterd: Improvements to peer identificationKaushal M2014-07-1520-722/+1462
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch improves the peer identification mechanism in glusterd and lays down the framework for further improvements, including better multi network support in glusterd. This patch mainly does two things, 1. Extend the peerinfo object to store a list of addresses instead of a single hostname as it does now. This also includes changes to make the peer update behaviour of 'peer probe' to add to the list. 2. Improve glusterd_friend_find_by_hostname() to perform better matching of hostnames. glusterd_friend_find_by_hostname() now does and initial quick string compare against all the peer addresses known to glusterd, after which it tries a more thorough search using address resolution and matching the struc sockaddr's. The above two changes together improve the peer identification situation in glusterd a lot. More information regarding the problem this patch attempts to resolve and the approach chosen can be found at http://www.gluster.org/community/documentation/index.php/Features/Better_peer_identification This commit is a squashed commit of the following changes, the development branch of which can be viewed at, https://github.com/kshlm/glusterfs/tree/better-peer-identification or, https://forge.gluster.org/~kshlm/glusterfs-core/kshlms-glusterfs/commits/better-peer-identification commit 198f86e60fab74faf082eaa02657a4d8f60b92f0 Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 15 14:34:06 2014 +0530 Update gluster.8 commit 35d597f3a6b3248373e727f7b7e889c92554d56c Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 15 09:01:01 2014 +0530 Address review comments https://review.gluster.org/#/c/8238/3 commit 47b5331e17304477322bd2daed5bbed503c34ca1 Merge: c71b12c 78128af Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 15 08:41:39 2014 +0530 Merge branch 'master' into better-peer-identification commit c71b12c164330e8d19d1df4734ab34ef9a8caad2 Merge: 57bc9de 0f5719a Author: Kaushal M <kaushal@redhat.com> Date: Thu Jul 10 19:50:19 2014 +0530 Merge branch 'master' into better-peer-identification commit 57bc9de9e4f49ff2b1620df9906cda50a3527a25 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jul 10 19:49:08 2014 +0530 More fixes to review comments commit 5482cc363a687a9e246a0780ec88acd53e218501 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jul 10 18:36:40 2014 +0530 Code refactoring in peer-utils based on review comments https://review.gluster.org/#/c/8238/2/xlators/mgmt/glusterd/src/glusterd-peer-utils.c commit 89b22c34757178f64d5fbaffa31e6302f841c060 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jul 10 12:30:00 2014 +0530 Hostnames in peer status commit 63ebf9485cf50d736cf640238a1ab241671fcaf1 Merge: c8c8fdd f5f9721 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jul 10 12:06:33 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit c8c8fdd2104b5b6b8a1af739b1dd952b74e6dd66 Author: Kaushal M <kaushal@redhat.com> Date: Wed Jul 9 18:35:27 2014 +0530 Hostnames in xml output commit 732a92a0167ad7b1d70edbc35ebd8307c2766ae1 Author: Kaushal M <kaushal@redhat.com> Date: Wed Jul 9 15:12:10 2014 +0530 Add hostnames to cli rsp dict during list-friends commit fcf43e3e317508f0c225024738a988a4af8e9205 Merge: c0e2624 72d96e2 Author: Kaushal M <kaushal@redhat.com> Date: Wed Jul 9 12:53:03 2014 +0530 Merge branch 'master' into better-peer-identification commit c0e262416728a3c536a8347a216e471eb2251535 Author: Kaushal M <kaushal@redhat.com> Date: Mon Jul 7 16:11:19 2014 +0530 Use list_for_each_entry_safe when cleaning peer hostnames commit 6132e60224eb592f3657e535a12a3e72c772da42 Author: Kaushal M <kaushal@redhat.com> Date: Mon Jul 7 15:52:19 2014 +0530 Fix crash in gd_add_friend_to_dict commit 88ffa9a508fd5aac0b2a76e6e76487ce0cab786a Author: Kaushal M <kaushal@redhat.com> Date: Mon Jul 7 13:19:44 2014 +0530 gd_peerinfo_destroy -> glusterd_peerinfo_destroy commit 4b36930a715b1e13cd1a77d136ef1cf78a06d574 Author: Kaushal M <kaushal@redhat.com> Date: Mon Jul 7 12:50:12 2014 +0530 More refactoring commit ee559b081d608c6501c10ae22166f26eeb65690e Author: Kaushal M <kaushal@redhat.com> Date: Mon Jul 7 12:14:40 2014 +0530 Major refactoring of code based on review comments at https://review.gluster.org/#/c/8238/1/xlators/mgmt/glusterd/src/glusterd-peer-utils.h commit e96dbc7bbb05fad2a9c424de41a394b8023fe48d Merge: 2613d1d 83c09b7 Author: Kaushal M <kaushal@redhat.com> Date: Mon Jul 7 09:47:05 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit 2613d1daebff0c56812de821c06ed4c16bb9d447 Merge: b242cf6 9a50211 Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 15:28:57 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit b242cf66d95dd3dd5e3975aa430baa6bd74b8a29 Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 15:08:18 2014 +0530 Fix a silly mistake, if (ctx->req) => if (ctx->req == NULL) commit c835ed26433830ceed57289143f596cf60421558 Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 14:58:23 2014 +0530 Fix reverse probe. commit 9ede17f9329b854b02e8ad159f173244789fd08c Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 13:31:32 2014 +0530 Fix friend import for existing peers commit 891bf74c7350064dfb008d1b7294bcec28d680fd Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 13:08:36 2014 +0530 Set first hostname in peerinfo->hostnames to peerinfo->hostname commit 9421d6a217381a7427a7d84f369280883ca4297a Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 12:21:40 2014 +0530 Fix gf_asprintf return val check in glusterd_store_peer_write commit defac978c1d94011ce8195e311839b9ffce057e7 Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 11:16:13 2014 +0530 Fix store_retrieve_peers to correctly cleanup. commit 00a799f5de1121b0cb7421da8285f9407063e1bd Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 10:52:11 2014 +0530 Update address list in glusterd_probe_cbk only when needed. commit 7a628e8a9c562d85709c69cfa13fb1774c521b75 Merge: d191985 dc46d5e Author: Kaushal M <kaushal@redhat.com> Date: Fri Jul 4 09:24:12 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit d1919858e6639d2b54d716a61f662d9752ec5ff1 Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 18:59:49 2014 +0530 gf_compare_addrinfo -> gf_compare_sockaddr commit 31d8ef730d408f8d9ba8f504fa648f7dcd59da87 Merge: 93bbede 86ee233 Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 18:16:13 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit 93bbedeac5181e29f59b2acd08f638146812ec41 Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 18:15:16 2014 +0530 Improve glusterd_friend_find_by_hostname glusterd_friend_find_by_hostname will now do an initial quick search for the peerinfo performing string comparisions on the given host string. It follows it with a more thorough match, by resolving the addresses and comparing addrinfos instead of strings. commit 2542cdbc45aa9cfcaf1f174686158d5565cdd07b Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 17:21:10 2014 +0530 New utility gf_compare_addrinfo commit 338676e8389a44bd91136eebd110197429c2566c Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 14:55:56 2014 +0530 Use gd_peer_has_address instead of strcmp commit 28d45be51f594328741c44455bd80ac9d64ca501 Merge: 728266e 991dd5e Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 14:54:40 2014 +0530 Merge branch 'master' into better-peer-identification commit 728266eb16d5f5a4bf36266044425ae164337f99 Merge: 7d9b87b 2417de9 Author: Kaushal M <kaushal@redhat.com> Date: Tue Jul 1 09:55:13 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit 7d9b87b84955ec17daeaf88a3e7462914039430f Merge: b890625 e02275c Author: Kaushal M <kshlmster@gmail.com> Date: Tue Jul 1 08:41:40 2014 +0530 Merge pull request #4 from vpshastry/better-peer-identification Better peer identification commit e02275c52fb83c72ad082c098fd3e432c2b9c526 Merge: 75ee90d b890625 Author: Varun Shastry <vshastry@redhat.com> Date: Mon Jun 30 16:44:29 2014 +0530 Merge branch 'better-peer-identification' of https://github.com/kshlm/glusterfs into better-peer-identification-kaushal-github commit 75ee90d2f272e49b94d24c9ca4571e89a83055ff Author: Varun Shastry <vshastry@redhat.com> Date: Mon Jun 30 15:36:10 2014 +0530 glusterd: add to the list if the probed uuid pre-exists Signed-off-by: Varun Shastry <vshastry@redhat.com> commit b890625d8164c660695daef3285c67979eef723e Merge: 04c5d60 187a7a9 Author: Kaushal M <kaushal@redhat.com> Date: Mon Jun 30 11:44:13 2014 +0530 Merge remote-tracking branch 'origin/master' into better-peer-identification commit 04c5d60cb938c8d94b214689580b40abb1b0ffcd Merge: 3a5bfa1 e01edb6 Author: Kaushal M <kshlmster@gmail.com> Date: Sat Jun 28 19:23:33 2014 +0530 Merge pull request #3 from vpshastry/better-peer-identification glusterd: search through the list of hostnames in the peerinfo commit 0c64f3346a977f9165ac55a84a1e03c40a7573a7 Merge: e01edb6 3a5bfa1 Author: Varun Shastry <vshastry@redhat.com> Date: Sat Jun 28 10:43:29 2014 +0530 Merge branch 'better-peer-identification' of https://github.com/kshlm/glusterfs into better-peer-identification-kaushal-github commit e01edb63153a1008db70b8fa76ae5b535e099326 Author: Varun Shastry <vshastry@redhat.com> Date: Fri Jun 27 12:29:36 2014 +0530 glusterd: search through the list of hostnames in the peerinfo Signed-off-by: Varun Shastry <vshastry@redhat.com> commit 3a5bfa15855e660db2bfde644727371dd2d618cc Merge: cda6d31 371ea35 Author: Kaushal M <kshlmster@gmail.com> Date: Fri Jun 27 11:31:17 2014 +0530 Merge pull request #1 from vpshastry/better-peer-identification glusterd: Add hostname to list instead of replaceing upon update commit 371ea354f198b4182382d5403c5960c0b2add6b6 Author: Varun Shastry <vshastry@redhat.com> Date: Fri Jun 27 11:24:54 2014 +0530 glusterd: Add hostname to list instead of replaceing upon update Signed-off-by: Varun Shastry <vshastry@redhat.com> commit cda6d3152886623ecbf46baf0048ebe0119b30b6 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jun 26 19:52:52 2014 +0530 Import address lists commit 6649b54aa0440130c08e827e0a1d1bbfb840eca9 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jun 26 19:15:37 2014 +0530 Implement export address list commit 55990034eead92bc9b936240029e460a4bf152d5 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jun 26 18:11:59 2014 +0530 Use first address in list to when setting up the peer RPC. commit a35fde8d19b9988eb04c652fb3a5e4f84d90ad00 Author: Kaushal M <kaushal@redhat.com> Date: Thu Jun 26 18:03:04 2014 +0530 Properly free addresses on glusterd_peer_destroy commit 1988081db09ac9205f3dc7268cef8be267f3ce8b Author: Kaushal M <kaushal@redhat.com> Date: Thu Jun 26 17:52:35 2014 +0530 Restore peerinfo with address list implemented. commit 66f524d5749a12f4910dd6b06c9d91f37e1d831e Author: Kaushal M <kaushal@redhat.com> Date: Mon Jun 23 13:02:23 2014 +0530 Move out all peer related utilities from glusterd-utils to glusterd-peer-utils commit 14a2a326a4dff11b55490dca2a14f39320931340 Author: Kaushal M <kaushal@redhat.com> Date: Tue May 27 12:16:41 2014 +0530 Compilation fix commit c59cd351d0a102d0d5f3ea9001fd33c4edcb262f Author: Kaushal M <kaushal@redhat.com> Date: Mon May 5 12:51:11 2014 +0530 Add store support for hostname list commit b70325f0beb884ad12645ef40185f0bf6cedd741 Author: Kaushal M <kaushal@redhat.com> Date: Fri May 2 15:58:07 2014 +0530 Add a hostnames list to glusterd_peerinfo_t glusterd_peerinfo_new will now init this list and add the given hostname as the lists first member. Signed-off-by: Kaushal M <kaushal@redhat.com> Signed-off-by: Varun Shastry <vshastry@redhat.com> Change-Id: Ief3c5d6d6f16571ee2fab0a45e638b9d6506a06e BUG: 1119547 Reviewed-on: http://review.gluster.org/8238 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
generated by cgit (git 2.34.1) at 2024-06-24 23:46:33 +0000