summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* nfs / mount files : porting log messages to a new frameworkManikandan Selvaganesh2015-05-063-234/+339
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10210/ Porting log messages to a new framework for mount3udp_svc.c, mount3.c, mount3-auth.c files. Cherry picked from ee6cd10ce947787bf6ea589bca1cb259c4bb214d > Change-Id: I1e9cab500b6052bfac43c5ae1c82278e20d2f898 > BUG: 1194640 > Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> > Reviewed-on: http://review.gluster.org/10210 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Niels de Vos <ndevos@redhat.com> Change-Id: I1e9cab500b6052bfac43c5ae1c82278e20d2f898 BUG: 1217722 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/10551 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* nfs / acl3.c, nlm files : port log messages to a new frameworkManikandan Selvaganesh2015-05-063-155/+277
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10217/ Porting log messages to a new framework for acl3.c, nlm4.c, nlmcbk_svc.c Cherry picked from a7f848245ae5b0e4bbe363de13ec1676753352c5 > Change-Id: Iee4daae4fe1460eddf95d285615335b836d7f958 > BUG: 1194640 > Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> > Reviewed-on: http://review.gluster.org/10217 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Niels de Vos <ndevos@redhat.com> Change-Id: Iee4daae4fe1460eddf95d285615335b836d7f958 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> BUG: 1217722 Reviewed-on: http://review.gluster.org/10549 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* nfs/auth,exports,netgroup : port log messages to a new frameworkManikandan Selvaganesh2015-05-063-45/+70
| | | | | | | | | | | | | | | | | | | | | | | | Five gf_log messages are not changed to gf_msg because gf_msg not sending the output to STDOUT. Bug id : BZ1218506 Cherry picked from de7c629d02b723e6b4c3d50a14ea5f072e27ccc1 > Change-Id: Ie8de286355becc3fd73fa80057734b834cf40b04 > BUG: 1194640 > Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> > Reviewed-on: http://review.gluster.org/10173 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Change-Id: Ie8de286355becc3fd73fa80057734b834cf40b04 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> BUG: 1217722 Reviewed-on: http://review.gluster.org/10546 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* glusterd: Use uint32_t for peerinfo generation numberKaushal M2015-05-064-11/+11
| | | | | | | | | | | | | | | | | | | | | Using a uint64_t for the peerinfo generation number was overkill for how the generation number is used within GlusterD. It also prevented GlusterD from running on 32-bit architechtures, as uatomic_add_return doesn't support 64-bit values on 32-bit architechtures. This change was developed on the git branch at [1]. This commit is a combination of the following commits on the development branch. b78dba4 Use 32-bit generation number 2c37e4b Change other generation number variables to uint32_t [1]: https://github.com/kshlm/glusterfs/tree/urcu Change-Id: I0f310f56a4fb97d6bcbc23255a379ed5bb1ed9e1 BUG: 1218031 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/10426 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* protocol: increase default group-cache-timeout to 300 secondsNiels de Vos2015-05-064-4/+4
| | | | | | | | | | | | | | | | | | | | | | sssd uses 300 seconds by default too. There is no need to overload sssd with requests that it would have cached. Cherry picked from commit 34833364e9839f0036bccd58ec0a8a963e69263e: > BUG: 1215187 > Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/10371 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34 BUG: 1215189 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10523 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Minimize rm -rf race in Geo-repAravinda VK2015-05-063-26/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While doing RMDIR worker gets ENOTEMPTY because same directory will have files from other bricks which are not deleted since that worker is slow processing. So geo-rep does recursive_delete. Recursive delete was done using shutil.rmtree. once started, it will not check disk_gfid in between. So it ends up deleting the new files created by other workers. Also if other worker creates files after one worker gets list of files to be deleted, then first worker will again get ENOTEMPTY again. To fix these races, retry is added when it gets ENOTEMPTY/ESTALE/ENODATA. And disk_gfid check added for original path for which recursive_delete is called. This disk gfid check executed before every Unlink/Rmdir. If disk gfid is not matching with GFID from Changelog, that means other worker deleted the directory. Even if the subdir/file present, it belongs to different parent. Exit without performing further deletes. Retry on ENOENT during create is ignored, since if CREATE/MKNOD/MKDIR failed with ENOENT will not succeed unless parent directory is created again. Rsync errors handling was handling unlinked_gfids_list only for one Changelog, but when processed in batch it fails to detect unlinked_gfids and retries again. Finally skips the entire Changelogs in that batch. Fixed this issue by moving self.unlinked_gfids reset logic before batch start and after batch end. Most of the Geo-rep races with rm -rf is eliminated with this patch, but in some cases stale directories left in some bricks and in mount point we get ENOTEMPTY.(DHT issue, Error will be logged in Slave log) BUG: 1218922 Change-Id: I8716b88e4c741545f526095bf789f7c1e28008cb Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10204 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10599 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Status EnhancementsAravinda VK2015-05-0613-717/+917
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Discussion in gluster-devel http://www.gluster.org/pipermail/gluster-devel/2015-April/044301.html MASTER NODE - Master Volume Node MASTER VOL - Master Volume name MASTER BRICK - Master Volume Brick SLAVE USER - Slave User to which Geo-rep session is established SLAVE - <SLAVE_NODE>::<SLAVE_VOL> used in Geo-rep Create command SLAVE NODE - Slave Node to which Master worker is connected STATUS - Worker Status(Created, Initializing, Active, Passive, Faulty, Paused, Stopped) CRAWL STATUS - Crawl type(Hybrid Crawl, History Crawl, Changelog Crawl) LAST_SYNCED - Last Synced Time(Local Time in CLI output and UTC in XML output) ENTRY - Number of entry Operations pending.(Resets on worker restart) DATA - Number of Data operations pending(Resets on worker restart) META - Number of Meta operations pending(Resets on worker restart) FAILURES - Number of Failures CHECKPOINT TIME - Checkpoint set Time(Local Time in CLI output and UTC in XML output) CHECKPOINT COMPLETED - Yes/No or N/A CHECKPOINT COMPLETION TIME - Checkpoint Completed Time(Local Time in CLI output and UTC in XML output) XML output: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> cliOutput> geoRep> volume> name> sessions> session> session_slave> pair> master_node> master_brick> slave_user> slave/> slave_node> status> crawl_status> entry> data> meta> failures> checkpoint_completed> master_node_uuid> last_synced> checkpoint_time> checkpoint_completion_time> BUG: 1218586 Change-Id: I944a6c3c67f1e6d6baf9670b474233bec8f61ea3 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10121 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10574 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: log ENTRY failures from slave on masterMilind Changire2015-05-063-10/+48
| | | | | | | | | | | | | | | | ENTRY operations failures on slave left no trace for debugging purposes. This patch captures such failures on slave cluster and forwards them to the master and logs them. Failures of specific interest are the ones which return code EEXIST on the failing operations. Change-Id: Iecab876f16593c746d53f4b7ec2e0783367856bb Signed-off-by: Milind Changire <mchangir@redhat.com> BUG: 1217429 Reviewed-on: http://review.gluster.org/10048 Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10517 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* doc: AFR arbiter volume usageRavishankar N2015-05-062-0/+60
| | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10541/ Contains information on creation and behaviour of replica 3 arbiter volumes. Signed-off-by: Ravishankar N <ravishankar@redhat.com> Change-Id: I6af4aa3488649686fdb9b839c733046160e0785b BUG: 1217689 Reviewed-on: http://review.gluster.org/10542 Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
* glusterd: Enable readdir-ahead by default on new volumesanand2015-05-062-4/+17
| | | | | | | | | | | | With gluster-3.7, 'performance.readdir-ahead' will be enabled by default on new volumes when the cluster op-version supports it. Change-Id: I44e76a69e7d1c11e6dfad72c941caf887bb810ee BUG: 1217135 Signed-off-by: anand <anekkunt@redhat.com> Reviewed-on: http://review.gluster.org/10450 Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* mgmt/glusterd: Porting messages to new logging frameworkNandaja Varma2015-05-065-150/+258
| | | | | | | | | | | | Backport of: http://review.gluster.org/9808 Change-Id: I25f3536446798ea1cffd6b5dfbb3d2398766fcf3 BUG: 1194640 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/9808 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/10547
* snapshot/scheduler: Use shutil.move instead of os.rename()Avra Sengupta2015-05-061-2/+3
| | | | | | | | | | | | | | | | os.rename is a wrapper on top of the rename function, which fails with invalid cross-device link if /tmp is a tmpfs. Hence using shutil.move Change-Id: Ia026d2a810b725ccd398db895e612c53bc6a2f95 BUG: 1218575 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/10347 Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/10560
* tests: fix failures due to not unmounting $M2 (/mnt/glusterfs/2)Jeff Darcy2015-05-061-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | Our failure to unmount meant that both mkdir and rmdir would fail in cleanup(). Because one of those mkdirs was the last thing cleanup() executed, it would fail, so the test would fail, so the entire regression run would fail. The fix has two parts. (1) Unmount the offending directory. (2) Make sure cleanup() returns success even if that last mkdir failed. That might keep us from consistently blowing up regression runs on the very first tests (basic/afr/data-self-heal.t) that we execute. Change-Id: I7a9761bd28761a5ee2face3db8112e9c3f6c5dc8 BUG: 1163543 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/10536 Reviewed-by: Justin Clift <justin@gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-on: http://review.gluster.org/10557 Tested-by: Vijay Bellur <vbellur@redhat.com>
* Tests: workaround NetBSD failures in cdc.tEmmanuel Dreyfus2015-05-061-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The volume reset network.compression operation cause brick processes to be restarted. If the volume is already started, a brick process is already there and the restart will fail, as the brick TCP port is already in use. Because the new brick process is not started, the volume is left with no brick online, and the volume stop operation will timeout waiting for bricks to stop. Obviosuly we have two bugs here - If volume reset network.compression needs to restart the bricks, it should first make sure the previous brick process is terminated - volume stop should not wait forever for bricks to come back online This change does not fix the bugs but just makes sure the volume is stoped before volume reset network.compression, so that the failure oes not happen. backport of: I9cd5cdc767ef6ee9dd31f2121d672dc3bfdce45f BUG: 1212676 Change-Id: If85c46f7ed675c4c84cc4170292b1f0f41e5b456 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10587 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* syncop: Implement syncop_fxattropPranith Kumar K2015-05-062-0/+23
| | | | | | | | | | | Backport of http://review.gluster.org/10382 BUG: 1216303 Change-Id: I4433002906efc6894b4ff8de8fefe8b7bc954dcf Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10438 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tools/glusterfind: New option to pre --regenerate-outfileAravinda VK2015-05-061-0/+9
| | | | | | | | | | | | | | | When pre command is run twice, it overwrites the outfile. Now pre command will fail when executed twice. To force the regeneration use --regenerate-outfile Change-Id: I0cf7a139522812ece4decdfbcba667a05ce5c35e Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1218166 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10418 Reviewed-on: http://review.gluster.org/10522 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* snapshot/scheduler: Use os.path.realpath() for path validationAvra Sengupta2015-05-061-1/+1
| | | | | | | | | | | | | | | In order to accomodate systems, where /var/run is a symlink to /run, we are using os.path.realpath() for path validations. Change-Id: I4eae536867ec6c88f92c762b92f5c1966b622bde BUG: 1218585 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/10464 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/10573
* snapshot/uss: fix regression failure in bug-1162498.tAvra Sengupta2015-05-063-7/+6
| | | | | | | | | | | | | | | | .snaps seems to take some time, before it is available based on the state of the system. Using EXPECT_WITHIN instead of TEST to check the contents of .snaps, hence giving it some time to come up. Change-Id: Iac166500d5a09ba8bab00d994c27a9ad0a01b9c3 BUG: 1218576 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/10518 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10561
* glusterd: Coverity fixGaurav Kumar Garg2015-05-062-4/+21
| | | | | | | | | | | | | | | | | | | | | CID: 1293504 (Calling xlator_set_option without checking return value ) CID: 1293502 (Dereferencing a pointer that might be null xl when calling xlator_set_option) CID: 1293500 (Assigning value from dict_get_int32(dict, "type", &type) to ret here, but that stored value is overwritten before it can be used.) Change-Id: I5314fb399480df70bd77bc374e3b573f2efd5710 BUG: 1093692 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10201 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kaushal M <kaushal@redhat.com> (cherry picked from commit ed9e9c3b1867867a828020b8569c0c7ab1d3be24) Reviewed-on: http://review.gluster.org/10578
* features/shard: Implement readv() fopKrutika Dhananjay2015-05-062-269/+635
| | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/10528/ Change-Id: I3ff03d146a8d49cc11e7bf22ffbf830b4dd1e9f1 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10569 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* doc/geo-rep: Doc changes w.r.t common shared gluster meta volumeKotresh HR2015-05-061-8/+23
| | | | | | | | | | BUG: 1217939 Change-Id: I0fbb03e809793030e614c40acc930a0579f88d52 Reviewed-on: http://review.gluster.org/10447 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10555 Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
* glusterd: do not pass volinfo in glusterd_svc_manager functionGaurav Kumar Garg2015-05-066-56/+42
| | | | | | | | | | | | | | | | | | | | | On restarting of glusterd first it will start all the bricks present in the volume then it will start all the services. During starting of all the services it may pass volinfo as a NULL. It will cause Assert failure in glusterd_bitdsvc_manager function and will cause a glusterd crash. Change-Id: Ia14cf5022da88516cdd576eb2d1e0e7b17a3782b BUG: 1218039 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10241 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com> (cherry picked from commit 7648c0de36c7927b588abc66734c5b94afb08c00) Reviewed-on: http://review.gluster.org/10481 Tested-by: NetBSD Build System Reviewed-by: Kaushal M <kaushal@redhat.com>
* tests: Fix spurious failuresPranith Kumar K2015-05-055-3/+7
| | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10480 http://review.gluster.org/10487 http://review.gluster.org/10488 http://review.gluster.com/10540 - Use REBALANCE_TIMEOUT in EXPECT_WITHIN - Use fdatasync to prevent write - behind from giving success - Add logfile to glupy - Check aux umount is unmounted for quota tests - Disable flush behind so that file will be closed Change-Id: Ia2447176273557738b7a71cc25dd9b9effc16d58 BUG: 1218485 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10543 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* build: glusterfs-api-devel requires libacl-develKaleb S. KEITHLEY2015-05-051-0/+1
| | | | | | | | | | | libvirt build fails Change-Id: Ib3aa77fb3c090950757bd47b026858c8e1fa55e6 BUG: 1218400 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/10583 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* tests: Check aux umount is unmounted for quota testsPranith Kumar K2015-05-0525-25/+57
| | | | | | | | | | | Change-Id: If57d08f3446755ea41f66ca258efcc8ea5a89063 BUG: 1218593 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10480 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10575
* glusterd: New logging framework, all the message labelsNandaja Varma2015-05-051-1/+3412
| | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/10332/ Change-Id: I5f83b070638384b3f2e3efe936f64080c3334eab BUG: 1194640 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/10332 Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Kaushal M <kaushal@redhat.com> (cherry picked from commit 6627f191ef28504780c8d27ae5c2f472478f8a14) Reviewed-on: http://review.gluster.org/10404 Reviewed-by: Avra Sengupta <asengupt@redhat.com> Tested-by: NetBSD Build System
* rpc: Maintain separate xlator pointer in 'rpcsvc_state'Kotresh HR2015-05-056-12/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The structure 'rpcsvc_state', which maintains rpc server state had no separate pointer to track the translator. It was using the mydata pointer itself. So callers were forced to send xlator pointer as mydata which is opaque (void pointer) by function prototype. 'rpcsvc_register_init' is setting svc->mydata with xlator pointer. 'rpcsvc_register_notify' is overwriting svc->mydata with mydata pointer. And rpc interprets svc->mydata as xlator pointer internally. If someone passes non xlator structure pointer to rpcsvc_register_notify as libgfchangelog currently does, it might corrupt mydata. So interpreting opaque mydata as xlator pointer is incorrect as it is caller's choice to send mydata as any type of data to 'rpcsvc_register_notify'. Maintaining two different pointers in 'rpcsvc_state' for xlator and mydata solves the issue. BUG: 1218381 Change-Id: I4c28937a30845e3f41b6fc7a09036149c816659b Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10366 Reviewed-on: http://review.gluster.org/10534 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Snapshot: New logging framework implementation of snapshot messages.Nandaja Varma2015-05-052-774/+1455
| | | | | | | | | | | | | Backport of: http://review.gluster.org/9796 Change-Id: I3e8b5f6ae59d16752beb8053c2afddfb5c00be14 BUG: 1194640 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/9796 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/10405 Reviewed-by: Sakshi Bansal
* feature/changelog: Capture path for deletesKotresh HR2015-05-057-6/+194
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROBLEM: There is no way to get the path of deleted file if we have gfid from changelog since the file is already deleted. SOLUTION: Do a recursive readlink on parent gfid in backend .glusterfs path to get the complete path in I/O callpath in changelog translator and capture it in callback. The path captured is relative from the brick root. The field separator used is '\0'. e.g., ......\0<pgfid>/bname\0<relative-path>\0<next-record> ADDITIONAL REQUIRED CHANGES: 1. The changelog translator option called "changelog.capture-del-path" is introduced to enable or disable the capturing of deleted entry path. e.g., gluster vol set <vol-name> changelog.capture-del-path on/off If capture-del-path is disabled, '\0' is captured instead of relative path. e.g., ......\0<pgfid>/bname\0\0\0<next-record> 2. The minor number in the version of changelog is bumped up from v1.1 to v1.2. 3. If recursive readlink is failed for some reason, it will capture \0 in place of <relative path>. e.g., ......\0<pgfid>/bname\0\0\0<next-record> (same as when caputre-del-path option is disabled) 4. If bname argument passed to "resolve_pargfid_to_path" function is NULL and pargfid is ROOT, "." is returned. This is not the case with changelog, where bname is always passed. This is applicable to other consumers of "resolve_pargfid_to_path" routine. NOTE: Changelog parser should consider the above new changes and should parse accordingly. BUG: 1218383 Change-Id: I5d89cf4157befd207771f6c0248d2493fbf85832 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10288 Reviewed-on: http://review.gluster.org/10535 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* doc/snapshot: doc for snapshot cloneMohammed Rafi KC2015-05-051-1/+18
| | | | | | | | | | | Change-Id: Ie53e6ab780ab67ffe0c4f6d92fe4c0b779cec2c9 BUG: 1218584 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10187 Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/10572
* libglusterfs: Implement cluster-syncopPranith Kumar K2015-05-057-422/+2480
| | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/10240 This patch implements syncop equivalent for cluster of xlators. The xlators on which the fop needs to be performed is taken in input arguments to the functions and the responses are gathered and provided as the output. This idea is taken from afr-v2 self-heal implementation by Avati. BUG: 1216303 Change-Id: I189400ea5bb3205aae928a72afbb6c960968b65a Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10439 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* afr: add arbitration supportRavishankar N2015-05-0510-47/+272
| | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10258/ Add logic in afr to work in conjunction with the arbiter xlator when a replica 3 arbiter volume is created. More specifically, this patch: * Enables full locks for afr data transaction for such volumes. * Removes the upfront marking of pending xattrs at the time of pre-op and defer it to post-op. (This is an arbiter independent change and is made for all afr transactions.) * After pre-op stage, check if we can proceed with the fop stage without ending up in split-brain by examining the changelog xattrs. * Unwinds the fop with failure if only one source was available at the time of pre-op and the fop happened to fail on particular source brick. * Skips data self-heal if arbiter brick is the only source available. * Adds the arbiter-count option to the shd graph. This patch is a part of the arbiter logic implementation for 3 way AFR details of which can be found at http://review.gluster.org/#/c/9656/ Change-Id: I9603db9d04de5626eb2f4d8d959ef5b46113561d BUG: 1217689 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/10514 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/changelog: Consider only changelog on/off as changelog breakageKotresh HR2015-05-057-11/+268
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, both chagelog on/off and brick restart were considered to be changelog breakage and treated as changelog not being continuous. As a result, new HTIME.TSTAMP file was created on both the above cases. Now the change is made such that only on changelog enable/disable, the changelog is considered to be discontinuous. New HTIME.TSTAMP file is not created on brick restart, the changelogs files are appended to last HTIME.TSTAMP file. Treating changelog as continuous in above scenario is important as changelog history API will fail otherwise. It can successfully get changes between start and end timestamps only when changelog is continuous (Changelogs in single HTIME.TSTAMP file are treated as continuous). Without this change, changelog history API would fail, and it would become necessary to fallback to other mechanisms like xsync FSCrawl in case geo-rep to detect changes in this time window. But Xsync FSCrawl would not be applicable to other consumers like glusterfind. Rationale: 1. In plain distributed volume, if brick goes down, no I/O can happen onto the brick. Hence changelog is intact with data on disk. 2. In distributed replicate volume, if brick goes down, since self-heal traffic is captured in changelog. Eventually, I/O happened whend brick down is captured in changelog. BUG: 1217944 Change-Id: Ifa6d932818fe1a3a914e87ac84f1d2ded01c1288 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10222 Reviewed-on: http://review.gluster.org/10507 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/trash : Notify CTR translator if an unlink happens to a fileJiffin Tony Thottan2015-05-046-16/+158
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/9989/ This implementation is same as the posix_unlink_cbk() where CTR sends a request during a unlink to send the number of links to the inode and posix obliges sending it using the unwind xdata dict. For Trash xlator a unlink is stat + mkdir(if parent is not present) + rename. And hence this is handled in trash_unlink_rename_cbk(). Change-Id: I402e83567b88e3c9fe171379693c82937af567f9 BUG: 1218032 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Anoop C S <achiraya@redhat.com> Reviewed-on: http://review.gluster.org/10513 Reviewed-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix ignoring geo-rep safe errorsKotresh HR2015-05-042-8/+11
| | | | | | | | | | | | | | | | | Fix ignoring geo-rep safe errors in fuse layer and also ignore logging in client translator for mknod. Though it is rare, to happen with mknod, it might happen with history crawl on overlapping changelogs replay. BUG: 1217938 Change-Id: If06f7a6b6f86a315b4e033e294d6f6be67135cb8 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10422 Reviewed-on: http://review.gluster.org/10533 Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Changes required as part of changelog version 1.2Kotresh HR2015-05-041-0/+6
| | | | | | | | | | | | | | | | | | | | The index value for UNLINK and RMDIR in changelog is no more the last index. It varies based on whether the 'changelog.capture-del-path' is enabled or not. Hence, fixed index is used. The option to capture deleted path in changelog comes with the patch: http://review.gluster.org/#/c/10288/ And the parser changes with http://review.gluster.org/#/c/10166/ BUG: 1217935 Change-Id: I6d8a941129416381e24226faed9803f7e5fae8d0 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10344 Reviewed-on: http://review.gluster.org/10500 Tested-by: NetBSD Build System Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Upcall: Feature doc for Upcall infrastrucutreSoumya Koduri2015-05-041-0/+33
| | | | | | | | | | | Change-Id: I802ba2f13fde6c05da1ed355e340f071e9d20d30 BUG: 1217711 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/10525 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/10527 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Fix minor bugs in meta-volume setupKotresh HR2015-05-041-1/+2
| | | | | | | | | | | | | | | | | | | | | | | 1. Access unreferenced access of fd: In meta volume configuration for geo-rep, if geo-rep directory is not created yet, open fails with no fd, but it is accessed in close(fd). So after creating 'geo-rep' directory in meta-volume, open the lock file to get fd. 2. Fix volume_id in forming lock file name. For the very first time, gconf.volume_id would be null, as config is not reloaded yet. Hence, use 'uuid' function to get the volume id. BUG: 1217939 Change-Id: I06bf659ec506943c7ebf5303cc56a2e7748fbad0 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10458 Reviewed-on: http://review.gluster.org/10506 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Don't log geo-rep safe errors in mount logsKotresh HR2015-05-043-2/+7
| | | | | | | | | | | | | | | | | | | | | EEXIST and ENOENT are safe errors for geo-replication. Since mkdir is captured in all the bricks of the changelog. mkdir is tried multiple times as per the number of bricks. The first one to process by gsyncd will succeed and all others will get EEXIST. Hence EEXIST is a safe error and can be ignored. Similarly ENOENT also in rm -rf case. And also gsyncd validates these errors and log them in master if it is genuine error. This is up with the patch http://review.gluster.org/#/c/10048/ Hence ignoring above said safe errors. BUG: 1217938 Change-Id: I1962a85f23fe5e30448ceec1b6ddcb5724ed5627 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10184 Reviewed-on: http://review.gluster.org/10501 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Adhering to the common storage for geo-repKotresh HR2015-05-044-35/+23
| | | | | | | | | | | | | | | | | | | | Making geo-rep use the common storage shared by nfs, snapshot and geo-rep. The meta volume should be named as gluster_shared_storage, and it should be mounted at "/var/run/gluster/shared_storage/". Geo-rep will create a directory called 'geo-rep' in the meta-volume and all the lock files are created inside it. BUG: 1217939 Change-Id: I1d88798376d68340e2b2eff018c7e4f0121a608a Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10196 Reviewed-on: http://review.gluster.org/10503 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* glusterd: volume info should not show "feature.scrub: resume" if scrub resumedGaurav Kumar Garg2015-05-042-1/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | If bitrot is enable on the volume and if user paused the scrub process and then resume the scrub process then command #gluster volume info <VOLNAME> should show status of option features.scrub: Active. If bitrot is enable on the volume and user disable the bitrot on the volume then command #gluster volume info <VOLNAME> should show status of option features.scrub: Inactive. If bitrot is enable on the volume and user paused the scrub porcess then command gluster volume info <VOLNAME> should show the status of option features.scrub: pause. Change-Id: I55972eef3b8570b7cb05dc28700d4e28dc45a86a BUG: 1218036 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10207 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> (cherry picked from commit 6faf89f4179c452be20f02966b9722641938599d) Reviewed-on: http://review.gluster.org/10482 Tested-by: NetBSD Build System Reviewed-by: Kaushal M <kaushal@redhat.com>
* arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvolRavishankar N2015-05-038-14/+111
| | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/10257 Logic for adding the 'glusterd_brickinfo->group' member and using it to find the brick positon has been taken from http://review.gluster.org/#/c/9919. Thanks to Jeff Darcy for that. This patch is a part of the arbiter logic implementation for 3 way AFR details of which can be found at http://review.gluster.org/#/c/9656/ Change-Id: Idbfe4f29ee8e098e0102def8f38b32314316b188 BUG: 1217689 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/10479 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* nfs: make it possible to disable nfs.mount-rmtabNiels de Vos2015-05-035-47/+145
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When there are many NFS-clients doing very often mount/unmount actions, the updating of the 'rmtab' can become a bottleneck and cause delays. In these situations, the output of 'showmount' may be less important than the responsiveness of the (un)mounting. By setting 'nfs.mount-rmtab' to the value "/-", the cache file is not updated anymore, and the entries are only kept in memory. Cherry picked from commit 331ef6e1a86bfc0a93f8a9dec6ad35c417873849: > BUG: 1169317 > Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d > Reported-by: Cyril Peponnet <cyril@peponnet.fr> > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/9223 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: soumya k <skoduri@redhat.com> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> This change also contains the fixes to the test-case from: > > nfs: fix spurious failure in bug-1166862.t > > In some environments, "showmount" could return an NFS-client that does > not start with "1". This would cause the test-case to fail. The check is > incorrect, the number of lines should get counted instead. > > Also moving the test-case to the .../nfs/... subdirectory. > > Cherry picked from commit ee9b35a780607daddc2832b9af5ed6bf414aebc0: > BUG: 1166862 > Change-Id: Ic03aa8145ca57d78aea01564466e924b03bb302a > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/10419 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> > Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d BUG: 1215385 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10379 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Take hole size into account while computing ia_sizeKrutika Dhananjay2015-05-032-2/+10
| | | | | | | | | | | Backport of: http://review.gluster.org/10446 Change-Id: Ic05e07801605c0d610545368a513b56d8df21bf4 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10493 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Add "is-directory" checks in stat/fstatKrutika Dhananjay2015-05-031-0/+12
| | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10427 During mount, NFS directly calls stat on the root of the volume without sending a lookup on it. This was causing inode_ctx_get_block_size() to fail on /. A check is now added in [f]stat which would ensure no action is taken by shard xlator when the operation is on a directory. Change-Id: I8645b7fe58b2d44b5f527d50c1c7102de44acc00 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10509 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Re Enable ignore_deletes OptionAravinda VK2015-05-033-4/+13
| | | | | | | | | | | | | | | | | | | | | | | If this option is set, Deletes will not be propogated to Slave. This option is applicable for UNLINK and RMDIR. gluster volume geo-replication <MASTER> <SLAVEHOST>::<SLAVEVOL> \ config ignore_deletes true Default value is false. PS: Use this option with caution, If you create the file in master with same path then it fails to sync to slave. Old file in Slave will have different GFID compared to New. BUG: 1217929 Change-Id: I1f7816d1ea36460a654873739d3fb1b6c13e0f8d Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9583 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10498 Tested-by: NetBSD Build System
* geo-rep: Do not fail-back to xsync if Changelog is failedAravinda VK2015-05-031-25/+16
| | | | | | | | | | | | | | | | | | Unless change_detector is set to xsync, do not fallback to xsync, except during Initial Sync or Partial History. When a brick goes down, Changelog exception is raised due to which geo-rep fallback to xsync. Even after brick comes back geo-rep will not consume Changelog. BUG: 1217928 Change-Id: I1f8ea26ac7735f6ee09b3b143ee3eb66bfc9fc37 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9758 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10496
* tools/glusterfind: Do not fail to list if Session is invalidAravinda VK2015-05-031-1/+7
| | | | | | | | | | | | If session is invalid, Show the session time as "Session Corrupted" BUG: 1217927 Change-Id: I10f7fb79f6a3a2cabab3f21bce86f2a2f424e8b6 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10164 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10497 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* nfs: Enable throttling for rpcsvcVijay Bellur2015-05-031-0/+7
| | | | | | | | | | | | | | | | Busy nfs servers need throttling to prevent duplicate requests from casuing an increase in memory consumption. Change-Id: I855bb1e841ba06b523e55f86f7788a0f7a49d444 BUG: 1216310 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10269 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/10444 Tested-by: NetBSD Build System
* rpc: Perform throttling conditionallyVijay Bellur2015-05-031-6/+19
| | | | | | | | | | | | | | | | This change makes rpc's throttling to be performed only if attribute throttle is set in rpcsvc_t. Change-Id: I24620095570e206f5dc8fc6208fcf55cb22a1658 BUG: 1216310 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10268 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/10443 Tested-by: NetBSD Build System
generated by cgit (git 2.34.1) at 2024-06-23 21:41:57 +0000