summaryrefslogtreecommitdiffstats
path: root/xlators
Commit message (Collapse)AuthorAgeFilesLines
* quota: support for inode quota in quota.confvmallika2015-05-043-111/+180
| | | | | | | | | | | | | | | | | Currently when quota limit is set, corresponding gfid is set in quota.conf. This patch supports storing inode-quota limits in quota.conf and also stores additional byte for each gfid to differentiate between usage quota limit and inode quota limit. Change-Id: I444d7399407594edd280e640681679a784d4c46a BUG: 1202244 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10069 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glupy: package glupy as a subpackage under gluster namespaceHumble Devassy Chirammal2015-05-042-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently glupy files resides in gluster namespace of python site packages. The other projects like libgfapi-python ..etc are evolving and need to share the gluster namespace. The current structure makes things difficult as all subpackages have its own __init__ files and other files. One subpackage can not any more own gluster namespace. The attempt is to make below structure for gluster namespace so that it is more portable and scalable for future use. <sitepackages>/gluster/ | -- __init__.py | | -- glupy | -- __init__.py -- glupy.py -- ........ | | -- gfapi | -- __init__.py -- gfapi.py -- ........ By above structure clients can import: >>> from gluster import glupy >>> from gluster import gfapi libgfapi-python project has been moved to this structure via http://review.gluster.org/#/c/9668/ Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Change-Id: I54886200ddb6a4153a74d9e187aeca7cad79ef9e BUG: 1211900 Reviewed-on: http://review.gluster.org/10248 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* geo-rep: Fix ignoring geo-rep safe errorsKotresh HR2015-05-042-8/+11
| | | | | | | | | | | | | | | | Fix ignoring geo-rep safe errors in fuse layer and also ignore logging in client translator for mknod. Though it is rare, to happen with mknod, it might happen with history crawl on overlapping changelogs replay. Change-Id: I7e145cd1dc53f04d444ad2e68e66e648be448e61 BUG: 1210562 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10422 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* glusterd: gluster volume status should show status of bitrot and scrubber daemonGaurav Kumar Garg2015-05-044-0/+128
| | | | | | | | | | | | | | | | | | | | | | | | Command gluster volume status <VOLNAME> should show the status of bitrot and scrubber daemon and its pid information. Along with displaying bitrot and scrubber daemon information in gluster volume status command there should be command to show its individual status separately. Command to show individual status of bitrot and scrubber daemon will following. command to show only bitd daemon information will be gluster volume status <VOLNAME> bitd command to show only scrubber daemon information gluster volume status <VOLNAME> scrub Change-Id: Id86aae1156c8c599347c98e2a538f294d37376e4 BUG: 1209752 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10175 Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Kaushal M <kaushal@redhat.com>
* dht: Add a null check before freeing dir_dfmeta and tmp_containerSusant Palai2015-05-041-10/+19
| | | | | | | | | | | Change-Id: Ifbba6f340adfe2b4e3ad07260fbf4a25698ad8df BUG: 1217949 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/10459 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* tier: relax libgfdb required version numberEmmanuel Dreyfus2015-05-031-1/+1
| | | | | | | | | | | | | | | | | | | When calling dlopen() for libgfdb, do not specify the library version number "libgfdb.so.0.0.1", since libtool will not always create libraries or link with that name with the full 3-digit version. For instance on NetBSD only up to the 2-digit version is available and "libgfdb.so.0.0.1" does not exist. Instead, just specify "libgfdb.so" and rely on smymlinks installed by libtool to find the relevant library. BUG: 1129939 Change-Id: I074b1009d3622a122fdaeb4b99658bca3277e211 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10407 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/bitrot: Per volume bitrot translatorGaurav Kumar Garg2015-05-036-154/+349
| | | | | | | | | | | | | | | | | | | | Currently whatever bitrot/scrubber tunable value user set for one volume that value is considering for all other volumes also. Each volume should act on their respective bitrot/scrubber tunable value. For handling bitrot/scrubber tunable value independently with respect to all the volume bitrot and scrubber translator should run seperatly for each volume. Change-Id: I1d9379508afe6cfd2f78e3ebf29c829c362d84a9 BUG: 1170075 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10352 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Kaushal M <kaushal@redhat.com>
* features/shard: Add "is-directory" checks in stat/fstatKrutika Dhananjay2015-05-031-0/+12
| | | | | | | | | | | | | | | | During mount, NFS directly calls stat on the root of the volume without sending a lookup on it. This was causing inode_ctx_get_block_size() to fail on /. A check is now added in [f]stat which would ensure no action is taken by shard xlator when the operation is on a directory. Change-Id: I81849eeddfdad9f271155442408d95b4a25d7647 BUG: 1207615 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10427 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Upcall: Handle missing fops in the upcall xlatorSoumya Koduri2015-05-034-130/+969
| | | | | | | | | | Change-Id: I968980dc4df458ec427e33503363bbd017e1163e BUG: 1200271 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/10194 Tested-by: NetBSD Build System Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* features/changelog: Consider only changelog on/off as changelog breakageKotresh HR2015-05-024-11/+211
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, both chagelog on/off and brick restart were considered to be changelog breakage and treated as changelog not being continuous. As a result, new HTIME.TSTAMP file was created on both the above cases. Now the change is made such that only on changelog enable/disable, the changelog is considered to be discontinuous. New HTIME.TSTAMP file is not created on brick restart, the changelogs files are appended to last HTIME.TSTAMP file. Treating changelog as continuous in above scenario is important as changelog history API will fail otherwise. It can successfully get changes between start and end timestamps only when changelog is continuous (Changelogs in single HTIME.TSTAMP file are treated as continuous). Without this change, changelog history API would fail, and it would become necessary to fallback to other mechanisms like xsync FSCrawl in case geo-rep to detect changes in this time window. But Xsync FSCrawl would not be applicable to other consumers like glusterfind. Rationale: 1. In plain distributed volume, if brick goes down, no I/O can happen onto the brick. Hence changelog is intact with data on disk. 2. In distributed replicate volume, if brick goes down, since self-heal traffic is captured in changelog. Eventually, I/O happened whend brick down is captured in changelog. Change-Id: I2eb66efe6ee9a9228fb1fcb38d6e7696b9559d5b BUG: 1211327 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10222 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* features/shard: Take hole size into account while computing ia_sizeKrutika Dhananjay2015-05-022-2/+10
| | | | | | | | | | Change-Id: I1a90ad6669c1cb79aaae6b4bd9621c75d9985c8a BUG: 1207615 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10446 Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* protocol: increase default group-cache-timeout to 300 secondsNiels de Vos2015-05-023-3/+3
| | | | | | | | | | | | | sssd uses 300 seconds by default too. There is no need to overload sssd with requests that it would have cached. BUG: 1215187 Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10371 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ctr/libgfdb: Performance enhancer for unlink/create/rename/link fopsJoseph Fernandes2015-05-015-1/+25
| | | | | | | | | | | | | | | | | | | | | | | | 1) ctr_link_consistency option for ctr xaltor is provided so that the user can choose to switch it on or off. /* For link consistency we do a double update i.e mark the link * during the wind and during the unwind we update/delete the link. * This has a performance hit. We give a choice here whether we need * link consistency to be spoton or not using link_consistency flag. * This will have only one link update */ 2) In delete the wind time recording is moved to unwind path. /* Special performance case: * Updating wind time in unwind for delete. This is done here * as in the wind path we will not know whether its the last * link or not. For a last link there is not use to update any * wind or unwind time!*/ Change-Id: I209472fb816f939db4a868b97ba053b028f17ea6 BUG: 1217786 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10170 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* glusterd/quota/tiering: Fixing volgen of quotadJoseph Fernandes2015-05-011-4/+10
| | | | | | | | | | | | | The quotad's graph generation was happening wrongly for tiered volume. The check is been inserted. Change-Id: I5554bc5280b0fbaec750e9008fdd930ad53a774f BUG: 1214219 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10474 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht/rebalancer: Marking tiering migration fopsJoseph Fernandes2015-05-013-6/+34
| | | | | | | | | | | | | | | | | | | | | | | | | This is a follow up patch for http://review.gluster.org/#/c/10080 In the above, the suggested change in http://review.gluster.org/#/c/10080/7/xlators/cluster/dht/src/dht-rebalance.c doesnot work. The reason it doesnt work is promotion and demotion are done in a multithread way. Whenever a promotion or demotion thread is called, the frame of the old sync_op thread is not carried with it. As a result the frame->root->pid is not set. Solution: When the file is getting migrated, we get a tiering.migration key_value in the xattr dict, so that we pass this dic key-value when we do syncop_setxattr() to do data migration and set the frame->root->pid GF_CLIENT_PID_TIER_DEFRAG in dht_setxattr() just before calling dht_start_rebalance_task(). Change-Id: I86fef2d961b32fdd2c0c69d8512cbe846b393404 BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10266 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Susant Palai <spalai@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* changelog: Fixing buffer overrun coverity issues.Nandaja Varma2015-04-302-7/+7
| | | | | | | | | | | | | | | | Coverity IDs: 1214630 1214631 1214633 1234643 Change-Id: I172c4f49bf651b2324522f9e661023f73ca05339 BUG: 789278 Signed-off-by: Nandaja Varma <nvarma@redhat.com> Reviewed-on: http://review.gluster.org/9557 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Sakshi Bansal Reviewed-by: Venky Shankar <vshankar@redhat.com>
* Upcall: Replaced opaque gfid with string in rpc cbk argsSoumya Koduri2015-04-302-4/+3
| | | | | | | | | | | | | | For GF_CBK_CACHE_INVALIDATION, have changed the type of gfid to be string (cannonical form) instead of opaque byte format to ensure correctness across platforms supporting different endianness. BUG: 1200262 Change-Id: Iac4372714f4b4ebcd9c4393aaf46ceba3f37f587 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/10224 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Upcall: Process each of the upcall events separatelySoumya Koduri2015-04-306-69/+102
| | | | | | | | | | | | | | | | As suggested during the code-review of Bug1200262, have modified GF_CBK_UPCALL to be exlusively GF_CBK_CACHE_INVALIDATION. Thus, for any new upcall event, a new CBK procedure will be added. Also made changes to store upcall data separately based on the upcall event type received. BUG: 1200262 Change-Id: I0f5e53d6f5ece16aecb514a0a426dca40fa1c755 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/10049 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* NFS-Ganesha: Handling CLI commands when NFS-Ganesha keys are setMeghana Madhusudhan2015-04-307-74/+238
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When ganesha.enable is set to on and features.ganesha is enabled, there are a few behaviour changes that should be seen in other volume operations. 1. ganesha.enable can be set to 'on' only when features.ganesha is set to 'enable' 2.When gluster vol is started, and if ganesha.enable key was set to 'on', it should automatically export the volume via NFS-Ganesha. 3.When ganesha.enable is set to 'on', and a volume is stopped, that volume should be unexported via NFS-Ganesha. 4. gluster vol reset <volname> If ganesha.enable was set to on, then unexport the volume via NFS-Ganesha. 5. gluster vol reset all If features.ganesha is set to enable, as part of reset all, set it to disable. This translates to teardown cluster. All the above problems are fixed by checking the global key and value, depending on the value, specific functions are called. And also, functions related to global commands are moved to cli-cmd-global.c Commit phase of features.ganesha enable/disable runs the ganesha-ha.sh setup/teardown respectively. Before the script begins, it is important that the NFS-Ganesha service starts on all the HA nodes. Having the start service commands in the commit phase could lead to problems. Moving the pre-requisite service start commands to the 'stage' phase. Change-Id: I5a256f94f8e1310ddcd5369f329b7168b2a24c47 BUG: 1200265 Signed-off-by: Meghana Madhusudhan <mmadhusu@redhat.com> Reviewed-on: http://review.gluster.org/10283 Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* features/bitrot: Use global timer wheelVenky Shankar2015-04-302-24/+25
| | | | | | | | | | | Change-Id: I761927ea263b4144b851881f25791fda5b794f59 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10381 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* bitrot: Scrubber log should report 'bad' file detection as ALERT in logGaurav Kumar Garg2015-04-301-2/+2
| | | | | | | | | | | | If scrubber detect any bad object by mismatching of checksum of scrubber and signer then log messages shold come as a Alert instead of warning. Change-Id: I075d80700cbe6182e525a04419a80ab18419ff91 BUG: 1210687 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10226 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* quota: Validate NULL inode from the entries received in readdirp_cbkvmallika2015-04-302-6/+13
| | | | | | | | | | | | | | | | | | | In quota readdirp_cbk, inode ctx filled for the all entries received. In marker readdirp_cbk, files/directories are inspected for dirty There is no guarantee that entry->inode is populated. If entry->inode is NULL, this needs to be treated as readdir Change-Id: Id2d17bb89e4770845ce1f13d73abc2b3c5826c06 BUG: 1215550 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/10416 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: do not pass volinfo in glusterd_svc_manager functionGaurav Kumar Garg2015-04-306-56/+42
| | | | | | | | | | | | | | | | | On restarting of glusterd first it will start all the bricks present in the volume then it will start all the services. During starting of all the services it may pass volinfo as a NULL. It will cause Assert failure in glusterd_bitdsvc_manager function and will cause a glusterd crash. Change-Id: Ia14cf5022da88516cdd576eb2d1e0e7b17a3782b BUG: 1207029 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10241 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* glusterd: volume info should not show "feature.scrub: resume" if scrub resumedGaurav Kumar Garg2015-04-301-1/+24
| | | | | | | | | | | | | | | | | | | | | | If bitrot is enable on the volume and if user paused the scrub process and then resume the scrub process then command #gluster volume info <VOLNAME> should show status of option features.scrub: Active. If bitrot is enable on the volume and user disable the bitrot on the volume then command #gluster volume info <VOLNAME> should show status of option features.scrub: Inactive. If bitrot is enable on the volume and user paused the scrub porcess then command gluster volume info <VOLNAME> should show the status of option features.scrub: pause. Change-Id: I55972eef3b8570b7cb05dc28700d4e28dc45a86a BUG: 1209818 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/10207 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
* glusterd: Use uint32_t for peerinfo generation numberKaushal M2015-04-304-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Using a uint64_t for the peerinfo generation number was overkill for how the generation number is used within GlusterD. It also prevented GlusterD from running on 32-bit architechtures, as uatomic_add_return doesn't support 64-bit values on 32-bit architechtures. This change was developed on the git branch at [1]. This commit is a combination of the following commits on the development branch. b78dba4 Use 32-bit generation number 2c37e4b Change other generation number variables to uint32_t [1]: https://github.com/kshlm/glusterfs/tree/urcu Change-Id: I0f310f56a4fb97d6bcbc23255a379ed5bb1ed9e1 BUG: 1205186 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/10425 Reviewed-by: Anand Nekkunti <anekkunt@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org> Tested-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* protocol/server : port log messages to new framework.Manikandan Selvaganesh2015-04-298-293/+1202
| | | | | | | | | | Change-Id: I7901f55d06716161cc31d2b79a600a16b5ec2ef8 BUG: 1194640 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/9874 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* features/shard: Implement rename() fopKrutika Dhananjay2015-04-292-44/+261
| | | | | | | | | | | Change-Id: Ia39fa0f29eac84c18d13a94f704b1703565b504e BUG: 1207615 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10373 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* dht: tackle thread race in dht_getxattr_cbkSusant Palai2015-04-291-17/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | problem: 1. When two threads execute in parallel in dht_getxattr_cbk it may so happen that, both may find local->xattr to be NULL. As a result dht_aggregate_xattr may not get executed. 2. In dht_getxattr_cbk, thread1 thread2 T1 this_call_cnt = 2 -1 T2 this_call_cnt = 1 - 1 T3 fills local_xattr T4 DHT_STACK_UNWIND -> local_wipe T5 tries to dereference local which is already freed, leading to crash. Solution: for problem1: Execute critical section inside frame lock to resolve race. for problem2: Calculate this_call_count just before out section. Change-Id: I9827ac8fafebb0c733a4e4f3c710b752f1cd45fa BUG: 1215592 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/10389 Reviewed-by: Anuradha Talur <atalur@redhat.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* rebalance: Introducing local crawl and parallel migrationSusant Palai2015-04-296-248/+1121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current patch address two part of the design proposed. 1. Rebalance multiple files in parallel 2. Crawl only bricks that belong to the current node Brief design explanation for the above two points. 1. Rebalance multiple files in parallel: ------------------------------------- The existing rebalance engine is single threaded. Hence, introduced multiple threads which will be running parallel to the crawler. The current rebalance migration is converted to a "Producer-Consumer" frame work. Where Producer is : Crawler Consumer is : Migrating Threads Crawler: Crawler is the main thread. The job of the crawler is now limited to fix-layout of each directory and add the files which are eligible for the migration to a global queue in a round robin manner so that we will use all the disk resources efficiently. Hence, the crawler will not be "blocked" by migration process. Producer: Producer will monitor the global queue. If any file is added to this queue, it will dqueue that entry and migrate the file. Currently 20 migration threads are spawned at the beginning of the rebalance process. Hence, multiple file migration happens in parallel. 2. Crawl only bricks that belong to the current node: -------------------------------------------------- As rebalance process is spawned per node, it migrates only the files that belongs to it's own node for the sake of load balancing. But it also reads entries from the whole cluster, which is not necessary as readdir hits other nodes. New Design: As part of the new design the rebalancer decides the subvols that are local to the rebalancer node by checking the node-uuid of root directory prior to the crawler starts. Hence, readdir won't hit the whole cluster as it has already the context of local subvols and also node-uuid request for each file can be avoided. This makes the rebalance process "more scalable". Change-Id: I73ed6ff807adea15086eabbb8d9883e88571ebc1 BUG: 1171954 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/9657 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* nfs/auth,exports,netgroup : port log messages to new frameworkManikandan Selvaganesh2015-04-283-45/+70
| | | | | | | | | | | | | | | Five gf_log messages are not changed to gf_msg because gf_msg not sending the output to STDOUT. Bug id : BZ1215017 Change-Id: Ie8de286355becc3fd73fa80057734b834cf40b04 BUG: 1194640 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/10173 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: NetBSD Build System Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* nfs: Enable throttling for rpcsvcVijay Bellur2015-04-281-0/+7
| | | | | | | | | | | | | | | Busy nfs servers need throttling to prevent duplicate requests from casuing an increase in memory consumption. Change-Id: I855bb1e841ba06b523e55f86f7788a0f7a49d444 BUG: 1212385 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10269 Tested-by: NetBSD Build System Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* nfs / acl3.c, nlm files : port log messages to new frameworkManikandan Selvaganesh2015-04-283-155/+277
| | | | | | | | | | | | | Porting log messages to a new framework for acl3.c, nlm4.c, nlmcbk_svc.c Change-Id: Iee4daae4fe1460eddf95d285615335b836d7f958 BUG: 1194640 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/10217 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* nfs / nfs files : porting log messages to new frameworkManikandan Selvaganesh2015-04-286-163/+245
| | | | | | | | | | | | | Porting log messages to a new framework for nfs-common.c, nfs-fops.c, nfs-fops.h, nfs-inodes.c, nfs3-fh.c, nfs3-helpers.c. Change-Id: I55cd94de62eafe6ad3754cd065d7459a81a9f285 BUG: 1194640 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/10214 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* nfs / mount files : porting log messages to a new frameworkManikandan Selvaganesh2015-04-283-234/+339
| | | | | | | | | | | | | Porting log messages to a new framework for mount3udp_svc.c, mount3.c, mount3-auth.c files. Change-Id: I1e9cab500b6052bfac43c5ae1c82278e20d2f898 BUG: 1194640 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/10210 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* cluster/ec: Handle unhandled statesPranith Kumar K2015-04-281-0/+3
| | | | | | | | | | | | This was leading to hangs when get_size_and_version fails Change-Id: Iad9408c2dacc9a74594b8d2f94c95f402533b0f1 BUG: 1215265 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10390 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
* libglusterfs: Implementation of sync lock as recursive lock to avoid crash.anand2015-04-285-33/+5
| | | | | | | | | | | | | | | | | | | | | | | Problem : In glusterd,we are using big lock which is implemented based on sync task frame work for thread synchronization and rcu lock for data consistency. sync task frame work swap the threads if there is no worker poll threads available,due to this rcu lock and rcu unlock was happening in different threads (urcu-bp will not allow this),resulting into glusterd crash. fix : To avoid releasing the sync lock(big lock) in between rcu critical section,implemented sync lock as recursive lock. More details: link : http://www.spinics.net/lists/gluster-devel/msg14632.html Change-Id: I2b56c1caf3f0470f219b1adcaf62cce29cdc6b88 BUG: 1211640 Signed-off-by: anand <anekkunt@redhat.com> Reviewed-on: http://review.gluster.org/10285 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* arbiter: load arbiter xlator on every 3rd brick of a replica 3 AFR subvolRavishankar N2015-04-277-14/+97
| | | | | | | | | | | | | | | | | | | Logic for adding the 'glusterd_brickinfo->group' member and using it to find the brick positon has been taken from http://review.gluster.org/#/c/9919. Thanks to Jeff Darcy for that. This patch is a part of the arbiter logic implementation for 3 way AFR details of which can be found at http://review.gluster.org/#/c/9656/ Change-Id: Idbfe4f29ee8e098e0102def8f38b32314316b188 BUG: 1199985 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/10257 Tested-by: NetBSD Build System Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* quota: display error when inode-quota cmds executed with cluster ver < 3.7vmallika2015-04-271-23/+44
| | | | | | | | | | | | | | | | | | | In a heterogeneous cluster with op_version less than 3.7, inode quotas will be accounted on those bricks which has glusterfs version 3.7 and this is not available in older version. This will have incorrect values displayed when user queries inode count from CLI. This patch will display error when inode-quota commands are executed with cluster version less than 3.7 Change-Id: Ia0e6d5635d1d8e7b2e2cfc3daa7b7f9e314a263a BUG: 1212253 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/10261 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tier: fix off-by-one overrun in UUID stringEmmanuel Dreyfus2015-04-271-1/+1
| | | | | | | | | | | | | | | | | UUID strings are UUID_CANONICAL_FORM_LEN (36) bytes long plus the trailing nul character that various function (e.g.: uuid_unparse) will add. As a consequence, UUID strings must be declared as UUID_CANONICAL_FORM_LEN+1 long, otherwise we get a off-by-one overrun that corrupts the next variable on stack. BUG: 1129939 Change-Id: I5837ad6ca06fa17cc7ab143eedd02d8099ecca2a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10394 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* storage/posix: prevent NULL dereferencePranith Kumar K2015-04-271-3/+3
| | | | | | | | | | | | | filler->fd is never set but used. Change-Id: Icf21c439b37c9faa3751658a9e63a74570ed153c BUG: 1215265 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10383 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Re Enable ignore_deletes OptionAravinda VK2015-04-271-2/+6
| | | | | | | | | | | | | | | | | | | | | If this option is set, Deletes will not be propogated to Slave. This option is applicable for UNLINK and RMDIR. gluster volume geo-replication <MASTER> <SLAVEHOST>::<SLAVEVOL> \ config ignore_deletes true Default value is false. PS: Use this option with caution, If you create the file in master with same path then it fails to sync to slave. Old file in Slave will have different GFID compared to New. BUG: 1189363 Change-Id: I1f7816d1ea36460a654873739d3fb1b6c13e0f8d Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9583 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* Snapshot: New logging framework implementation of snapshot messages.Nandaja Varma2015-04-272-774/+1455
| | | | | | | | | | Change-Id: I3e8b5f6ae59d16752beb8053c2afddfb5c00be14 BUG: 1194640 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/9796 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com>
* glusterd: New logging framework, all the message labelNandaja Varma2015-04-271-1/+3412
| | | | | | | | | Change-Id: I5f83b070638384b3f2e3efe936f64080c3334eab BUG: 1194640 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/10332 Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Kaushal M <kaushal@redhat.com>
* glusterd: Remove direct references to peerinfo in frame cookiesKaushal M2015-04-263-67/+158
| | | | | | | | | | | | | | | | | | | | | | | | | RCU protection requires that we don't have direct references to protected data structures outside read-critical sections This change was developed on the git branch at [1]. This commit is a combination of the following commits on the development branch. 82ebfdd Remove direct references to peerinfo in frame cookies dec4bec Remove incorrect and unneeded code from gd_syncop_mgmt_v3_unlock_cbk_fn 7aced7b Use stack allocated uuid for frame cookie. 38e4124 Address comments from 10192/2 [1]: https://github.com/kshlm/glusterfs/tree/urcu Change-Id: Ic50e5fca0be72af5090f4cf318efa55d29075de9 BUG: 1205186 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/10192 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* glusterd: incorrect rsp.op_ret handling in volume get in cliAtin Mukherjee2015-04-261-1/+0
| | | | | | | | | | Change-Id: Iabe99c06166578fc90121e7cfdca4a6a3f5328ae BUG: 1211132 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/10229 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushal M <kaushal@redhat.com>
* glusterd: initialize snapd svc at volume restore pathAtin Mukherjee2015-04-263-11/+9
| | | | | | | | | | | | | | | | In restore path snapd svc was not initialized because of which any glusterd instance which went down and came back may have uninitialized snapd svc. The reason I used 'may' is because depending on the nodes in the cluster. In a single node cluster this wouldn't be a problem since glusterd_spawn_daemon takes care of initializing it. Change-Id: I2da1e419a0506d3b2742c1cf39a3b9416eb3c305 BUG: 1213295 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/10304 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kaushal M <kaushal@redhat.com>
* bitrot/scrub: fix induced throttling in syncop_ftw_throttle()Venky Shankar2015-04-261-12/+14
| | | | | | | | | | | | | | | | | | | | | | | | Failing to reset scanning counter causes "incorrect" delay of around 50 seconds per directory entry. This causes scrubber to run extremely slowly. [ NOTE: This is a temporary fix. With the introduction of token bucket based throttling, inducing throttle via sleep() call would be unneeded. ] Also, fix logging messages in scrubber to log brick and full path of the object which is identified/marked as corrupted. Change-Id: Id501bd15dcdbd8a09613f80f9d84050304740027 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10375 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
* nfs: make it possible to disable nfs.mount-rmtabNiels de Vos2015-04-262-45/+77
| | | | | | | | | | | | | | | | | | | | When there are many NFS-clients doing very often mount/unmount actions, the updating of the 'rmtab' can become a bottleneck and cause delays. In these situations, the output of 'showmount' may be less important than the responsiveness of the (un)mounting. By setting 'nfs.mount-rmtab' to the value "/-", the cache file is not updated anymore, and the entries are only kept in memory. BUG: 1169317 Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d Reported-by: Cyril Peponnet <cyril@peponnet.fr> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/9223 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* tiering: skip CTR-xlator and config options when disabledNiels de Vos2015-04-262-0/+7
| | | | | | | | | | | | | When compiling with --disable-tiering, GlusterD should not contain any references to the ChangeTimeRecorder xlator or any of the data tiering options. BUG: 1213125 Change-Id: Idb46fb80f0ca8b66115e06841d9ec15ba14c24a0 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10296 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Don't log geo-rep safe errors in mount logsKotresh HR2015-04-252-2/+3
| | | | | | | | | | | | | | | | | | | | EEXIST and ENOENT are safe errors for geo-replication. Since mkdir is captured in all the bricks of the changelog. mkdir is tried multiple times as per the number of bricks. The first one to process by gsyncd will succeed and all others will get EEXIST. Hence EEXIST is a safe error and can be ignored. Similarly ENOENT also in rm -rf case. And also gsyncd validates these errors and log them in master if it is genuine error. This is coming up with the patch http://review.gluster.org/#/c/10048/ Hence ignoring above said safe errors. Change-Id: I10ae86b11d49c7c3ba2be3110dace6b33daa509e BUG: 1210562 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10184 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-21 20:23:17 +0000