summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* core: use reference counting for mem_acct structuresJeff Darcy2015-05-0913-187/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When freeing memory, our memory-accounting code expects to be able to dereference from the (previously) allocated block to its owning translator. However, as we have already found once in option validation and twice in logging, that translator might itself have been freed and the dereference attempt causes on of our daemons to crash with SIGSEGV. This patch attempts to fix that as follows: * We no longer embed a struct mem_acct directly in a struct xlator, but instead allocate it separately. * Allocated memory blocks now contain a pointer to the mem_acct instead of the xlator. * The mem_acct structure contains a reference count, manipulated in both the normal and translator allocate/free code using atomic increments and decrements. * Because it's now a separate structure, we can defer freeing the mem_acct until its reference count reaches zero (either way). * Some unit tests were disabled, because they embedded their own copies of the implementation for what they were supposedly testing. Life's too short to spend time fixing tests that seem designed to impede progress by requiring a certain implementation as well as behavior. Change-Id: Id929b11387927136f78626901729296b6c0d0fd7 BUG: 1219026 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/10417 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10723 Tested-by: NetBSD Build System
* cli/tiering: Enhance cli output for tieringMohammed Rafi KC2015-05-099-20/+394
| | | | | | | | | | | | | | | | | | | | | | | back port of http://review.gluster.org/10284 Fix for handling cli output for attach-tier and detach-tier >Change-Id: I4d17f4b09612754fe1b8cec6c2e14927029b9678 >BUG: 1211562 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/10284 >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System >Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: Ic30c8f0a104d89bf98f5d0069937a34674ee3f2d BUG: 1220052 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10713 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Disable all known bad testsPranith Kumar K2015-05-091-0/+23
| | | | | | | | | Change-Id: I80175e06efc12fd2dab126d6e0143d63caf11209 BUG: 1220059 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10726 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* cli/tiering : Parse detach-tier command properlyMohammed Rafi KC2015-05-092-5/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | Back port of http://review.gluster.org/10303 gluster detach-tier help should show the usage of detach-tier command properly. Also, instead of force option, if any other was given, then the command should fail with a usage message >Change-Id: Ie801529176db067c98fd1bfada056cbc647973fb >BUG: 1211570 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/10303 >Tested-by: NetBSD Build System >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I51ef7c3450c219b2b24724592270461570436010 BUG: 1220050 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10709 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glusterd: support for tier volumes 'detach start' and 'detach commit'Dan Lambright2015-05-0918-75/+259
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Back port of http://review.gluster.org/10108 These commands work in a manner analagous to rebalancing when removing a brick. The existing migration daemon detects "detach start" and switches to moving data off the hot tier. While in this state all lookups are directed to the cold tier. gluster v detach-tier <vol> start gluster v detach-tier <vol> commit The status and stop cli commands shall be submitted separately. >Change-Id: I24fda5cc3ba74f5fb8aa9a3234ad51f18b80a8a0 >BUG: 1205540 >Signed-off-by: Dan Lambright <dlambrig@redhat.com> >Signed-off-by: root <root@localhost.localdomain> >Signed-off-by: Dan Lambright <dlambrig@redhat.com> >Reviewed-on: http://review.gluster.org/10108 >Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Change-Id: I212d748d077fb5870ee84b316c653acbafbea3f7 BUG: 1220047 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10708 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* libgfapi: Store and restore THIS in every API exposed by libgfapiPoornima G2015-05-096-146/+591
| | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/9797 Storing and restoring THIS: When the APIs exposed by libgfapi are called by other xlators like snapview server etc. the THIS value is overwritten to contain the THIS of libgfapi(viz libgfapi master xlator). Hence using 'THIS' in any xlator after calling libgfapi API will lead to issues. One such issue was uncovered in snapview and the patch http://review.gluster.org/#/c/9469/ was sent to workaround this issue. Hence, storing and restoring THIS, at the entry and exit of every API exposed by libgfapi. Change-Id: I6f330dde25e7700fb26339d667a7ccd193ec6ba0 BUG: 1210934 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/9797 Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> (cherry picked from commit 1162bb36108ab8dba8303b86927a99835b791d79) Reviewed-on: http://review.gluster.org/10414 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glupy: package glupy as a subpackage under gluster namespaceNiels de Vos2015-05-092-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently glupy files resides in gluster namespace of python site packages. The other projects like libgfapi-python ..etc are evolving and need to share the gluster namespace. The current structure makes things difficult as all subpackages have its own __init__ files and other files. One subpackage can not any more own gluster namespace. The attempt is to make below structure for gluster namespace so that it is more portable and scalable for future use. <sitepackages>/gluster/ | -- __init__.py | | -- glupy | -- __init__.py -- glupy.py -- ........ | | -- gfapi | -- __init__.py -- gfapi.py -- ........ By above structure clients can import: >>> from gluster import glupy >>> from gluster import gfapi libgfapi-python project has been moved to this structure via http://review.gluster.org/#/c/9668/ Cherry picked from commit 40df2ed4d098d4cd2c6abbed23e497ac3e2e5804: > Change-Id: I54886200ddb6a4153a74d9e187aeca7cad79ef9e > BUG: 1211900 > Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> > Reviewed-on: http://review.gluster.org/10248 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> This backport is really minimal, because commit 44036808 removed the need for the changes in the glusterfs.spec.in file. Change-Id: I54886200ddb6a4153a74d9e187aeca7cad79ef9e BUG: 1220022 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10706 Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/dht: change log level of developer logs to DEBUGVijay Bellur2015-05-091-2/+2
| | | | | | | | | | | | | | | | | | Backport of : http://review.gluster.org/10281 A few log messages in dht directory self heal at log level INFO are useful only for developers and these logs tend to casue excessive logs in our log files. Hence moving the log level of such logs to DEBUG. Change-Id: I8a543f4ddeb5c20b2978a0f7b18d8baccc935a54 BUG: 1217949 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10281 Reviewed-by: N Balachandran <nbalacha@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-on: http://review.gluster.org/10704 Tested-by: NetBSD Build System Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
* tools/glusterfind: Partial FindAravinda VK2015-05-098-289/+367
| | | | | | | | | | | | | | | | | | | | | | | | | This is optional and enabled by default, if one node fails Glusterfind will not fail to return list of files from other nodes. This behavior can be changed using --disable-partial Now session is maintained in each nodes as well as in initiator node. Every pre command will pick the status file from respective node and start collecting list of changes happened after the status time. --reset-session-time, new option to force reset the session time. Next incremental run will start from this time. Change-detector argument is removed since Changelog mode is required to detect deletes and Renames. BUG: 1219467 Change-Id: I1d0a0629facc3d26780200ccdf39b221ab4037c4 Original-Author: Aravinda VK <avishwan@redhat.com> Reviewed-On: http://review.gluster.org/#/c/10320 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10639 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix corrupt gsyncd outputAravinda VK2015-05-092-5/+5
| | | | | | | | | | | | | | | When gsyncd fails with Python traceback, glusterd fails parsing gsyncd output and shows error. BUG: 1219938 Change-Id: Ic32fd897c49a5325294a6588351b539c6e124338 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10694 Reviewed-on: http://review.gluster.org/10695 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests : workaround fix for volume-snapshot-clone spurious failureAtin Mukherjee2015-05-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10619/ Snippet of the existing test: TEST kill_glusterd 2; TEST $glusterd_2; sleep 5 EXPECT_WITHIN $PROCESS_UP_TIMEOUT 'Started' volinfo_field ${V0}_clone 'Status'; EXPECT_WITHIN $PROCESS_UP_TIMEOUT 'Started' volinfo_field ${V1}_clone 'Status'; stop_force_volumes 2 The above EXPECT_WITHINs will not wait as the volumes were never stopped which results into immediate trigger of volume stop where handshaking is not completed. A workaround would be to check whether handshaking is completed and then proceed. Change-Id: Id3468b56f801212fc8e19ff196bc99e5e2358471 BUG: 1220012 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/10619 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Justin Clift <justin@gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> (cherry picked from commit 0950e6c29fc51ddcc5bb7e1e4187d54ff7c171d2) Reviewed-on: http://review.gluster.org/10700 Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: rename handling in dht volume(changelog changes)Saravanakumar Arumugam2015-05-091-38/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: Glusterfs changelogs are stored in each brick, which records the changes happened in that brick. Georep will run in all the nodes of master and processes changelogs "independently". Processing changelogs is in brick level, but all the fops will be replayed on "slave mount" point. Problem: With a DHT volume, in changelog "internal fops" are NOT recorded. For Rename case, Rename is recorded in "hashed" brick changelog. (DHT's internal fops like creating linkto file, unlink is NOT recorded). This lead us to inconsistent rename operations. For example, Distribute volume created with Two bricks B1, B2. //Consider master volume mounted @ /mnt/master and following operations executed: cd /mnt/master touch f1 // f1 falls on B1 Hash mv f1 f2 // f2 falls on B2 Hash // Here, Changelogs are recorded as below: @B1 CREATE f1 @B2 RENAME f1 f2 Here, race exists between Brick B1 and B2, say B2 will get executed first. Source file f1 itself is "NOT PRESENT", so it will go ahead and create f2 (Current implementation). We have this problem When rename falls in another brick and file is unlinked in Master. Similar kind of issue exists in following case too(multiple rename): CREATE f1 RENAME f1 f2 RENAME f2 f1 Solution: Instead of carrying out "changelogging" at "HASHED volume", carry out at the "CACHED volume". This way we have rename operations carried out where actual files are present. So,Changelog recorded as : @B1 CREATE f1 RENAME f1 f2 Note: This patch is dependent on dht changes from this patch. http://review.gluster.org/10410/ changelog related changes are separated out for review. In changelog, xdata passed from DHT is considered as: 1. In case of unlink (internal operation as part of rename), xdata value is set , it is considered as RENAME and recorded accordingly. 2. In case of rename (Hash and Cache different), xdata value is NOT set, recording rename operation is SKIPPED. BUG: 1219412 Change-Id: I7691166c84991482b2cfe073df64e2317c935b13 Reviewed-On: http://review.gluster.org/#/c/10220/ Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/10633 Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/afr : Prevent inode-evict during split-brain resolutionAnuradha2015-05-098-58/+316
| | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/10134/ 1) Provided setfattr command to set timeout for split-brain choice. 2) If split-brain inspection/resolution is being done from the mount for a file, ref the inode when split-brain-choice is set. This inode will be unconditionally unref-ed after timeout seconds set by the user/default otherwise. 3) Updated the doc and testcase to reflect the changes. Change-Id: I15c9037dee28855f21e680e7e3632e1f48dba4e1 BUG: 1219388 Reviewed-on: http://review.gluster.org/10134 Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Signed-off-by: Anuradha <atalur@redhat.com> Reviewed-on: http://review.gluster.org/10679
* cli: Error out when there is not enough memoryHumble Devassy Chirammal2015-05-091-1/+5
| | | | | | | | | | | | | | Change-Id: Ia8d061de5be1343cc10a945f6cf011686a770d33 BUG: 1220020 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/10144 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com> (cherry picked from commit f6e24fc54453669f3943e50f44cce9bb6070a71a) Reviewed-on: http://review.gluster.org/10674 Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Kaushal M <kaushal@redhat.com>
* tests: Spurious failure in fop-sanity.tNithya Balachandran2015-05-093-18/+21
| | | | | | | | | | | | | | | | | | | | | | Modified the calls to open in fops-sanity.c to pass in the mode as well if flags includes O_CREAT (as per man page). The missing mode randomly caused T files to be created causing DHT to treat them as linkto files and fail the fop. Modified 2 other files where the mode was not being provided. Change-Id: I047573d43655b4957d0703f7df36238f7e729c1f BUG: 1218940 Signed-off-by: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/10590 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit a661f7f54cef34aa39894818568a2c1b462e8cbc) Signed-off-by: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/10627 Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tiering: Correction to tiering documentationJoseph Fernandes2015-05-091-0/+168
| | | | | | | | | | | | | | | | 1) convert to md format 2) Add info about ctr and libgfdb Change-Id: I531d8a0bff8195f759302c5e613c7af2113729eb BUG: 1218638 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10665 Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com> (cherry picked from commit 9c21c5a632ba22a6f46d037bd4fa4d825b24d07f) Reviewed-on: http://review.gluster.org/10669 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* rpm: reduce package dependencies and add -client-xlatorsNiels de Vos2015-05-081-48/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Restructuring the RPM packages in order to reduce the dependencies that get installed for glusterfs-api (mainly on request for Qemu hosts). The dependencies of these packages look roughly like this: .------------------. | glusterfs-server | '-------+----------' |\ | \ | '---------------+--------------------. | | | | v v | .----------------. .---------------. | | glusterfs-fuse | | glusterfs-api | | '--------+-------' '------+--------' | /| /| | / | / | | / | / | | / | / | | .-----------+----=---------------' | |/ | | v v v .-----------. .--------------------------. | glusterfs | | glusterfs-client-xlators | '-----------' '--------------------------' With this structure, users can install glusterfs-server, glusterfs-fuse or glusterfs-api (libgfapi) without getting any unneeded xlators or other scripts/binaries. The "glusterfs" and "glusterfsd" binary (symlinked) is now part of the glusterfs-fuse package (moved from glusterfs). This does not make a difference for glusterfs-server installations, because a server installation always needs the glusterfs-fuse package for doing internal mounts. The advantage is that glusterfs-api does not pull in any executables that get into the $PATH. This has caused confusion before when people tried to remove the (wrongly assumed) server-only "glusterfsd" binaries. Cherry picked from commit fd8b8a81c243776b20e6d48993524ee3f1351ca6: > URL: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10643 > Change-Id: Id03f1a634ea3c62ab7008345be92e01ccf43b1a6 > BUG: 1195947 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/10554 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Tested-by: NetBSD Build System > Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Change-Id: Id03f1a634ea3c62ab7008345be92e01ccf43b1a6 BUG: 1219089 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10697 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: Fix buffer overflow in snprintfKotresh HR2015-05-081-8/+8
| | | | | | | | | | BUG: 1219823 Change-Id: I3a1c7b7742671847ed3fec13e06d861c3d09f7a9 Reviewed-On: http://review.gluster.org/#/c/10687/ Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10688 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* cluster/ec: Change meaning of trusted.ec.dirtyPranith Kumar K2015-05-087-208/+555
| | | | | | | | | | | | | | | | | - With this change, the xattr will represent if the file needs to be healed or not. It will have different values for data/entry and metadata changes. - inode ref leaks and dict_set_dynstr related leaks fixed - Added support for trylock/lock based on heal-cmd execution or not in data heal. - Made fixes to pass regression runs Change-Id: I9d8def4c2badde18a76b7898816fecfac113737a BUG: 1216303 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10385 Reviewed-on: http://review.gluster.org/10693 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cluster/ec: data heal implementation for ecPranith Kumar K2015-05-084-8/+823
| | | | | | | | | | | | | | | | | | | | | | | | | Data self-heal: 1) Take inode lock in domain 'this->name:self-heal' on 0-0 range (full file), So that no other processes try to do self-heal at the same time. 2) Take inode lock in domain 'this->name' on 0-0 range (full file), 3) perform fxattrop+fstat and get the xattrs on all the bricks 3) Choose the brick with ec->fragment number of same version as source 4) Truncate sinks 5) Unlock lock taken in 2) 5) For each block take full file lock, Read from sources write to the sinks, Unlock 6) Take full file lock and see if the file is still sane copy i.e. File didn't become unusable while the bricks are offline. Update mtime to before healing 7) xattrop with -ve values of 'dirty' and difference of highest and its own version values for version xattr 8) unlock lock acquired in 6) 9) unlock lock acquired in 1) Change-Id: I6f4d42cd5423c767262c9d7bb5ca7767adb3e5fd BUG: 1216303 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10384 Reviewed-on: http://review.gluster.org/10692 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cluster/ec: metadata/name/entry heal implementation for ecPranith Kumar K2015-05-082-0/+1058
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Metadata self-heal: 1) Take inode lock in domain 'this->name' on 0-0 range (full file) 2) perform lookup and get the xattrs on all the bricks 3) Choose the brick with highest version as source 4) Setattr uid/gid/permissions 5) removexattr stale xattrs 6) Setxattr existing/new xattrs 7) xattrop with -ve values of 'dirty' and difference of highest and its own version values for version xattr 8) unlock lock acquired in 1) Entry self-heal: 1) take directory lock in domain 'this->name:self-heal' on 'NULL' to prevent more than one self-heal 2) we take directory lock in domain 'this->name' on 'NULL' 3) Perform lookup on version, dirty and remember the values 4) unlock lock acquired in 2) 5) readdir on all the bricks and trigger name heals 6) xattrop with -ve values of 'dirty' and difference of highest and its own version values for version xattr 7) unlock lock acquired in 1) Name heal: 1) Take 'name' lock in 'this->name' on 'NULL' 2) Perform lookup on 'name' and get stat and xattr structures 3) Build gfid_db where for each gfid we know what subvolumes/bricks have a file with 'name' 4) Delete all the stale files i.e. the file does not exist on more than ec->redundancy number of bricks 5) On all the subvolumes/bricks with missing entry create 'name' with same type,gfid,permissions etc. 6) Unlock lock acquired in 1) Known limitation: At the moment with present design, it conservatively preserves the 'name' in case it can not decide whether to delete it. this can happen in the following scenario: 1) we have 3=2+1 (bricks: A, B, C) ec volume and 1 brick is down (Lets say A) 2) rename d1/f1 -> d2/f2 is performed but the rename is successful only on one of the bricks (Lets say B) 3) Now name self-heal on d1 and d2 would re-create the file on both d1 and d2 resulting in d1/f1 and d2/f2. Because we wanted to prevent data loss in the case above, the following scenario is not healable, i.e. it needs manual intervention: 1) we have 3=2+1 (bricks: A, B, C) ec volume and 1 brick is down (Lets say A) 2) We have two hard links: d1/a, d2/b and another file d3/c even before the brick went down 3) rename d3/c -> d2/b is performed 4) Now name self-heal on d2/b doesn't heal because d2/b with older gfid will not be deleted. One could think why not delete the link if there is more than 1 hardlink, but that leads to similar data loss issue I described earlier: Scenario: 1) we have 3=2+1 (bricks: A, B, C) ec volume and 1 brick is down (Lets say A) 2) We have two hard links: d1/a, d2/b 3) rename d1/a -> d3/c, d2/b -> d4/d is performed and both the operations are successful only on one of the bricks (Lets say B) 4) Now name self-heal on the 'names' above which can happen in parallel can decide to delete the file thinking it has 2 links but after all the self-heals do unlinks we are left with data loss. Change-Id: I3a68218a47bb726bd684604efea63cf11cfd11be BUG: 1216303 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10298 Reviewed-on: http://review.gluster.org/10691 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* cluster/ec: Fix dictionary compare functionPranith Kumar K2015-05-087-119/+152
| | | | | | | | | | | | | | | | | | | | | If both dicts are NULL then equal. If one of the dicts is NULL but the other has only ignorable keys then also they are equal. If both dicts are non-null then check if for each non-ignorable key, values are same or not. value_ignore function is used to skip comparing values for the keys which must be present in both the dictionaries but the value could be different. geo-rep's stime xattr doesn't need to be present in list xattr but when getxattr comes on stime xattr even if there aren't enough responses with the xattr we should still give out an answer which is maximum of the stimes available. Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d BUG: 1216303 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10078 Reviewed-on: http://review.gluster.org/10690 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* build: Introducing python-gluster packageHumble Devassy Chirammal2015-05-081-5/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently the gluster namespace/module is owned by glusterfs-api rpm. Because of this, other packages (extra-xlators or libgfapi-python) have to depend on glusterfs-api to get the gluster namespace. The gluster namespace should be owned by a core package.This patch introduce new package called python-gluster and moves the ownership of gluster module to python-gluster. Cherry picked from commit cf1c53343d4b6fe834833b76c50c61fd8a312de0: > Change-Id: Ib75ab922736f3bea6355da4d8dbec3a08b2844d5 > BUG: 1211848 > Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> > Reviewed-on: http://review.gluster.org/10244 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Change-Id: Ib75ab922736f3bea6355da4d8dbec3a08b2844d5 BUG: 1219787 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/10672 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* build: RPM build fails (make -C extras/LinuxRPM glusterrpms)Kaleb S. KEITHLEY2015-05-081-2/+2
| | | | | | | | | | | minor typo in glusterfs.spec.in prevents building RPMS Change-Id: I876943e256d393921009d75cab84944f420632ba BUG: 1219787 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/10689 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Niels de Vos <ndevos@redhat.com>
* Tests: fix file count in ec-readdir.tEmmanuel Dreyfus2015-05-081-1/+1
| | | | | | | | | | | | | | | | | On BSD systems, ls(1) reports dot-files when it is run as root, while Linux ls(1) does not do that. In ec-readdir.t, this means the directory count is different because of .trashcan directory. Fix this by using ls $M0/* instead of ls $M0 Backport of: Ia337c8a8e68a163ed527c0e4cbe313072ab1672b BUG: 1212676 Change-Id: Ic3eb43f22a7c12a51e955b459af5159afc9835aa Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10658 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/ec: add separate versions for data/entry, metadataAshish Pandey2015-05-088-29/+104
| | | | | | | | | | | | | | | | | | | Adding 64 bits in "version" key of extended attributes. First 64 bits (Left) represents Data version. Last 64 bits (right) represents Meta Data version. Note: 3.7 and 3.6 version ec can't co-exist with this change because xattrop in 3.6 will fail with ERANGE as the buffer passed to it will be '8' bytes where as the value will be 16 bytes in 3.7. Where as 3.7 version clients can work with old version files. For upgrades we need to tell users to complete heals and then upgrade BUG: 1215265 Change-Id: Ib85114680cb7e75b8371c984d9f7b6401c1ffb93 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/10312 Reviewed-on: http://review.gluster.org/10626 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* cluster/ec: Use fd instead of loc for get_size_versionAshish Pandey2015-05-083-57/+63
| | | | | | | | | | Change-Id: Ia7d43cb3b222db34ecb0e35424f1766715ed8e6a BUG: 1219358 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/10176 Reviewed-on: http://review.gluster.org/10625 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/changelog: Fixing compilation issueAravinda VK2015-05-082-2/+5
| | | | | | | | | | | | | This issue introduced due to manual rebase. Change-Id: I0589f4a0a1270190340f419b8022d6483bcf853d Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1219479 Reviewed-on: http://review.gluster.org/10685 Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10686
* features/changelog: Avoid creation of empty changelogsSaravanakumar Arumugam2015-05-083-23/+154
| | | | | | | | | | | | | | | | | An empty changelog when rolled over gets unlinked and indexed with a modified path-name in htime file. The modification is "changelog" not "CHANGELOG" in basename of the empty changelog file. BUG: 1219479 Change-Id: Ib5b825ab563fa34d8dcf4368cf6cbf4b25d78a6d Original-Author: Ajeet Jha <ajha@redhat.com> Original-Author: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-On: http://review.gluster.org/#/c/9572/ Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10642 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests: Marked tests/bugs/snapshot/bug-1112559.t as bad-testAvra Sengupta2015-05-081-0/+1
| | | | | | | | | | | | | | | | | backport of http://review.gluster.org/#/c/10660/ The fix that introduced this test has been reverted as part of design change. Hence marking it as a bad test. Change-Id: I811b6050edbf4bbb40b9ef518604d21d386f6d5c BUG: 1219782 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/10660 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> (cherry picked from commit a5d124728640f58fa3f90007c149ad69809b6ca4) Reviewed-on: http://review.gluster.org/10663
* quota/marker: turn off inode quotas by defaultvmallika2015-05-0813-42/+289
| | | | | | | | | | | | | | | | | | | | | | | inode quota is a new feature implemented in glusterfs-3.7 if quota is enabled in the older version and is upgraded to a new version, we can hit setxattr spike during self-heal of inode quotas. So, when a quota is enabled, turn off inode-quotas with a xlator option. With this patch, we still account for inode quotas but only when a write operation is performed for a particular file. User will be able to query inode quotas once the Inode-quota xlator option is enabled. Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a BUG: 1218243 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10152 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10621
* cluster/tier: don't use hot tier until subvolumes readyDan Lambright2015-05-082-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of fix 10435 to Gluster 3.7. When we attach a tier, the hot tier becomes the hashed subvolume. But directories may not yet have been replicated by the fix layout process. Hence lookups to those directories will fail on the hot subvolume. We should only go to the hashed subvolume once the layout has been fixed. This is known if the layout for the parent directory does not have an error. If there is an error, the cold tier is considered the hashed subvolume. The exception to this rules is ENOCON, in which case we do not know where the file is and must abort. Note we may revalidate a lookup for a directory even if the inode has not yet been populated by FUSE. This case can happen in tiering (where one tier has completed a lookup but the other has not, in which case we revalidate one tier when we call lookup the second time). Such inodes are still invalid and should not be consulted for validation. > http://review.gluster.org/#/c/10435/ > Change-Id: Ia2bc62e1d807bd70590bd2a8300496264d73c523 > BUG: 1214289 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/10435 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia2bc62e1d807bd70590bd2a8300496264d73c523 BUG: 1219547 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10649 Tested-by: NetBSD Build System Reviewed-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Implement [f]truncate fopsKrutika Dhananjay2015-05-083-309/+804
| | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10631 To-Do: * Make ftruncate work even in the absence of path * Aggregate and update ia_blocks appropriately when a file is truncated to a lower size. Change-Id: Icd424430066233ba61a030e72fdddf692d2b3f22 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10638 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: rename handling in dht volumeNithya Balachandran2015-05-083-0/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: Glusterfs changelogs are stored in each brick, which records the changes happened in that brick. Georep will run in all the nodes of master and processes changelogs "independently". Processing changelogs is in brick level, but all the fops will be replayed on "slave mount" point. Problem: With a DHT volume, in changelog "internal fops" are NOT recorded. For Rename case, Rename is recorded in "hashed" brick changelog. (DHT's internal fops like creating linkto file, unlink is NOT recorded). This lead us to inconsistent rename operations. For example, Distribute volume created with Two bricks B1, B2. //Consider master volume mounted @ /mnt/master and following operations executed: cd /mnt/master touch f1 // f1 falls on B1 Hash mv f1 f2 // f2 falls on B2 Hash // Here, Changelogs are recorded as below: @B1 CREATE f1 @B2 RENAME f1 f2 Here, race exists between Brick B1 and B2, say B2 will get executed first. Source file f1 itself is "NOT PRESENT", so it will go ahead and create f2 (Current implementation). We have this problem When rename falls in another brick and file is unlinked in Master. Similar kind of issue exists in following case too(multiple rename): CREATE f1 RENAME f1 f2 RENAME f2 f1 Solution: Instead of carrying out "changelogging" at "HASHED volume", carry out at the "CACHED volume". This way we have rename operations carried out where actual files are present. So,Changelog recorded as : @B1 CREATE f1 RENAME f1 f2 credit: sarumuga@redhat.com PS: Some of the races as the one below are _NOT_ fixed by this patch * f1 and f2 exist. B1 and B2 are their respective cached subvols. For both files hashed-subvol == cached-subvol * mv f1 f2 on master. * B1 has change-log entry of rename f1 f2 * rebalance migrates f2 from B1 and B2 * mv f2 f1 on master. * B2 has change-log entry of rename f2 f1 Since changelog entries (rename f1 f2) and (rename f2 f1) are processed independently by gsyncds, which of either f1 and f2 survives on slave is subject to race. Note that on master its file f1 with name f1 which survived. On slave it can be either file f1 with name f1 or file f2 with name f2 based on who wins the race of processing changelog. BUG: 1219412 Change-Id: I43725d69635e2ce065135691ef629014e8df7d50 Original-Author: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/10410 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/10628 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: Version support for Changelog ParserAravinda VK2015-05-082-11/+70
| | | | | | | | | | | | | | | | | | As optional feature, during unlink, full path will be recorded. Changelog Version number to be bumped up to 1.2. With this patch, parser checks the version number before parsing and handles accordingly. Change-Id: Ic1ad98259c39e417029a08e26a1d4b467817e65a BUG: 1218383 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10166 Reviewed-on: http://review.gluster.org/10620 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* snapshot: Handshake with glusterd is not properMohammed Rafi KC2015-05-083-20/+167
| | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/9664 If a snap is activated or deactivated, when a node is down, it is not retrieving the data properly during the handshake of glusterd With this patch, a version check will made when a glusterd is started running. If there is a mismach in version, then peers will exchange the healed data. Change-Id: I8bd2a347723db2194d3fa73295878b4dd2e9be5d BUG: 1219744 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9664 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/10661
* tools/glusterfind: Rpm install dependencies addedAravinda VK2015-05-081-0/+4
| | | | | | | | | | | | | | | | Argparse Python library is available as standard library in Python 2.7 In rhel, we need to install python-argparse. Also added pyxattr dependency to the spec file. BUG: 1219457 Change-Id: I2949d43b68731498130dbd27bf03c87f101141f0 Reviewed-on: http://review.gluster.org/10321 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10637 Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix Default values in Xml outputAravinda VK2015-05-081-0/+9
| | | | | | | | | | | | | | Default Values for last_synced, checkpoint_time and checkpoint_completion_time was zero instead of 'N/A' BUG: 1218586 Change-Id: I8a96688588dd6e22edf8bc06ee0f3afe12a9e0f5 Reviewed-on: http://review.gluster.org/10580 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10636 Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix Rsync hang issueAravinda VK2015-05-081-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | When rsync is executed using Python subprocess, by default stdout of subprocess will be None. With the log rsync performance patch stdout is assigned to PIPE. Rsync writes to that PIPE whenever it syncs files. If log_rsync_performance is disabled then nobody will consume stdout and that gets full. Rsync hangs if PIPE is full. log_rsync_performance option is introduced with patch 10070 With this patch stdout=PIPE only if log_rsync_performance is enabled. Also removed -v option from Rsync. Thanks Venky and Kotresh for RCA. BUG: 1219444 Change-Id: I4b06ca2ebdcb93ac8319f60bc637182fb3d38091 Original-Author: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10556 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10634 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* guster/dht: tiered volumes may not allow access to files undergoing migrationDan Lambright2015-05-081-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of fix 10324 to Gluster 3.7. If a read IO occurs against a file that has reached rebalance phase 2, we redirect the IO to the destination. For tiered volumes, when we try to reopen the file (on the destination), the lower level DHT receives the open call and fails; it does not have a "cached subvol". Fix is to "teach" the lower level DHT of the new location by sending a locate before the open. > http://review.gluster.org/#/c/10324/ > Change-Id: Ia4acb0035ff1da15f6a8f9ed54f43c76e8b98f5f > BUG: 1214048 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Signed-off-by: root <root@gprfs018.sbu.lab.eng.bos.redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/10324 > Tested-by: NetBSD Build System > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Tested-by: Raghavendra G <rgowdapp@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia4acb0035ff1da15f6a8f9ed54f43c76e8b98f5f BUG: 1219608 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10654 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Joseph Fernandes Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ctr/xlator: Named lookup heal of pre-existing files, before ctr was ON.Joseph Fernandes2015-05-088-6/+1021
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The CTR xlator records file meta (heat/hardlinks) into the data. This works fine for files which are created after ctr xlator is switched ON. But for files which were created before CTR xlator is ON, CTR xlator is not able to record either of the meta i.e heat or hardlinks. Thus making those files immune to promotions/demotions. Solution: The solution that is implemented in this patch is do ctr-db heal of all those pre-existent files, using named lookup. For this purpose we use the inode-xlator context variable option in gluster. The inode-xlator context variable for ctr xlator will have the following, a. A Lock for the context variable b. A hardlink list: This list represents the successful looked up hardlinks. These are the scenarios when the hardlink list is updated: 1) Named-Lookup: Whenever a named lookup happens on a file, in the wind path we copy all required hardlink and inode information to ctr_db_record structure, which resides in the frame->local variable. We dont update the database in wind. During the unwind, we read the information from the ctr_db_record and , Check if the inode context variable is created, if not we create it. Check if the hard link is there in the hardlink list. If its not there we add it to the list and send a update to the database using libgfdb. Please note: The database transaction can fail(and we ignore) as there already might be a record in the db. This update to the db is to heal if its not there. If its there in the list we ignore it. 2) Inode Forget: Whenever an inode forget hits we clear the hardlink list in the inode context variable and delete the inode context variable. Please note: An inode forget may happen for two reason, a. when the inode is delete. b. the in-memory inode is evicted from the inode table due to cache limits. 3) create: whenever a create happens we create the inode context variable and add the hardlink. The database updation is done as usual by ctr. 4) link: whenever a hardlink is created for the inode, we create the inode context variable, if not present, and add the hardlink to the list. 5) unlink: whenever a unlink happens we delete the hardlink from the list. 6) mknod: same as create. 7) rename: whenever a rename happens we update the hardlink in list. if the hardlink was not present for updation, we add the hardlink to the list. What is pending: 1) This solution will only work for named lookups. 2) We dont track afr-self-heal/dht-rebalancer traffic for healing. > http://review.gluster.org/#/c/10370/ > Cherry picked from commit cb11dd91a6cc296e4a3808364077f4eacb810e48 > Change-Id: Ia4bbaf84128ad6ce8c3ddd70bcfa82894c79585f > BUG: 1212037 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/10370 > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I367aa46c3f4b8f912248fb8be75866507f2538df BUG: 1219075 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10370 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10615 Tested-by: Vijay Bellur <vbellur@redhat.com>
* quota: support for inode quota in quota.confvmallika2015-05-0712-157/+518
| | | | | | | | | | | | | | | | | | Currently when quota limit is set, corresponding gfid is set in quota.conf. This patch supports storing inode-quota limits in quota.conf and also stores additional byte for each gfid to differentiate between usage quota limit and inode quota limit. Change-Id: I444d7399407594edd280e640681679a784d4c46a BUG: 1218170 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10069 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10524
* libgfapi : anonymous fd supportJiffin Tony Thottan2015-05-079-1/+355
| | | | | | | | | | | | | | | | | | | | | | | | | | | backport of http://review.gluster.org/#/c/9971/ Anonymous fd's are floating fd assigned to a glusterfs client without a explicit file open. Here either it will create a new anonymous fd or existing anonymous fd in the client stack for requested file.The anonymous fd's are mainly used for IO's. This patch introduces two api's glfs_h_anonymous_read and glfs_h_anonymous_write which performs read and write respectively cherry-picked as fa0ad231745846918b2625d0e1a89c0a5c3c24dc >Change-Id: Id646f2220e8387b2f8bb244c848dc1db6761444f >BUG: 1204651 >Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> >Reviewed-by: Niels de Vos <ndevos@redhat.com> >Reviewed-on: http://review.gluster.org/9971 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Change-Id: I6b01d88f92ad045e48debee23aa79f4517c6bdc2 BUG: 1218857 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: http://review.gluster.org/10635 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* glusterd: Use generation number to find peerinfo in RPC notificationsKaushal M2015-05-075-8/+53
| | | | | | | | | | | | | | | | | | | | | | The generation number for each peerinfo object is unique. It can be used to find the exact peerinfo object, which is required for peer RPC notifications. Using hostname and uuid matching to find peerinfos can return incorrect peerinfos to be returned in certain cases like multi network peer probe. This could cause updates to happen to incorrect peerinfos. Change-Id: Ia0aada8214fd6d43381e5afd282e08d53a277251 BUG: 1215018 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/10495 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 02583099a219ce327aac62af22b486c7b9fcb531) Reviewed-on: http://review.gluster.org/10623 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* geo-rep: Limit number of changelogs to process in batchAravinda VK2015-05-071-6/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changelog processing is done in batch, for example if 10 changelogs available for processing then process all at once. Collect Entry, Meta and Data operations separately, All the entry operations like CREATE, MKDIR, MKNOD, LINK, UNLINK will be executed first then rsync will be triggered for whole batch. Stime will get updated once the complete batch is complete. In case of large number of Changelogs in a batch, If geo-rep fails after Entry operations, but before rsync then on restart, it again starts from the beginning since stime is not updated. It has to process all the changelogs again. While processing same changelogs again, all CREATE will get EEXIST since all the files created in previous run. Big hit for performance. With this patch, Geo-rep limits number of changelogs per batch based on Changelog file size. So that when geo-rep fails it has to retry only last batch changelogs since stime gets updated after each batch. BUG: 1217930 Change-Id: I844448c4cdcce38a3a2e2cca7c9a50db8f5a9062 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10202 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/10499 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Venky Shankar <vshankar@redhat.com>
* Tests: use a portable way to flush kernel cacheEmmanuel Dreyfus2015-05-073-16/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Linux, kernel cache can be flushed using echo 3 > /proc/sys/vm/drop_caches This non-portable approach can be replaced by an on-purpose failed attempt to unmount: if the mount point is the current directory and umount is called, the kernel will flush inodes until it realize it cannot complete the operation because root of filesystem is busy: ( cd $M0 ; umount $M0 ) Unfortunately this does not flush everything. Entries may still be present in the kenrel FUSE cache. Using $GFS to mount the filesystem ensure --entry-timeout=0 and clears this problem. Some stall information may also remain in glusterfs caches, and that may have to be adressed by appropriate volume option. For instance tests/bugs/rpc/bug-954057.t needs to disable performance.stat-prefetch. Qtherwise, root's new credentials are not evaluated after root-quash is enabled. The test could also be done with performance.stat-prefetch enabled using various tricks: copying the file to read, creating a hard link on it, or just waiting long enough for metadata cache to expire. Backport of: I54929e899d55c04dcd9d947809133549f01fd0e1 BUG: 1212676 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Change-Id: I2849a27acaa0334ef30aae3b852019b5a6eeb419 Reviewed-on: http://review.gluster.org/10648 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* glupy: fix tuntime search path and python module directory layoEmmanuel Dreyfus2015-05-074-2/+19
| | | | | | | | | | | | | | | | | | | | | | | | | ut 1) The glupy.so xlator should embed the runtime search path for the python libraries. Unfortunately, python-config does not gives the appprioate flags, therefore we need to also use pkg-config to obtain them 2) Fix the glupy python module directory layout so that python can import the module without problem That two fixes seems to let glupy.t pass on NetBSD again. Backport of: I397aa726ab8bf7d91fa0d6d870a30910a5f4a5d9 BUG: 1212676 Change-Id: Ie48916d71f0f1a357d65c3c22b5e7d7276d720db Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10650 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* nfs: allocate the auth_cache->cache_dict on auth_cache_init()Niels de Vos2015-05-071-14/+10
| | | | | | | | | | | | | | | | | | | | | | | | It seems possible that auth_cache->cache_dict is not always allocated before it is accessed. Instead of allocating the dict upon the 1st access, just create it in auth_cache_init(). Cherry picked from commit eb8847703b8560a045e7ed0336f895bcceda98ea: > Change-Id: I00e60522478b433cb0aae0c1f0948eac544dfd2b > URL: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/10710 > BUG: 1143880 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/10600 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > Tested-by: NetBSD Build System Change-Id: I00e60522478b433cb0aae0c1f0948eac544dfd2b BUG: 1212182 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10655 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* NFS-Ganesha: Add-node and delete-node changes in HA implementationMeghana Madhusudhan2015-05-071-4/+60
| | | | | | | | | | | | | | | | | NFS-Ganesha related config files have to be copied over to the new node and NFS-Ganesha service has to be started. Similary NFS-Ganesha service has to be stopped when a node is deleted from the HA cluster. This is the backport of the patch merged on master, http://review.gluster.org/#/c/10596/ Change-Id: I21f09e30a738d2ba01861ce480ecf906667d887b BUG: 1218884 Signed-off-by: Meghana Madhusudhan <mmadhusu@redhat.com> Reviewed-on: http://review.gluster.org/10632 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* Restore build on non Linux systemsEmmanuel Dreyfus2015-05-072-4/+7
| | | | | | | | | | | | | | | | | | | | | | This change broke the build on NetBSD, FreeBSD, and MacOS X: http://review.gluster.org/10526/ We restore the build with two fixes: - Use POSIX-compliant sysconf(_SC_NPROCESSORS_ONLN) to get the number of processors, instead of Linux specific get_nprocs(). That let us remove Linux-specific #include <sys/sysinfo.h> - Only define MAX() if it is not already defined. NetBSD defines it in <sys/param.h> which is already included Backport of: I62341c670598670e47ea2f69ab94864f96588b18 BUG: 1212676 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Change-Id: I0f098153e76954bb85b5dca3f054a069e31dd94c Reviewed-on: http://review.gluster.org/10653 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-22 02:34:05 +0000