summaryrefslogtreecommitdiffstats
path: root/xlators/features/marker/src/marker-quota.c
Commit message (Collapse)AuthorAgeFilesLines
* Marker: Logically deadcode found by coverityHari Gowtham2020-03-311-15/+11
| | | | | | | | Have removed the deadcode found by the coverity id:1356503 Change-Id: Ieaa41e864538fb82dc967b4a214d4db09e267098 Updates: #1060 Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
* across: clang-scan: fix NULL dereferencing warningsAmar Tumballi2019-06-041-0/+3
| | | | | | | | | All these checks are done after analyzing clang-scan report produced by the CI job @ https://build.gluster.org/job/clang-scan updates: bz#1622665 Change-Id: I590305af4ceb779be952974b2a36066ffc4865ca Signed-off-by: Amar Tumballi <amarts@redhat.com>
* marker: remove some unused functionsAmar Tumballi2019-05-301-7/+1
| | | | | | | | | After basic analysis, found that these methods were not being used at all. updates: bz#1693692 Change-Id: If9cfa1ab189e6e7b56230c4e1d8e11f9694a9a65 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* marker-quota: remove dead codeAmar Tumballi2019-04-151-37/+4
| | | | | | | | | also make minor changes for signature (int -> void) where return value was not checked anywhere. updates: bz#1693692 Change-Id: Iff117712eb65e0b6b8b441a779202a117fcdf1fb Signed-off-by: Amar Tumballi <amarts@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-051-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* Quota related files: use dict_{setn|getn|deln|get_int32n|set_int32n|set_strn}Yaniv Kaul2018-09-261-24/+38
| | | | | | | | | | | | | | In a previous patch (https://review.gluster.org/20769) we've added the key length to be passed to dict_* funcs, to remove the need to strlen() it. This patch moves some code to use it. Please review carefully. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: If4f425a9827be7c36ccfbb9761006ae824a818c6
* features/marker: Avoid passing structures as parameters to functionsVijay Bellur2018-09-251-9/+12
| | | | | | | | | | | | | | | Fixes CIDs: 1388779,1390242,1390057 The patch does the following: 1. Changes the prototype of mq_inspect_file_xattr() and mq_xattr_state() to have a parameter of struct iatt * instead of struct iatt. 2. Remove unneeded struct iatt parameter from mq_inspect_directory_xattr() Change-Id: I1f6eabda042f0fe56d8c9bca010302f5e518dbe4 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-121-1691/+1800
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Quota: Fix for clang issuessanoj-unnikrishnan2018-09-071-3/+3
| | | | | | Updates: bz#1193929 Change-Id: I549f741fbf97ee3fa2d63edacee5b705ef37c49a Signed-off-by: sanoj-unnikrishnan <sunnikri@redhat.com>
* Multiple files: calloc -> mallocYaniv Kaul2018-09-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xlators/storage/posix/src/posix-inode-fd-ops.c: xlators/storage/posix/src/posix-helpers.c: xlators/storage/bd/src/bd.c: xlators/protocol/client/src/client-lk.c: xlators/performance/quick-read/src/quick-read.c: xlators/performance/io-cache/src/page.c xlators/nfs/server/src/nfs3-helpers.c xlators/nfs/server/src/nfs-fops.c xlators/nfs/server/src/mount3udp_svc.c xlators/nfs/server/src/mount3.c xlators/mount/fuse/src/fuse-helpers.c xlators/mount/fuse/src/fuse-bridge.c xlators/mgmt/glusterd/src/glusterd-utils.c xlators/mgmt/glusterd/src/glusterd-syncop.h xlators/mgmt/glusterd/src/glusterd-snapshot.c xlators/mgmt/glusterd/src/glusterd-rpc-ops.c xlators/mgmt/glusterd/src/glusterd-replace-brick.c xlators/mgmt/glusterd/src/glusterd-op-sm.c xlators/mgmt/glusterd/src/glusterd-mgmt.c xlators/meta/src/subvolumes-dir.c xlators/meta/src/graph-dir.c xlators/features/trash/src/trash.c xlators/features/shard/src/shard.h xlators/features/shard/src/shard.c xlators/features/marker/src/marker-quota.c xlators/features/locks/src/common.c xlators/features/leases/src/leases-internal.c xlators/features/gfid-access/src/gfid-access.c xlators/features/cloudsync/src/cloudsync-plugins/src/cloudsyncs3/src/libcloudsyncs3.c xlators/features/bit-rot/src/bitd/bit-rot.c xlators/features/bit-rot/src/bitd/bit-rot-scrub.c bxlators/encryption/crypt/src/metadata.c xlators/encryption/crypt/src/crypt.c xlators/performance/md-cache/src/md-cache.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible It doesn't make sense to calloc (allocate and clear) memory when the code right away fills that memory with data. It may be optimized by the compiler, or have a microscopic performance improvement. In some cases, also changed allocation size to be sizeof some struct or type instead of a pointer - easier to read. In some cases, removed redundant strlen() calls by saving the result into a variable. 1. Only done for the straightforward cases. There's room for improvement. 2. Please review carefully, especially for string allocation, with the terminating NULL string. Only compile-tested! .. and allocate memory as much as needed. xlators/nfs/server/src/mount3.c : Don't blindly allocate PATH_MAX, but strlen() the string and allocate appropriately. Also, align error messges. updates: bz#1193929 Original-Author: Yaniv Kaul <ykaul@redhat.com> Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ibda6f33dd180b7f7694f20a12af1e9576fe197f5
* Various files: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-311-1/+5
| | | | | | | | | | | | | | | | | | | | strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Check for truncated output where applicable. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN (sizeof() ) for const strings. Compile-tested only! Change-Id: I54e80d4f4a80e98d3775e376efe05c51af0b29eb updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* marker: fix for coverity issuesHari Gowtham2018-08-241-8/+10
| | | | | | | | Fixes CID: 1325591 1356503 1325540 1351704 Change-Id: I325b1184baa81402eaa009cb36ab95e5de812f6c updates: bz#789278 Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
* marker: FORWARD_NULL coverity fixShwetha Acharya2018-08-201-1/+1
| | | | | | | | | | | | | | Problem: Coverity false positive. Solution: Added a comment to ignore the false positive. CID: 1325591 BUG: 789278 Change-Id: I95037efd12c059efcfc04f3c4c13f60c530150b4 Signed-off-by: Shwetha Acharya <shwetha174@gmail.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-221-2/+2
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* libglusterfs: Capture the dict response in syncop_xattrop_cbkkarthik-us2018-04-271-4/+4
| | | | | | | | | | | | | | | Problem: Currently it is not possible to capture the xattrs values which are set on the bricks by calling syncop_(f)xattrop, because the response dict is not being assigned to any of the dictionaries. Fix: In the xattrop callback capture the response dict and send it back to the caller if it is requested. Change-Id: I9de9bcd97d6008091c9b060bcca3676cb9ae8ef9 fixes: bz#1572076 Signed-off-by: karthik-us <ksubrahm@redhat.com>
* quota: fix a crash by using bad regfile inode as parentKinglong Mee2017-07-171-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 0 0x00007f1482f1f1d7 in raise () from /lib64/libc.so.6 1 0x00007f1482f208c8 in abort () from /lib64/libc.so.6 2 0x00007f1482f18146 in __assert_fail_base () from /lib64/libc.so.6 3 0x00007f1482f181f2 in __assert_fail () from /lib64/libc.so.6 4 0x00007f148484986a in __inode_link (inode=inode@entry=0x7f14742404d4, parent=parent@entry=0x7f14742404d4, name=name@entry=0x7f1460001c48 "testfile5308", iatt=iatt@entry=0x7f1460001bc8) at inode.c:954 5 0x00007f1484849969 in inode_link (inode=0x7f14742404d4, parent=parent@entry=0x7f14742404d4, name=name@entry=0x7f1460001c48 "testfile5308", iatt=iatt@entry=0x7f1460001bc8) at inode.c:1060 6 0x00007f147591b895 in quota_build_ancestry_cbk ( frame=frame@entry=0x7f1482315e80, cookie=<optimized out>, this=0x7f147000e910, op_ret=op_ret@entry=6904, op_errno=op_errno@entry=0, entries=entries@entry=0x7f1474731c00, xdata=xdata@entry=0x0) at quota.c:779 7 0x00007f1475b2f505 in marker_build_ancestry_cbk (frame=0x7f1482315988, cookie=<optimized out>, this=<optimized out>, op_ret=<optimized out>, op_errno=<optimized out>, entries=0x7f1474731c00, xdata=0x0) at marker.c:3055 8 0x00007f14848b9cd9 in default_readdirp_cbk ( frame=frame@entry=0x7f1482315b30, cookie=<optimized out>, this=<optimized out>, op_ret=op_ret@entry=6904, op_errno=op_errno@entry=0, entries=entries@entry=0x7f1474731c00, xdata=xdata@entry=0x0) at defaults.c:1403 9 0x00007f1475f68132 in pl_readdirp_cbk (frame=0x7f1482315dac, cookie=<optimized out>, this=<optimized out>, op_ret=6904, op_errno=0, entries=0x7f1474731c00, xdata=0x0) at posix.c:2700 10 0x00007f1476e26819 in posix_readdirp (frame=0x7f1482315f54, this=<optimized out>, fd=<optimized out>, size=<optimized out>, off=<optimized out>, dict=<optimized out>) at posix.c:6282 11 0x00007f1475f6599a in pl_readdirp (frame=0x7f1482315dac, this=0x7f147000a200, fd=0x7f1484b5106c, size=0, offset=0, xdata=0x7f1481ab4f34) at posix.c:2711 12 0x00007f14848ce954 in default_readdirp_resume (frame=0x7f1482315b30, this=0x7f147000b690, fd=0x7f1484b5106c, size=0, off=0, xdata=0x7f1481ab4f34) at defaults.c:2019 13 0x00007f148485c92d in call_resume (stub=0x7f1481b65710) at call-stub.c:2508 14 0x00007f1475d54743 in iot_worker (data=0x7f147004e7d0) at io-threads.c:210 15 0x00007f148369cdc5 in start_thread () from /lib64/libpthread.so.0 16 0x00007f1482fe173d in clone () from /lib64/libc.so.6 Change-Id: I740dc691e7be1bc2a9ae3a0cb14bbf566ea77bc5 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17730 Tested-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* marker: optimize mq_update_dirty_inode_taskvmallika2016-04-061-50/+44
| | | | | | | | | | | | | | | | In function mq_update_dirty_inode_task we do readdirp on a dirty directory and for entry we again do lookup to fecth the contribution xattr. We can fetch this contribution as part of readdirp Change-Id: I766593c0dba793f1ab3b43625acce1c7d9af8d7f BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13892 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
* marker: do mq_reduce_parent_size_txn in FG for unlink & rmdirvmallika2016-04-061-4/+19
| | | | | | | | | | | | | | | | | | | | | | | * If a "rm -rf" is performed by a client, we initiate a marker background operation mq_reduce_parent_size_txn for rmdir and unlink. mq_reduce_parent_size_txn can fail when updating size on the ancestor directories, if these directories are removed during the txn as the child-parent association removed in the dentry list. So execute mq_reduce_parent_size_txn in foreground and then do the UNWIND for rmdir and unlink FOP Change-Id: Iefcdced4c6ae0dbd43f92814d0ddcd1e33825864 BUG: 1322489 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13874 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* marker: build_ancestry in markervmallika2016-04-061-17/+151
| | | | | | | | | | | | | | | | | | | | | | | * quota-enforcer doesn't execute build_ancestry in the below code path 1) Special client (PID < 0) 2) unlink 3) rename within the same directory 4) link within the same directory In these cases, marker accounting can fail as parent not found. We need to build_ancestry in marker if it doesn't find parent during update txn Change-Id: Idb7a2906500647baa6d183ba859b15e34769029c BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13857 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com>
* marker: set inode ctx before lookup unwindvmallika2016-03-211-18/+6
| | | | | | | | | | | | | | | | | | When a file is unlinked before it was accounted by the marker, then we may see error "ctx for the node ... is NULL" at many places. This is actually not an error and can be ignored. It is better to set the inode ctx before lookup/create is unwind back to protocol server Change-Id: I462b5542951f2fc2964f59af7a31978979dab1de BUG: 1318158 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13748 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* marker: do remove xattr only for last linkvmallika2015-11-091-6/+18
| | | | | | | | | | | | | | | | | | With unlink, rename, rmdir, contribution xattrs are removed. If the file is a last link then remove_xattr will fail with ENOENT. So it better to perform remove_xattr only if there are more links to the file Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d BUG: 1257694 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12033 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota: add version to quota xattrsvmallika2015-11-021-26/+46
| | | | | | | | | | | | | | | | | | | | | When a quota is disable and the clean-up process terminated without completely cleaning-up the quota xattrs. Now when quota is enabled again, this can mess-up the accounting A version number is suffixed for all quota xattrs and this version number is specific to marker xaltor, i.e when quota xattrs are requested by quotad/client marker will remove the version suffix in the key before sending the response Change-Id: I1ca2c11460645edba0f6b68db70d476d8d26e1eb BUG: 1272411 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12386 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: dir_count accounting is not atomicvmallika2015-10-121-72/+98
| | | | | | | | | | | | | | | | | | | | | | | Consider below scenario: Quota enabled on pre-existing data Now quota-crawl process will start healing xattrs Now if write is performed where healing is not complete, there is a possibility that 'update txn' is started before 'create xattr txn', in this case dir count can be missed on a dir where quota size xattr is not yet created. Solution is to get size xattr and if xattr is missing, add 1 for dir_count, this requires one additional fop if done in marker during each update iteration Better solution is to us xattrop GF_XATTROP_ADD_ARRAY64_WITH_DEFAULT Change-Id: Idc8978860a3914e70c98f96effeff52e9a24e6ba BUG: 1243798 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11694 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: marker code cleanupvmallika2015-09-301-2316/+0
| | | | | | | | | | | | | marker is re-factored with syncop approach, remove unused old code Change-Id: I36e670e63b6c166db5e64d3149d2978981e2f7c2 BUG: 1240581 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11560 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* marker: don't account destination linkto-file during internal migrationvmallika2015-09-221-32/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | During a DHT re-balance operation, quota accounts for the destination. Problem of accounting this destination file are: 1) Migration is an internal operation, 'quota list' shows more usage on the CLI and this will come to the normal numbers once the migration is complete 2) If the usage is close to the limit set, then we can get 'Disk Quota Exceeded' errors in the I/O path during file migration Solution is we should not account of the usage on the destination file during migration, at the end of the migration. We need to reduce size of the source directory and accounting for the migrated dest file We assume that there are sufficent disk space in the back-end. DHT migrator should make sure that there are sufficient disk space before it starts the migration process. Change-Id: Ie3cfe3e4ab5241c2a127ba0edc599a053d30c3a0 BUG: 1260545 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12113 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* marker: preserve previous dirty flag during update txnvmallika2015-08-291-15/+91
| | | | | | | | | | | | | | | | | | In case if dir has become dirty because of brick crash, this dirty flag on disk was getting reset in the next update txn. This patch now gets the dirty flag before setting the flag in the update txn and if this value is dirty, it keeps the flag dirty, so that inspect_directory can fix the dirty dir Change-Id: Iab2c343dbe19bd3b291adbfeebe6d9785b6bb9e3 BUG: 1251454 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12032 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* fd: Do fd_bind on successful openPranith Kumar K2015-08-281-1/+4
| | | | | | | | | | | | | | | - fd_unref should decrement fd->inode->fd_count only if it is present in the inode's fd list. - successful open/opendir should perform fd_bind. Change-Id: I81dd04f330e2fee86369a6dc7147af44f3d49169 BUG: 1207735 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/11044 Reviewed-by: Anoop C S <anoopcs@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* marker: fix log when loc.parent and inode gfid is NULLvmallika2015-08-271-89/+167
| | | | | | | | | | | | | | | | | This patch does the following 1) Set loc.parent if it is NULL Don't log warning in txn if parent is NULL 2) Don't initiate txn when inode gfid is NULL 3) optimize invoking dirty txn with status flag Change-Id: I67dd9e6268014b0b257c136e951e6ded0a2e911f BUG: 1251454 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11863 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix inode quota with renamevmallika2015-08-191-279/+270
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are three problems with marker-rename which is fixed in this patch Problem 1) 1) mq_reduce_parent_size is not handling inode-quota contribution 2) When dest files exists and IO is happening Now renaming will overwrite existing file mq_reduce_parent_size called on dest file with saved contribution, this can be a problem is IO is still happening contribution might have changed Problem 2) There is a small race between rename and in-progress write Consider below scenario 1) rename FOP invoked on file 'x' 2) write is still in progress for file 'x' 3) rename takes a lock on old-parent 4) write-update txn blocked on old-parent to acquire lock 5) in rename_cbk, contri xattrs are removed and contribution is deleted and lock is released 6) now write-update txn gets the lock and updates the wrong parent as it was holding lock on old parent so validate parent once the lock is acquired Problem 3) when a rename operation is performed, a lock is held on old parent. This lock is release before unwinding the rename operation. This can be a problem if there are in-progress writes happening during rename, where update txn can take a lock and update the old parent as inode table is not updated with new parent Change-Id: Ic3316097c001c33533f98592e8fcf234b1ee2aa2 BUG: 1240991 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11578 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: set log-level to debug for ESTALE/ENOENT for marker txnvmallika2015-07-311-58/+33
| | | | | | | | | | | | | | | For all marker txn, it is possible that while the txn in progress file can be removed. So log all ENOENT/ESTALE error in debug level Change-Id: I776ba3cda33fe992f9dcc7baede7206c4ce19553 BUG: 1245544 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11736 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dict: dict_set_bin() should never free the pointer on errorNiels de Vos2015-07-241-8/+14
| | | | | | | | | | | | | | | | | | | | | | dict_set_bin() is handling the pointer that it passed inconsistently. Depending on the errors that can occur, the pointer passed to the dict can be free'd, but there is no guarantee. It is cleaner to have the caller free the pointer that allocated it and dict_set_bin() returned an error. When dict_set_bin() returned success, the given pointer will be free'd when dict_unref() calls data_destroy(). Many callers of dict_set_bin() already take care of free'ing the pointer on error. The ones that did not, are corrected with this change too. Change-Id: I39a4f7ebc0cae6d403baba99307d7ce408f25966 BUG: 1242280 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11638 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* quota/marker: contribution with list_del can cause mem corruptionvmallika2015-07-191-1/+1
| | | | | | | | | | | | | | | | There is a possibility that contribution is removed twice from list during unlink operation (with hard links) or during rename operation Use list_del_init for a thread safe deltion of member from list Change-Id: Iff5e0c03cc8f0ed85da0db1739b84b695abf9ea6 BUG: 1244109 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11706 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix spurious failure afr-quota-xattr-mdata-heal.tvmallika2015-07-101-7/+5
| | | | | | | | | | | | | | | | During quota-update process if inode info is present in size-xattr and missing in contri-xattrs, then in function '_mq_get_metadata', we set contri-size as zero (on error -2, which means usage info present, but inode info missing). With this we are calculating wrong delta and updating the same. With this patch we are ignoring errors if inode info in xattrs are missing Change-Id: I7940a0e299b8bb425b5b43746b1f13f775c7fb92 BUG: 1241153 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11583 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: use smaller stacksize in synctask for marker updationvmallika2015-07-091-2/+2
| | | | | | | | | | | | | | | | | Default stacksize that synctask uses is 2M. For marker we set it to 16k Also move market xlator close to io-threads to have smaller stack Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651 BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11499 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix mem leak in markervmallika2015-07-091-197/+285
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem-1) Now the marker accounting happens in background, There is a possibility that before completing create_xattr_txn another create txn can be initiated for the same inode. suppose if few 100 txns are initiated before completion, this can block all synctask threads waiting on a lock and this can also consume lot of memory and can take more time to complete the background accounting operation. This patch improves the locking mechanism which can improve the performance as well reduce memory consumption Problem-2) For every lookup and for all inodes in readdirp we were initiating a new txn, this can result in more txn pending in synctask queue and lead to huge memory consumption. inspect file/dir should start a txn only if there is some delta Problem-3) When there are multiple write operations on same inode and all the synctask threads are busy. As we are checking for updation_status flag in background, all txn will be move to synctask queue. This can increase the mem usage. Only one txn for inode in a queue will be sufficient, so check and set updation falg before moving txn to background Change-Id: Ic42ce00f0a50ce51c7128ba68a1b6a0699a1cd14 BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11457 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota: marker accounting goes bad with rename while writing a filevmallika2015-06-261-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | With below test-case, marker accounting becomes bad: 1) Create a volume with 1 brick 2) fuse mount 3) on one terminal write some data dd if=/dev/zero of=f1 bs=1M count=500 oflag=sync 4) on another terminal execute below rename operation while the write is still in progress for i in {1..50}; do ii=`expr $i + 1`; mv f$i f$ii; done remove-xattr is already on while doing rename operation, we should not be doing again in background when reducing the parent size Change-Id: I969a64bb559e2341315928b55b99203e9ddee3f2 BUG: 1235195 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11403 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix mem-leak in markervmallika2015-06-251-50/+87
| | | | | | | | | | | | | | | | | | | | | When removing contribution xattr, we also need to free contribution node in memory. Use ref/unref mechanism to handle contribution node memory local->xdata should be freed in mq_local_unref There is another huge memory consumption happens in function mq_inspect_directory_xattr_task where dirty flag is not set. Change-Id: Ieca3ab4bf410c51259560e778bce4e81b9d888bf BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11361 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* Revert "quota/marker: fix mem-leak, free contribution node"Raghavendra Bhat2015-06-231-2/+0
| | | | | | | | | | | | | | | | | | This reverts commit 8ab6608accb62d6320d1fc1fbe651fcafd376270. This patch is resulting in memory corruption: http://build.gluster.org/job/rackspace-regression-2GB-triggered/11193/consoleFull contribution object might be being used by some other transaction when we free it. The correct way to handle this is to have a reference based scheme to manage the contribution object. Change-Id: Idf9993ed8268029073a3e2d699865587f20d9aea BUG: 1207735 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11362 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix mem-leak, free contribution nodevmallika2015-06-191-0/+2
| | | | | | | | | | | | When removing contribution xattr, we also need to free contribution node in memory Change-Id: I5fe97813a8f39e2f00401976046bd280f2eea54d BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11311 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota: fix double accounting with rename operationvmallika2015-06-181-1/+3
| | | | | | | | | | | | | | | | | | | | When a rename operation is performed, we are renaming the file first and performing remove-xattr when reducing the contri size from parents. This remove-xattr fails as the file is alreday renamed, this failure causes reduce-parent-size to abort resulting in double quota accounting This patch fixes the problem. We don't need to perform remove-xattr operation on a file when performing reduce-parent-size txn as this will be alreday done before starting reduce-parent-size txn Change-Id: If86e3dbb0233f6deaaa90bee72cb0ec1689c7325 BUG: 1232572 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11264 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/marker: Cleanup loc in case of errorsVijay Bellur2015-06-041-3/+1
| | | | | | | | | | | | | | | | | Missing loc_wipe() for error paths in mq_readdir_cbk() can cause memory leaks. loc_wipe() is now done for both happy and unhappy paths. Change-Id: I882aa5dcca06e25b56a828767fb2b91a1efaf83b BUG: 1227904 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/11074 Reviewed-by: Sachin Pandit <spandit@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* build: do not #include "config.h" in each fileNiels de Vos2015-05-291-5/+0
| | | | | | | | | | | | | | | | | | Instead of including config.h in each file, and have the additional config.h included from the compiler commandline (-include option). When a .c file tests for a certain #define, and config.h was not included, incorrect assumtions were made. With this change, it can not happen again. BUG: 1222319 Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10808 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* quota/marker: turn off inode quotas by defaultvmallika2015-05-061-8/+47
| | | | | | | | | | | | | | | | | | | | | inode quota is a new feature implemented in glusterfs-3.7 if quota is enabled in the older version and is upgraded to a new version, we can hit setxattr spike during self-heal of inode quotas. So, when a quota is enabled, turn off inode-quotas with a xlator option. With this patch, we still account for inode quotas but only when a write operation is performed for a particular file. User will be able to query inode quotas once the Inode-quota xlator option is enabled. Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a BUG: 1209430 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10152 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* quota/marker: fix inode quota healing after glusterfs upgradevmallika2015-04-161-3/+7
| | | | | | | | | | | | | | | | | | | | There is a problem during upgrade where, inode quotas are not healed in the contri xattrs. Healing happens if contri xattrs are missing. But healing doesn't happen if contri xattrs are present and inode quota values are missing in the contri xattrs. This patch fixes the problem Change-Id: I6c88b74b5bb333a97c5419e24cc4ada82839f474 BUG: 1211808 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/10239 Tested-by: NetBSD Build System Reviewed-by: Sachin Pandit <spandit@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* libglusterfs/syncop: Add xdata to all syncop callsRaghavendra Talur2015-04-081-14/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support for xdata in both the request and response path of syncops. Few calls like lookup already had the support; have renamed variables in few places to maintain uniformity. xdata passed downwards is known as xdata_in and xdata passed upwards is known as xdata_out. There is an old patch by Jeff Darcy at http://review.gluster.org/#/c/8769/3 which does the same for some selected calls. It also brings in xdata support at gfapi level. xdata support at gfapi level would be introduced in subsequent patches. Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc BUG: 1158621 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/9859 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-041-45/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* marker, quota: Fixing the coverity issuesNandaja Varma2015-03-311-1/+2
| | | | | | | | | | | | | | | Coverity CIDs: 1124601 1124690 1134008 1134009 Change-Id: I93992b11bb6d8f7edd065f602aec2cd7a8b433d0 BUG: 789278 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/9614 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/quota : Introducing inode quotavmallika2015-03-181-86/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ========================================================================== Inode quota ========================================================================== = Currently, the only way to retrieve the number of files/objects in a = = directory or volume is to do a crawl of the entire directory/volume. = = This is expensive and is not scalable. = = = = The proposed mechanism will provide an easier alternative to determine = = the count of files/objects in a directory or volume. = = = = The new mechanism proposes to store count of objects/files as part of = = an extended attribute of a directory. Each directory's extended = = attribute value will indicate the number of files/objects present = = in a tree with the directory being considered as the root of the tree. = = = = The count value can be accessed by performing a getxattr(). = = Cluster translators like afr, dht and stripe will perform aggregation = = of count values from various bricks when getxattr() happens on the key = = associated with file/object count. = A new interface is introduced: ------------------------------ limit-objects : limit the number of inodes at directory level list-objects : list the directories where the limit is set remove-objects : remove the limit from the directory ========================================================================== CLI COMMAND: gluster volume quota <volname> limit-objects <path> <number> [<percent>] * <number> is a hard-limit for number of objects limitation for path "<path>" If hard-limit is exceeded, creation of file/directory is no longer permitted. * <percent> is a soft-limit for number of objects creation for path "<path>" If soft-limit is exceeded, a warning is issued for each creation. CLI COMMAND: gluster volume quota <volname> remove-objects [path] ========================================================================== CLI COMMAND: gluster volume quota <volname> list-objects [path] ... Sample output: ------------------ Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded? ------------------------------------------------------------------------ -------------------------------------- /dir 10 80% 10 0 Yes Yes ========================================================================== [root@snapshot-28 dir]# ls a b file11 file12 file13 file14 file15 file16 file17 [root@snapshot-28 dir]# touch a1 touch: cannot touch `a1': Disk quota exceeded * Nine files are created in directory "dir" and directory is included in * the count too. Hence the limit "10" is reached and further file creation fails ========================================================================== Note: We have also done some re-factoring in cli for volume name validation. New function cli_validate_volname is created ========================================================================== Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b BUG: 1190108 Signed-off-by: Sachin Pandit <spandit@redhat.com> Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/9769 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* marker: fix compile time warning on buf arg.Humble Devassy Chirammal2015-03-181-2/+0
| | | | | | | | | | | | | | | | | Problem: marker-quota.c: In function 'mq_inspect_directory_xattr_task': marker-quota.c:3451:31: warning: variable 'buf' set but not used [-Wunused-but-set-variable] struct iatt buf = {0,}; Change-Id: I211378328bdb2509a5d2a186d173f7f30a670c8a BUG: 1198849 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/9928 Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Quota/marker : Support for inode quotavmallika2015-03-171-178/+1545
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the only way to retrieve the number of files/objects in a directory or volume is to do a crawl of the entire directory/volume. This is expensive and is not scalable. The new mechanism proposes to store count of objects/files as part of an extended attribute of a directory. Each directory's extended attribute value will indicate the number of files/objects present in a tree with the directory being considered as the root of the tree. Currently file usage is accounted in marker by doing multiple FOPs like setting and getting xattrs. Doing this with STACK WIND and UNWIND can be harder to debug as involves multiple callbacks. In this code we are replacing current mechanism with syncop approach as syncop code is much simpler to follow and help us implement inode quota in an organized way. Change-Id: Ibf366fbe07037284e89a241ddaff7750fc8771b4 BUG: 1188636 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/9567 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>