summaryrefslogtreecommitdiffstats
path: root/xlators/features/marker/src/marker-quota.c
Commit message (Collapse)AuthorAgeFilesLines
* marker: optimize mq_update_dirty_inode_taskvmallika2016-04-061-50/+44
| | | | | | | | | | | | | | | | In function mq_update_dirty_inode_task we do readdirp on a dirty directory and for entry we again do lookup to fecth the contribution xattr. We can fetch this contribution as part of readdirp Change-Id: I766593c0dba793f1ab3b43625acce1c7d9af8d7f BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13892 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
* marker: do mq_reduce_parent_size_txn in FG for unlink & rmdirvmallika2016-04-061-4/+19
| | | | | | | | | | | | | | | | | | | | | | | * If a "rm -rf" is performed by a client, we initiate a marker background operation mq_reduce_parent_size_txn for rmdir and unlink. mq_reduce_parent_size_txn can fail when updating size on the ancestor directories, if these directories are removed during the txn as the child-parent association removed in the dentry list. So execute mq_reduce_parent_size_txn in foreground and then do the UNWIND for rmdir and unlink FOP Change-Id: Iefcdced4c6ae0dbd43f92814d0ddcd1e33825864 BUG: 1322489 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13874 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* marker: build_ancestry in markervmallika2016-04-061-17/+151
| | | | | | | | | | | | | | | | | | | | | | | * quota-enforcer doesn't execute build_ancestry in the below code path 1) Special client (PID < 0) 2) unlink 3) rename within the same directory 4) link within the same directory In these cases, marker accounting can fail as parent not found. We need to build_ancestry in marker if it doesn't find parent during update txn Change-Id: Idb7a2906500647baa6d183ba859b15e34769029c BUG: 1320818 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13857 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com>
* marker: set inode ctx before lookup unwindvmallika2016-03-211-18/+6
| | | | | | | | | | | | | | | | | | When a file is unlinked before it was accounted by the marker, then we may see error "ctx for the node ... is NULL" at many places. This is actually not an error and can be ignored. It is better to set the inode ctx before lookup/create is unwind back to protocol server Change-Id: I462b5542951f2fc2964f59af7a31978979dab1de BUG: 1318158 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13748 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* marker: do remove xattr only for last linkvmallika2015-11-091-6/+18
| | | | | | | | | | | | | | | | | | With unlink, rename, rmdir, contribution xattrs are removed. If the file is a last link then remove_xattr will fail with ENOENT. So it better to perform remove_xattr only if there are more links to the file Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d BUG: 1257694 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12033 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota: add version to quota xattrsvmallika2015-11-021-26/+46
| | | | | | | | | | | | | | | | | | | | | When a quota is disable and the clean-up process terminated without completely cleaning-up the quota xattrs. Now when quota is enabled again, this can mess-up the accounting A version number is suffixed for all quota xattrs and this version number is specific to marker xaltor, i.e when quota xattrs are requested by quotad/client marker will remove the version suffix in the key before sending the response Change-Id: I1ca2c11460645edba0f6b68db70d476d8d26e1eb BUG: 1272411 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12386 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: dir_count accounting is not atomicvmallika2015-10-121-72/+98
| | | | | | | | | | | | | | | | | | | | | | | Consider below scenario: Quota enabled on pre-existing data Now quota-crawl process will start healing xattrs Now if write is performed where healing is not complete, there is a possibility that 'update txn' is started before 'create xattr txn', in this case dir count can be missed on a dir where quota size xattr is not yet created. Solution is to get size xattr and if xattr is missing, add 1 for dir_count, this requires one additional fop if done in marker during each update iteration Better solution is to us xattrop GF_XATTROP_ADD_ARRAY64_WITH_DEFAULT Change-Id: Idc8978860a3914e70c98f96effeff52e9a24e6ba BUG: 1243798 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11694 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: marker code cleanupvmallika2015-09-301-2316/+0
| | | | | | | | | | | | | marker is re-factored with syncop approach, remove unused old code Change-Id: I36e670e63b6c166db5e64d3149d2978981e2f7c2 BUG: 1240581 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11560 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* marker: don't account destination linkto-file during internal migrationvmallika2015-09-221-32/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | During a DHT re-balance operation, quota accounts for the destination. Problem of accounting this destination file are: 1) Migration is an internal operation, 'quota list' shows more usage on the CLI and this will come to the normal numbers once the migration is complete 2) If the usage is close to the limit set, then we can get 'Disk Quota Exceeded' errors in the I/O path during file migration Solution is we should not account of the usage on the destination file during migration, at the end of the migration. We need to reduce size of the source directory and accounting for the migrated dest file We assume that there are sufficent disk space in the back-end. DHT migrator should make sure that there are sufficient disk space before it starts the migration process. Change-Id: Ie3cfe3e4ab5241c2a127ba0edc599a053d30c3a0 BUG: 1260545 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12113 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* marker: preserve previous dirty flag during update txnvmallika2015-08-291-15/+91
| | | | | | | | | | | | | | | | | | In case if dir has become dirty because of brick crash, this dirty flag on disk was getting reset in the next update txn. This patch now gets the dirty flag before setting the flag in the update txn and if this value is dirty, it keeps the flag dirty, so that inspect_directory can fix the dirty dir Change-Id: Iab2c343dbe19bd3b291adbfeebe6d9785b6bb9e3 BUG: 1251454 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12032 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* fd: Do fd_bind on successful openPranith Kumar K2015-08-281-1/+4
| | | | | | | | | | | | | | | - fd_unref should decrement fd->inode->fd_count only if it is present in the inode's fd list. - successful open/opendir should perform fd_bind. Change-Id: I81dd04f330e2fee86369a6dc7147af44f3d49169 BUG: 1207735 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/11044 Reviewed-by: Anoop C S <anoopcs@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* marker: fix log when loc.parent and inode gfid is NULLvmallika2015-08-271-89/+167
| | | | | | | | | | | | | | | | | This patch does the following 1) Set loc.parent if it is NULL Don't log warning in txn if parent is NULL 2) Don't initiate txn when inode gfid is NULL 3) optimize invoking dirty txn with status flag Change-Id: I67dd9e6268014b0b257c136e951e6ded0a2e911f BUG: 1251454 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11863 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix inode quota with renamevmallika2015-08-191-279/+270
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are three problems with marker-rename which is fixed in this patch Problem 1) 1) mq_reduce_parent_size is not handling inode-quota contribution 2) When dest files exists and IO is happening Now renaming will overwrite existing file mq_reduce_parent_size called on dest file with saved contribution, this can be a problem is IO is still happening contribution might have changed Problem 2) There is a small race between rename and in-progress write Consider below scenario 1) rename FOP invoked on file 'x' 2) write is still in progress for file 'x' 3) rename takes a lock on old-parent 4) write-update txn blocked on old-parent to acquire lock 5) in rename_cbk, contri xattrs are removed and contribution is deleted and lock is released 6) now write-update txn gets the lock and updates the wrong parent as it was holding lock on old parent so validate parent once the lock is acquired Problem 3) when a rename operation is performed, a lock is held on old parent. This lock is release before unwinding the rename operation. This can be a problem if there are in-progress writes happening during rename, where update txn can take a lock and update the old parent as inode table is not updated with new parent Change-Id: Ic3316097c001c33533f98592e8fcf234b1ee2aa2 BUG: 1240991 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11578 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: set log-level to debug for ESTALE/ENOENT for marker txnvmallika2015-07-311-58/+33
| | | | | | | | | | | | | | | For all marker txn, it is possible that while the txn in progress file can be removed. So log all ENOENT/ESTALE error in debug level Change-Id: I776ba3cda33fe992f9dcc7baede7206c4ce19553 BUG: 1245544 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11736 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dict: dict_set_bin() should never free the pointer on errorNiels de Vos2015-07-241-8/+14
| | | | | | | | | | | | | | | | | | | | | | dict_set_bin() is handling the pointer that it passed inconsistently. Depending on the errors that can occur, the pointer passed to the dict can be free'd, but there is no guarantee. It is cleaner to have the caller free the pointer that allocated it and dict_set_bin() returned an error. When dict_set_bin() returned success, the given pointer will be free'd when dict_unref() calls data_destroy(). Many callers of dict_set_bin() already take care of free'ing the pointer on error. The ones that did not, are corrected with this change too. Change-Id: I39a4f7ebc0cae6d403baba99307d7ce408f25966 BUG: 1242280 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11638 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* quota/marker: contribution with list_del can cause mem corruptionvmallika2015-07-191-1/+1
| | | | | | | | | | | | | | | | There is a possibility that contribution is removed twice from list during unlink operation (with hard links) or during rename operation Use list_del_init for a thread safe deltion of member from list Change-Id: Iff5e0c03cc8f0ed85da0db1739b84b695abf9ea6 BUG: 1244109 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11706 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix spurious failure afr-quota-xattr-mdata-heal.tvmallika2015-07-101-7/+5
| | | | | | | | | | | | | | | | During quota-update process if inode info is present in size-xattr and missing in contri-xattrs, then in function '_mq_get_metadata', we set contri-size as zero (on error -2, which means usage info present, but inode info missing). With this we are calculating wrong delta and updating the same. With this patch we are ignoring errors if inode info in xattrs are missing Change-Id: I7940a0e299b8bb425b5b43746b1f13f775c7fb92 BUG: 1241153 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11583 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: use smaller stacksize in synctask for marker updationvmallika2015-07-091-2/+2
| | | | | | | | | | | | | | | | | Default stacksize that synctask uses is 2M. For marker we set it to 16k Also move market xlator close to io-threads to have smaller stack Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651 BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11499 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix mem leak in markervmallika2015-07-091-197/+285
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem-1) Now the marker accounting happens in background, There is a possibility that before completing create_xattr_txn another create txn can be initiated for the same inode. suppose if few 100 txns are initiated before completion, this can block all synctask threads waiting on a lock and this can also consume lot of memory and can take more time to complete the background accounting operation. This patch improves the locking mechanism which can improve the performance as well reduce memory consumption Problem-2) For every lookup and for all inodes in readdirp we were initiating a new txn, this can result in more txn pending in synctask queue and lead to huge memory consumption. inspect file/dir should start a txn only if there is some delta Problem-3) When there are multiple write operations on same inode and all the synctask threads are busy. As we are checking for updation_status flag in background, all txn will be move to synctask queue. This can increase the mem usage. Only one txn for inode in a queue will be sufficient, so check and set updation falg before moving txn to background Change-Id: Ic42ce00f0a50ce51c7128ba68a1b6a0699a1cd14 BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11457 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* quota: marker accounting goes bad with rename while writing a filevmallika2015-06-261-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | With below test-case, marker accounting becomes bad: 1) Create a volume with 1 brick 2) fuse mount 3) on one terminal write some data dd if=/dev/zero of=f1 bs=1M count=500 oflag=sync 4) on another terminal execute below rename operation while the write is still in progress for i in {1..50}; do ii=`expr $i + 1`; mv f$i f$ii; done remove-xattr is already on while doing rename operation, we should not be doing again in background when reducing the parent size Change-Id: I969a64bb559e2341315928b55b99203e9ddee3f2 BUG: 1235195 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11403 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix mem-leak in markervmallika2015-06-251-50/+87
| | | | | | | | | | | | | | | | | | | | | When removing contribution xattr, we also need to free contribution node in memory. Use ref/unref mechanism to handle contribution node memory local->xdata should be freed in mq_local_unref There is another huge memory consumption happens in function mq_inspect_directory_xattr_task where dirty flag is not set. Change-Id: Ieca3ab4bf410c51259560e778bce4e81b9d888bf BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11361 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* Revert "quota/marker: fix mem-leak, free contribution node"Raghavendra Bhat2015-06-231-2/+0
| | | | | | | | | | | | | | | | | | This reverts commit 8ab6608accb62d6320d1fc1fbe651fcafd376270. This patch is resulting in memory corruption: http://build.gluster.org/job/rackspace-regression-2GB-triggered/11193/consoleFull contribution object might be being used by some other transaction when we free it. The correct way to handle this is to have a reference based scheme to manage the contribution object. Change-Id: Idf9993ed8268029073a3e2d699865587f20d9aea BUG: 1207735 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11362 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota/marker: fix mem-leak, free contribution nodevmallika2015-06-191-0/+2
| | | | | | | | | | | | When removing contribution xattr, we also need to free contribution node in memory Change-Id: I5fe97813a8f39e2f00401976046bd280f2eea54d BUG: 1207735 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11311 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* quota: fix double accounting with rename operationvmallika2015-06-181-1/+3
| | | | | | | | | | | | | | | | | | | | When a rename operation is performed, we are renaming the file first and performing remove-xattr when reducing the contri size from parents. This remove-xattr fails as the file is alreday renamed, this failure causes reduce-parent-size to abort resulting in double quota accounting This patch fixes the problem. We don't need to perform remove-xattr operation on a file when performing reduce-parent-size txn as this will be alreday done before starting reduce-parent-size txn Change-Id: If86e3dbb0233f6deaaa90bee72cb0ec1689c7325 BUG: 1232572 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11264 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/marker: Cleanup loc in case of errorsVijay Bellur2015-06-041-3/+1
| | | | | | | | | | | | | | | | | Missing loc_wipe() for error paths in mq_readdir_cbk() can cause memory leaks. loc_wipe() is now done for both happy and unhappy paths. Change-Id: I882aa5dcca06e25b56a828767fb2b91a1efaf83b BUG: 1227904 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/11074 Reviewed-by: Sachin Pandit <spandit@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* build: do not #include "config.h" in each fileNiels de Vos2015-05-291-5/+0
| | | | | | | | | | | | | | | | | | Instead of including config.h in each file, and have the additional config.h included from the compiler commandline (-include option). When a .c file tests for a certain #define, and config.h was not included, incorrect assumtions were made. With this change, it can not happen again. BUG: 1222319 Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10808 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* quota/marker: turn off inode quotas by defaultvmallika2015-05-061-8/+47
| | | | | | | | | | | | | | | | | | | | | inode quota is a new feature implemented in glusterfs-3.7 if quota is enabled in the older version and is upgraded to a new version, we can hit setxattr spike during self-heal of inode quotas. So, when a quota is enabled, turn off inode-quotas with a xlator option. With this patch, we still account for inode quotas but only when a write operation is performed for a particular file. User will be able to query inode quotas once the Inode-quota xlator option is enabled. Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a BUG: 1209430 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/10152 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* quota/marker: fix inode quota healing after glusterfs upgradevmallika2015-04-161-3/+7
| | | | | | | | | | | | | | | | | | | | There is a problem during upgrade where, inode quotas are not healed in the contri xattrs. Healing happens if contri xattrs are missing. But healing doesn't happen if contri xattrs are present and inode quota values are missing in the contri xattrs. This patch fixes the problem Change-Id: I6c88b74b5bb333a97c5419e24cc4ada82839f474 BUG: 1211808 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/10239 Tested-by: NetBSD Build System Reviewed-by: Sachin Pandit <spandit@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* libglusterfs/syncop: Add xdata to all syncop callsRaghavendra Talur2015-04-081-14/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support for xdata in both the request and response path of syncops. Few calls like lookup already had the support; have renamed variables in few places to maintain uniformity. xdata passed downwards is known as xdata_in and xdata passed upwards is known as xdata_out. There is an old patch by Jeff Darcy at http://review.gluster.org/#/c/8769/3 which does the same for some selected calls. It also brings in xdata support at gfapi level. xdata support at gfapi level would be introduced in subsequent patches. Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc BUG: 1158621 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/9859 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-041-45/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* marker, quota: Fixing the coverity issuesNandaja Varma2015-03-311-1/+2
| | | | | | | | | | | | | | | Coverity CIDs: 1124601 1124690 1134008 1134009 Change-Id: I93992b11bb6d8f7edd065f602aec2cd7a8b433d0 BUG: 789278 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/9614 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/quota : Introducing inode quotavmallika2015-03-181-86/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ========================================================================== Inode quota ========================================================================== = Currently, the only way to retrieve the number of files/objects in a = = directory or volume is to do a crawl of the entire directory/volume. = = This is expensive and is not scalable. = = = = The proposed mechanism will provide an easier alternative to determine = = the count of files/objects in a directory or volume. = = = = The new mechanism proposes to store count of objects/files as part of = = an extended attribute of a directory. Each directory's extended = = attribute value will indicate the number of files/objects present = = in a tree with the directory being considered as the root of the tree. = = = = The count value can be accessed by performing a getxattr(). = = Cluster translators like afr, dht and stripe will perform aggregation = = of count values from various bricks when getxattr() happens on the key = = associated with file/object count. = A new interface is introduced: ------------------------------ limit-objects : limit the number of inodes at directory level list-objects : list the directories where the limit is set remove-objects : remove the limit from the directory ========================================================================== CLI COMMAND: gluster volume quota <volname> limit-objects <path> <number> [<percent>] * <number> is a hard-limit for number of objects limitation for path "<path>" If hard-limit is exceeded, creation of file/directory is no longer permitted. * <percent> is a soft-limit for number of objects creation for path "<path>" If soft-limit is exceeded, a warning is issued for each creation. CLI COMMAND: gluster volume quota <volname> remove-objects [path] ========================================================================== CLI COMMAND: gluster volume quota <volname> list-objects [path] ... Sample output: ------------------ Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded? ------------------------------------------------------------------------ -------------------------------------- /dir 10 80% 10 0 Yes Yes ========================================================================== [root@snapshot-28 dir]# ls a b file11 file12 file13 file14 file15 file16 file17 [root@snapshot-28 dir]# touch a1 touch: cannot touch `a1': Disk quota exceeded * Nine files are created in directory "dir" and directory is included in * the count too. Hence the limit "10" is reached and further file creation fails ========================================================================== Note: We have also done some re-factoring in cli for volume name validation. New function cli_validate_volname is created ========================================================================== Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b BUG: 1190108 Signed-off-by: Sachin Pandit <spandit@redhat.com> Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/9769 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* marker: fix compile time warning on buf arg.Humble Devassy Chirammal2015-03-181-2/+0
| | | | | | | | | | | | | | | | | Problem: marker-quota.c: In function 'mq_inspect_directory_xattr_task': marker-quota.c:3451:31: warning: variable 'buf' set but not used [-Wunused-but-set-variable] struct iatt buf = {0,}; Change-Id: I211378328bdb2509a5d2a186d173f7f30a670c8a BUG: 1198849 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/9928 Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Quota/marker : Support for inode quotavmallika2015-03-171-178/+1545
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the only way to retrieve the number of files/objects in a directory or volume is to do a crawl of the entire directory/volume. This is expensive and is not scalable. The new mechanism proposes to store count of objects/files as part of an extended attribute of a directory. Each directory's extended attribute value will indicate the number of files/objects present in a tree with the directory being considered as the root of the tree. Currently file usage is accounted in marker by doing multiple FOPs like setting and getting xattrs. Doing this with STACK WIND and UNWIND can be harder to debug as involves multiple callbacks. In this code we are replacing current mechanism with syncop approach as syncop code is much simpler to follow and help us implement inode quota in an organized way. Change-Id: Ibf366fbe07037284e89a241ddaff7750fc8771b4 BUG: 1188636 Signed-off-by: vmallika <vmallika@redhat.com> Signed-off-by: Sachin Pandit <spandit@redhat.com> Reviewed-on: http://review.gluster.org/9567 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* features/marker: log message clean upVijay Bellur2014-12-251-4/+3
| | | | | | | | | | | | | | | | | 1. Changed log messages to be more appropriate. 2. Changed loglevel of failures in fop_cbks to be recorded as TRACE. Logging of failures at higher loglevels is unessential in non-endpoint translators. 3. Removed a log message related to memory allocation failure. BUG: 1174087 Change-Id: I63c560c3bbd12706357fb3f696378c1a1e1efb44 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8168 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* features/marker: Fill loc->path before sending the control to healingVarun Shastry2014-07-221-1/+13
| | | | | | | | | | | | | | | | | Problem: The xattr healing part of the marker requires path to be present in the loc. Currently path is not filled while triggering from the readdirp_cbk. Solution: Current patch tries to fill the loc with path. Change-Id: I5c7dc9de60fa79ca0fe9b58d2636fd1355add0d3 BUG: 1118591 Signed-off-by: Varun Shastry <vshastry@redhat.com> Reviewed-on: http://review.gluster.org/8296 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* build: MacOSX Porting fixesHarshavardhana2014-04-241-2/+4
| | | | | | | | | | | | | | | | | | | | | git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs Working functionality on MacOSX - GlusterD (management daemon) - GlusterCLI (management cli) - GlusterFS FUSE (using OSXFUSE) - GlusterNFS (without NLM - issues with rpc.statd) Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac BUG: 1089172 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Signed-off-by: Dennis Schafroth <dennis@schafroth.com> Tested-by: Harshavardhana <harsha@harshavardhana.net> Tested-by: Dennis Schafroth <dennis@schafroth.com> Reviewed-on: http://review.gluster.org/7503 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: Filter quota xattrs on file as wellPranith Kumar K2013-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | | Problem: Quota contributions of a file/directory are tracked by quota xlator using xattrs on the file. Quota allows these xattrs to be healed as part of metadata self-heal. This leads to wrong quota calculations on this brick after self-heal because quota xattrs don't represent the actual contributions on the brick anymore. Fix: Don't let self-heal of this xattr happen as part of self-heal by filtering quota xattrs on file in listxattr. Change-Id: Iea68a116595ba271e58c6fdcc3dd21c7bb55ebb3 BUG: 1035576 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/6374 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/marker-quota: exclude dht-linkfiles from being accounted.Raghavendra G2013-11-261-2/+2
| | | | | | | | | Change-Id: I3239f5e8477664dcc04434e4d455ae447493a7ac BUG: 1022995 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/6153 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: quota friendly changesRaghavendra G2013-11-261-48/+231
| | | | | | | | | | | | | | | | | | | | | | | | | * handles renames on dht linkfiles correctly * nameless lookup friendly changes. uses gfid-to-path conversion functionality from storage/posix to build ancestry till root. * log message cleanup. * build inode contexts in readdirp * Accounting still not correct with hardlinks. Credits: ======== Vijay Bellur <vbellur@redhat.com> Raghavendra Bhat <rabhat@redhat.com> Change-Id: I415b6fbbc9691f5a38d9fd3c5d083a61e578bb81 BUG: 969461 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/5953 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: if parent inode is NULL, then get it by inode_parentRaghavendra Bhat2012-09-171-2/+3
| | | | | | | | | | | | | | | | | | | * If parent inode is NULL (nameless lookups which uses gfid for looking up the inode), then try to get it by inode_parent, instead of returning which results in the inode's contribution not being added to the list. * Prevent exceesive logging while adding the inode's contribution to the list if the operation fails. (Check if the inode's gfid is null which indicates that the inode is not yet linked to the inode table and hence addition of its contribution to the list can fail). Change-Id: I5300f00195ae92a5122613898ae3dabddb2d0bfe BUG: 851953 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/3935 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* All: License message changeVarun Shastry2012-09-131-7/+6
| | | | | | | | | | | | License message changed for server-side, dual license GPLV2 and LGPLv3+. Change-Id: Ia9e53061b9d2df3b3ef3bc9778dceff77db46a09 BUG: 852318 Signed-off-by: Varun Shastry <vshastry@redhat.com> Reviewed-on: http://review.gluster.org/3940 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* All: License message changeVarun Shastry2012-08-281-14/+6
| | | | | | | | | | | | | | | | | | The license message is changed to Copyright (c) 2008-2012 Red Hat, Inc. <http://www.redhat.com> This file is part of GlusterFS. This file is licensed to you under your choice of the GNU Lesser General Public License, version 3 or any later version (LGPLv3 or later), or the GNU General Public License, version 2 (GPLv2), in all cases as published by the Free Software Foundation. Change-Id: I07d2b63ed5fbbbd1884f1e74f2dd56013d15b0f4 BUG: 852318 Signed-off-by: Varun Shastry <vshastry@redhat.com> Reviewed-on: http://review.gluster.org/3858 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/marker: use buf->ia_gfid in all the lookup callbacksRaghavendra Bhat2012-06-191-3/+3
| | | | | | | | | | | | | | | | | * In general use buf->ia_gfid for gfid instead of inode's gfid in the callbacks of the fops where new inode is created (such as create, mkdir, mknod, symlink). In the callback path inode would not be having the gfid within it, if it is not yet linked to the inode table which happens in protocol/server. Change-Id: I75b348ad152a1bcd634a4c2db34aca97956ccb21 BUG: 822067 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.com/3567 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pranithk@gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* quota: log msgs indicating NULL parent inode in DEBUG level.Raghavendra G2012-04-031-2/+2
| | | | | | | | | | | | with name-less lookups its common to hit situations where parent inode is NULL. Signed-off-by: Raghavendra G <raghavendra@gluster.com> Change-Id: I980b14c3d4691e850ae15768f73cf1b7d867e930 BUG: 809032 Reviewed-on: http://review.gluster.com/3071 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* core: adding extra data for fopsAmar Tumballi2012-03-221-79/+85
| | | | | | | | | | | | | with this change, the xlator APIs will have a dictionary as extra argument, which is passed between all the layers. This can be utilized for overloading in some of the operations. Change-Id: I58a8186b3ef647650280e63f3e5e9b9de7827b40 Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 782265 Reviewed-on: http://review.gluster.com/2960 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: GFID file handle based backend related fixesv3.3.0qa26Raghavendra G2012-03-071-1/+4
| | | | | | | | | | | | * make sure loc->gfid is filled. Change-Id: I5304e650d9fc181ee1f3b65be7f2ff3847d9722e BUG: 790389 Signed-off-by: Raghavendra G <raghavendra@gluster.com> Reviewed-on: http://review.gluster.com/2888 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendrabhat@gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* features/marker,quota: Incorporate changes to support nameless lookupRaghavendra Bhat2012-02-021-0/+60
| | | | | | | | | | Change-Id: Ic5f00a9891bd835ebee5a3e103ef0f75d0b7fc25 BUG: 783925 Signed-off-by: Raghavendra Bhat <raghavendrabhat@gluster.com> Reviewed-on: http://review.gluster.com/2702 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amar@gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* core: get xattrs also as part of readdirpAmar Tumballi2012-01-251-1/+3
| | | | | | | | | | | | | readdirp_req() call sends a dict_t * as an argument, which contains all the xattr keys for which the entries got in readdirp_rsp() are having xattr value filled dictionary. Change-Id: I8b7e1290740ea3e884e67d19156ce849227167c0 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 765785 Reviewed-on: http://review.gluster.com/771 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* core: change lk-owner as a 1k bufferAmar Tumballi2012-01-241-1/+1
| | | | | | | | | | | | | so, NLM can send the lk-owner field directly to the locks translators, while doing the same effort, also enabled sending maximum of 500 aux gid over protocol. Change-Id: I87c2514392748416f7ffe21d5154faad2e413969 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 767229 Reviewed-on: http://review.gluster.com/779 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
generated by cgit (git 2.34.1) at 2024-06-18 08:14:08 +0000