summaryrefslogtreecommitdiffstats
path: root/xlators/features/shard
Commit message (Collapse)AuthorAgeFilesLines
* features/shard: Fix NULL-dereference when fsync failsKrutika Dhananjay2016-03-041-0/+4
| | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/13562/ Change-Id: I6a642203949227452e0f64bcffae23ade1a34fd0 BUG: 1313315 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/13563 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Implement zerofill FOPKrutika Dhananjay2016-02-101-18/+35
| | | | | | | | | | | | | | Backport of: http://review.gluster.org/13234 Change-Id: I5ef61285dbe3ff218e50b8a272c88227c30c35c6 BUG: 1299712 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/13287 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Implement fallocate FOPKrutika Dhananjay2016-01-212-179/+329
| | | | | | | | | | | | | | | Backport of: http://review.gluster.org/13196 Change-Id: Iab0c41319af42210c871a3ed6cf52a987c5d88d7 BUG: 1299712 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/13259 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Do not update inode-ctx size (again) after xattropKrutika Dhananjay2015-12-081-2/+4
| | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12907/ This is to fix race between parallel writevs that could mess up the file size value in inode ctx. Thanks to Pranith for helping with RCA'ing the issue. Change-Id: I8e26d92568434bc6c60b0c23089532bf3bdac509 BUG: 1289570 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12911 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Set ctime to 0 in fsync callbackKrutika Dhananjay2015-11-301-0/+1
| | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12759/ ... to indicate to md-cache that it should not be caching file attributes. Change-Id: I95c94779caa26fe972aaccf6c4400278e2404267 BUG: 1285762 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12765 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/changelog: Capture FXATTROP and XATTROP in changelogKotresh HR2015-11-241-2/+0
| | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE shard xlator updates size of the file using FXATTROP or XATTROP. Hence record the same in changelog. BUG: 1284453 Change-Id: I2f14b6075f863c0bed3ee2a159085b9b536a756d Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12225 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/12732 Reviewed-by: Aravinda VK <avishwan@redhat.com>
* features/shard: Eliminate extra update to postbuf in writevKrutika Dhananjay2015-11-241-16/+17
| | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12717/ After an extending write is complete, shard translator updates postbuf at two places: 1. shard_update_file_size_cbk(), and 2. shard_post_update_size_writev_handler(). This can lead to unexpected behavior if md-cache is part of the client stack and caches and serves values returned by shard translator in postbuf. This patch eliminates the update to postbuf in shard_post_update_size_writev_handler(). Change-Id: I1b97a46931b12d5a2f5d60877e57e0caf9e9fcb6 BUG: 1285139 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12737 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Force cache-refresh when lookup/readdirp/stat detect that ↵Krutika Dhananjay2015-10-292-19/+113
| | | | | | | | | | | | | | xattr value has changed Backport of: http://review.gluster.org/12400 Change-Id: Ifa51979bc530e31d36781759ca62bcac1de7af24 BUG: 1274600 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12457 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Support geo-rep for sharded volumeKotresh HR2015-10-291-34/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Approach: Shard xlator on slave side is by passed for all the fops to geo-rep mount. So each shard on master is considered as a separate file for geo-rep and it syncs them separately on to slave. The extended attribute in which shard maintains the size is also synced from master and shard on slave doesn't calculate by itself. Pre-requisites: 1. If master is sharded volume, slave also should be sharded. 2. Slave's shard configurations should be same as master. 3. Geo-rep config of xattr sync should not be disabled. All other dependant patches: 1. http://review.gluster.org/#/c/12205/ 2. http://review.gluster.org/#/c/12206/ 3. http://review.gluster.org/#/c/12225/ 4. http://review.gluster.org/#/c/12226/ BUG: 1275972 Change-Id: Ieba70e75ebaebd70851454e1b85c0fe86022ad8d Reviewed-on: http://review.gluster.org/12228 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12438 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Return ENOTSUP as opposed to ENOTCONN in unimplemented fopsKrutika Dhananjay2015-10-132-4/+17
| | | | | | | | | | | | Backport of: http://review.gluster.org/12340 Change-Id: I0a3f9fffa148c89be0da8a904b3dfcb0d72bc1c5 BUG: 1271204 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12349 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Dump private members and addresses in statedumpKrutika Dhananjay2015-10-131-1/+15
| | | | | | | | | | | | Backport of: http://review.gluster.org/12334 Change-Id: I35c4dcb4937edcb39e1e6a15655f53be0417ac2a BUG: 1271204 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12348 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Regulate memory consumption by individual shards' inode_t ↵Krutika Dhananjay2015-10-082-18/+141
| | | | | | | | | | | | | | | | | | | | objects Backport of: http://review.gluster.org/#/c/12254/ Shard translator will now maintain an lru list of inodes associated with individual shards of constant size, and will make sure that at no point the number of these inodes will exceed the configured limit. This is to keep the memory consumption by the thousands of shards of every large file from exploding. Change-Id: I7290f7cf1d76d5545ef04dc061c170f9f27329d2 BUG: 1269730 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12313 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Use the xattr rsp dict to pick shard xattrs in xattrop cbkKrutika Dhananjay2015-10-061-1/+1
| | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12300/ The change http://review.gluster.org/#/c/11938/ makes a fix in posix translator which would cause sharding to fail fops post xattrop without this patch. Change-Id: Id86b332a14b190694f5098e94bceb86e5b73e127 BUG: 1268804 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12301 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Port log messages to new frameworkKrutika Dhananjay2015-09-294-93/+339
| | | | | | | | | | | | Backport of: http://review.gluster.org/12217 Change-Id: I129ff0d6d1cab15078fa474132c290950f5e1137 BUG: 1266822 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12236 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Performance improvements in IO path - Part 2Krutika Dhananjay2015-09-271-0/+80
| | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12138/ This is change 2/2 of the performance improvements for sharding. The changes are with respect to maintaining up-to-date values of file attributes in [f]stat, [f]setattr, link, and [f]truncate codepaths. Change-Id: I67de99e969b54d65f30fdcd63bc83d247afa84de BUG: 1261716 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12214 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Performance improvements in IO pathKrutika Dhananjay2015-09-272-70/+233
| | | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12126/ This is patch 1/2 of the performance improvement work for sharding in the IO path. What this patch does: Since the primary use-case where sharding is targeted - VM store - is a single-writer workload, instead of performing lookup on the base file everytime to gather the size and block count from the backend in reads, writes and truncate, now the size and block count is also cached and kept up-to-date after every inode write in the inode ctx. TO-DO: Make changes in rename, link, unlink, [f]setattr and [f]stat to keep the relevant iatt members up-to-date in the inode ctx. Change-Id: Id4f5c33044411b87b55968083a70a0a11a335ab2 BUG: 1261716 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12213 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Filter internal shard xattrs in {get,remove,set}xattrKrutika Dhananjay2015-09-132-0/+168
| | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12121/ and http://review.gluster.org/#/c/12136/ Change-Id: Ifadebe1cda80fa4a525605e10513e6e64ee72709 BUG: 1261008 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12127 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Do not return non-negative status on failure in writevKrutika Dhananjay2015-09-111-1/+1
| | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12140/ Change-Id: I7c49a083894cead528901ebc0a88fcfa17e53da3 BUG: 1261715 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12144 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Fix incorrect op_ret in READVKrutika Dhananjay2015-09-031-2/+8
| | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12099/ Change-Id: Ica5f951a3cb2243fe179fe84a27c369bd7de23cf BUG: 1259726 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12101 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Fix unlink failure due to non-existent shard(s)v3.7.4Krutika Dhananjay2015-08-311-4/+3
| | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12059/ Unlink of a sharded file with holes was leading to EINVAL errors because it was being wound on non-existent shards (those blocks that fall in the hole region). loc->inode was NULL in these cases and dht_unlink used to fail the FOP with EINVAL for failure to fetch cached subvol for the inode. The fix involves winding unlink on only those shards whose corresponding inodes exist in memory. Change-Id: I1e5d492a2e60491601da23f64a5d0089e536b305 BUG: 1258353 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12061 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Fix permission issuesKrutika Dhananjay2015-08-312-8/+57
| | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/11992 This patch does the following: * reverts commit b467af0e99b39ef708420d3f7f6696b0ca618512 * changes ownership on shards under /.shard to be root:root * makes readv, writev, [f]truncate, rename, and unlink fops to perform operations on files under /.shard with frame->root->{uid,gid} as 0. This would ensure that a [f]setattr on a sharded file does not need to be called on all the shards associated with it. Change-Id: I50d8533bd2b769a4dfe8cd1b49bdcfc117a7e660 BUG: 1253151 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12052 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Fix size update for writes at hole regionKrutika Dhananjay2015-08-292-2/+10
| | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/12020 Change-Id: I4b6e9101ccb881d3d285704902484e1e89ccaceb BUG: 1257204 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12026 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Ensure shards are owned by the same owner/group as the ↵Krutika Dhananjay2015-08-192-31/+45
| | | | | | | | | | | | | | original file Backport of: http://review.gluster.org/#/c/11874/ Change-Id: I8c905917430c32eb2e6573968722920d3b27fb55 BUG: 1253151 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11927 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Fill inode ctx in readdir(p) callback tooKrutika Dhananjay2015-08-121-26/+45
| | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/11854/ The only place where shard translator was initialising inode ctx was lookup callback. But if the inodes are created and linked through readdirp, shard_lookup() path _may_ not be exercised before FUSE winds other fops on them. Since shard translator does an inode_ctx_get() first thing in most fops, an uninitialised ctx could cause it to fail the operation with ENOMEM. The solution would be to also initialise inode ctx if it has not been done already in readdir(p) callback. Change-Id: I6384e2d4c6b443c02c6620bbbc38279ddfad18ae BUG: 1251106 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11866 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* features/shard: Fix excessive logging in readdir(p) callbackKrutika Dhananjay2015-08-071-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/11843/ On enabling features.shard on a volume which already has few files, performing operations on the mount was causing excessive logging of messages of the following kind: [2015-08-05 10:57:48.743352] E [shard.c:232:shard_modify_size_and_block_count] 2-dis-shard: Failed to get trusted.glusterfs.shard.file-size for 0b2bd401-c438-4d57-8ae5-8d26105d3396 Turns out this is coming from shard_readdir_cbk() where the shard translator unconditionally looks for the xattr 'trusted.glusterfs.shard.file-size' in every entry's rsp dict and logs this error on not finding it. But files that are not sharded (i.e., the ones that were created before sharding was enabled on the volume) will not (and should not) have this xattr associated with them. So these logs are misleading and must be suppressed in readdir(p). Change-Id: I2f084023fc3a2a5147d7f364253290280b816db9 BUG: 1250834 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11850 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Create /.shard with 0777 permissions (for now)Krutika Dhananjay2015-08-051-1/+1
| | | | | | | | | | | | Backport of: http://review.gluster.org/11791 Change-Id: I45d5cd696ed531ab900cb2bbb95cd8e3f4c1cb21 BUG: 1247833 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11802 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Fix block size get from xdataPranith Kumar K2015-07-301-2/+3
| | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/11778 Instead of using dict_get_ptr, dict_get_uint64 was used. If the first byte of the value is '\0' then size is returned as 0 because strtoull is used in data_to_uint64. This will make it seem like the file is not sharded at all. Original author: Pranith Kumar K <pkarampu@redhat.com> Change-Id: Id07a7d9523cb29d096b65dd68bbfcef395031aef BUG: 1247833 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11789 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Handle unlink of files with holes appropriately in unlink() ↵Krutika Dhananjay2015-07-301-2/+30
| | | | | | | | | | | | | | and rename(). Backport of: http://review.gluster.org/11737 Change-Id: I94b089dc943ca20952834aa513134f7b9e925bd4 BUG: 1247014 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11767 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: (Re)initialize local->call_count before winding lookupKrutika Dhananjay2015-07-301-2/+4
| | | | | | | | | | | | Backport of: http://review.gluster.org/11770 Change-Id: I96c86823b82595b471b09b75cc813a02dcc96663 BUG: 1247833 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11783 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* posix,shard: Fill in ia_size and ia_blocks before unwinding (f)setattrKrutika Dhananjay2015-07-271-17/+141
| | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/11754 The shard xlator will request for size xattrs in its request and posix will perform xattr_fill of requested keys before unwinding (f)setattr. Change-Id: I1a4c655bee99fb0d6c03062d876b36816282c2b0 BUG: 1246988 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11765 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Use xattrop (as opposed to setxattr) for updates to size xattrKrutika Dhananjay2015-07-213-20/+49
| | | | | | | | | | | | Backport of: http://review.gluster.org/11467 Change-Id: I9effecbb1296d11cf1629b5e5cc38192f84cfcb3 BUG: 1243655 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11689 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Skip shards resolution if lookup on /.shard returns ENOENTKrutika Dhananjay2015-06-221-22/+118
| | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/11065 This change is done in [f]truncate, rename, unlink and readv. Also, this patch also makes lookup in shard delete GF_CONTENT_KEY as a workaround for the problems with read caching of sparse files by quick-read. A proper solution would involve shard_lookup_cbk() performing a readv, aggregating and ordering the responses and setting it in the xdata before unwinding the response to upper translators, which will be done in a separate patch. Change-Id: I31e5cec8815db0269e664c17ce3e221c55c8863f BUG: 1227572 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11332 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Handle symlinks appropriately in fopsKrutika Dhananjay2015-06-061-5/+15
| | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10995 (f)stat, unlink and rename must skip doing inode_ctx_get() of shard block size on symbolic links. Change-Id: Iaf2502512a5838db137e5e1f0c14b12f5058865f BUG: 1227572 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11066 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Fix incorrect parameter to get_lowest_block()Krutika Dhananjay2015-06-031-2/+3
| | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10804 Due to get_lowest_block() being a macro, what needs to be passed to it is the evaluation of the expression (local->offset - 1), without which its substitution can cause junk values to be assigned to local->first_block. This patch also fixes calls to get_highest_block() where if offset and size are both equal to zero, it could return negative values. Change-Id: I8f1bc54b536587d6af3a5c193434d06dccbf76dc BUG: 1227572 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11051 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Fix issue with readdir(p) fopKrutika Dhananjay2015-06-022-43/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10809 Problem: When readdir(p) is performed on '/' and ".shard" happens to be the last of the entries read in a given iteration of dht_readdir(p) (in other words the entry with the highest offset in the dirent list sorted in ascending order of d_offs), shard xlator would delete this entry as part of handling the call so as to avoid exposing its presence to the application. This would cause xlators above (like fuse, readdir-ahead etc) to wind the next readdirp as part of the same req at an offset which is (now) the highest d_off (post deletion of .shard) from the previously unwound list of entries. This offset would be less than that of ".shard" and therefore cause /.shard to be read once again. If by any chance this happens to be the only entry until end-of-directory, shard xlator would delete this entry and unwind with 0 entries, causing the xlator(s) above to think there is nothing more to readdir and the fop is complete. This would prevent DHT from gathering entries from the rest of its subvolumes, causing some entries to disappear. Fix: At the level of shard xlator, if ".shard" happens to be the last entry, make shard xlator wind another readdirp at offset equal to d_off of ".shard". That way, if ".shard" happens to be the only other entry under '/' until end-of-directory, DHT would receive an op_ret=0. This would enable it to wind readdir(p) on the rest of its subvols and gather the complete picture. Also, fixed a bug in shard_lookup_cbk() wherein file_size should be fetched unconditionally in cbk since it is set unconditionally in the wind path, failing which, lookup would be unwound with ia_size and ia_blocks only equal to that of the base file. Change-Id: I0ff0b48b6c9c12edbef947b6840a77a54c131650 BUG: 1226880 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11031 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/shard: Skip block count and size update for directoriesKrutika Dhananjay2015-05-281-0/+2
| | | | | | | | | | | Backport of: http://review.gluster.org/10772 Change-Id: I3594641ef0bf6a17e1ceab3c9ad87ef18b981d2e BUG: 1225922 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10972 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Implement [f]truncate fopsKrutika Dhananjay2015-05-082-307/+802
| | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10631 To-Do: * Make ftruncate work even in the absence of path * Aggregate and update ia_blocks appropriately when a file is truncated to a lower size. Change-Id: Icd424430066233ba61a030e72fdddf692d2b3f22 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10638 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Implement readv() fopKrutika Dhananjay2015-05-062-269/+635
| | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/10528/ Change-Id: I3ff03d146a8d49cc11e7bf22ffbf830b4dd1e9f1 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10569 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/shard: Take hole size into account while computing ia_sizeKrutika Dhananjay2015-05-032-2/+10
| | | | | | | | | | | Backport of: http://review.gluster.org/10446 Change-Id: Ic05e07801605c0d610545368a513b56d8df21bf4 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10493 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Add "is-directory" checks in stat/fstatKrutika Dhananjay2015-05-031-0/+12
| | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10427 During mount, NFS directly calls stat on the root of the volume without sending a lookup on it. This was causing inode_ctx_get_block_size() to fail on /. A check is now added in [f]stat which would ensure no action is taken by shard xlator when the operation is on a directory. Change-Id: I8645b7fe58b2d44b5f527d50c1c7102de44acc00 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10509 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Implement rename() fopKrutika Dhananjay2015-04-302-44/+261
| | | | | | | | | | | | | Backport of: http://review.gluster.org/10373 Change-Id: I15867667d50b2b4aad0ee3738a29f7a410d61ef4 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10455 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Implement unlink fopKrutika Dhananjay2015-04-262-233/+495
| | | | | | | | | | | | Backport of: http://review.gluster.org/10249 Change-Id: I01761721224c4efbbc5e4992e70ecf68b3868d63 BUG: 1214247 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10377 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Consume size and block count in metadata read opsKrutika Dhananjay2015-04-262-98/+271
| | | | | | | | | | | | | | | | | | Backport of : http://review.gluster.org/10098 Metadata read fops like lookup, stat etc will now fetch the xattr that holds the size and block count information, extract the size and block count fields and set them in respective stbuf before unwinding the resultant iatt to the parent xlator. Change-Id: If7d2c4af886f8d70dc69d7cb09f1f66be391f198 BUG: 1214248 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10331 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Introduce file size xattrKrutika Dhananjay2015-04-083-43/+268
| | | | | | | | | | | | | | | With each inode write FOP, the size and block count of the file will be updated within the xattr. There are two 64 byte fields that are intentionally left blank for now for future use when consistency guarantee is introduced later in sharding. Change-Id: I40a2e700150c1f199a6bf87909f063c84ab7bb43 BUG: 1207603 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10097 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/shard: Refactor codeKrutika Dhananjay2015-04-062-49/+40
| | | | | | | | | | | | | | | | | * Renamed shard_writev_create_write_shards() to shard_common_resolve_shards() to appropriately reflect its functionality and for reuse in other fops too. * Move code common to MKNOD and CREATE into a macro. * Cut down on if nesting in shard_lookup_cbk() Change-Id: I488255499673accd426390c6d42f2b39bab3d637 BUG: 1205661 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10096 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-041-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* features/shard: Create and use xattr_req dict as and when neededKrutika Dhananjay2015-04-042-5/+38
| | | | | | | | | | | | | | | | | Reusing local->xattr_req for the several calls and callbacks per xlator fop would cause keys set from previous call/cbk (sometimes even by the xlators below) to remain which in some cases can lead to errors. For instance, the presence of "trusted.glusterfs.dht.*" keys (which are remnants of the previous call/cbk), can cause the GF_IF_INTERNAL_XATTR_GOTO() check in DHT to fail when the same dict is used to wind [f]setxattr. Change-Id: I8612d020f83f3dc55e4a34d10ccbdaf11d7b4fdd BUG: 1205661 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10095 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Fail writes if /.shard already exists as a fileKrutika Dhananjay2015-04-031-0/+8
| | | | | | | | | | Change-Id: Id7250ca4637c37a005cf2def43d5b843c1ea6562 BUG: 1205661 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10094 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Bug fixesKrutika Dhananjay2015-03-272-30/+50
| | | | | | | | | | | | | | | | | * Return number of bytes written in writev cbk on success * Eliminate separate inode table for sharding xlator. * Fix appearance of "shard" as an option within the volfile for subvolume of type features/shard. * Fix values of min and max allowed shard block size * Return @new as opposed to NULL in shard_create_gfid_dict() on success Change-Id: I6319d377a196d1c5ceed1f65d337ff8eabcb21f8 BUG: 1205661 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10003 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/shard: Introducing sharding translatorKrutika Dhananjay2015-03-195-0/+1950
Based on the high-level design by Anand V. Avati which can be found @ https://gist.github.com/avati/af04f1030dcf52e16535#sharding-xlator-stripe-20 Still to-do: * complete implementation of inode write fops - [f]truncate, zerofill, fallocate, discard * introduce transaction mechanism in inode write fops * complete readv * Handle open with O_TRUNC * Handle unlinking of all shards during unlink/rename * Compute total ia_size and ia_blocks in lookup, readdirp, etc * wind fsync/flush on all shards Note: Most of the items above are related. Once we come up with a clean way to determine the last shard/shard count for a file/file size and the mgmt of sparse regions of the file, implementing them becomes trivial. Change-Id: Id871379b53a4a916e4baa2e06f197dd8c0043b0f BUG: 1200082 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/9841 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-22 03:20:20 +0000