summaryrefslogtreecommitdiffstats
path: root/tests/basic/tier
Commit message (Collapse)AuthorAgeFilesLines
* cluster/tier re-enable tier.t in automatic testsDan Lambright2015-09-291-39/+123
| | | | | | | | | | | | | | | | | | This is a backport of 12208 > Re-enable tier.t in automatic tests. Disable check > for BSD until recurring problem with SQLlite on it is understood. > Change-Id: Ib13b269ab841a59a0a41d8478c8627b180b16c61 > BUG: 1231268 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Id462e9f67cc90c79aa7a230f2baae87c0746ff6f BUG: 1262335 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12253 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com>
* cluster/tier: Handle FOPs on files being migratedN Balachandran2015-09-251-0/+121
| | | | | | | | | | | | | | | | | | | | | Determine which DHT level is responsible for handling fops on a file undergoing migration based on the name of the the linkto xattr set on the file being migrated and process accordingly. Change-Id: I82772e39314d4fe7f2ba0dcf22de0c6a374ee139 BUG: 1265892 Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/12090 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 470869a954c17f32a3ba43ccda7442f82c0da6b2) Reviewed-on: http://review.gluster.org/12224 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: add gluster v tier <vol>Dan Lambright2015-09-101-5/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of 11984. > Currently the tier feature piggy backs off the rebalance command > syntax to obtain status and this is clumsy. Introduce a new > tier command that can do tier specific operations, starting > with volume status to display counters. > Old commands: > gluster volume attach-tier <vol> [replica count] {bricklist..} > gluster volume detach-tier <vol> {start|stop|commit} > New commands: > gluster volume tier <vol> attach [replica count] {bricklist} | > detach {start|stop|commit} | > status > Change-Id: Ic07b3c6260588162de7d34380f8cbd3d8a7f35d3 > BUG: 1255693 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/11984 > Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Id45bd0fa6b8606dd47863de83a694908da393229 BUG: 1261664 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12143 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
* glusterd/tier: configure tier daemon during volume restartMohammed Rafi KC2015-08-191-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | rebalance daemon will be running on every tier volume for promoting/demoting the files. When volume/glusterd is restarted, then we need to configure the daemon. Back port of > >Change-Id: Ib565240a70edea2ec8bc1601c52b40c0783491d3 >BUG: 1225330 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Signed-off-by: Joseph Fernandes <josferna@redhat.com> >Reviewed-on: http://review.gluster.org/10933 >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit 7aaa414c67be460aca3c1c5eed047e3db17bd435) Change-Id: Id15f5a18fd2ebaf8642b02561ae3666e619e3f83 BUG: 1235202 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/11376 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tiering/rebalance: tier daemon stopped with out updating statusMohammed Rafi KC2015-06-261-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a subvol goes down, tier daemon stopped immediately, and the status shows as "Progressing". With this change, with respect to tier xlator, when a subvol goes offline it will update the status as failed. Back port of: >Change-Id: I9f722ed0d35cda8c7fc1a7e75af52222e2d0fdb7 >BUG: 1227803 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/11068 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit d3714f252d91f4d1d5df05c4dcc8bc7c2ee75326) Change-Id: I268b7e2631779d15db5d9aec495dfd00ded94c5c BUG: 1235203 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/11414 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/volume set: Validate volume set option for tierMohammed Rafi KC2015-06-161-6/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Volume set option related to tier volume can only be set for tier volume, also currently all volume set i for tier option accepts a non-negative integer. This patch validate both condition. Back port of: >Change-Id: I3611af048ff4ab193544058cace8db205ea92336 >BUG: 1216960 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Signed-off-by: Dan Lambright <dlambrig@redhat.com> >Reviewed-on: http://review.gluster.org/10751 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Joseph Fernandes (cherry picked from commit f6a062044a3447bea5bf0fcf21a3f85c00fb6c7d) Change-Id: Ic6081f0ce7ae7effac69ba192bd35c8d382a11d5 BUG: 1230560 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/11173 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes
* tiering/nfs: duplication of nodes in client graphMohammed Rafi KC2015-05-312-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Back port of http://review.gluster.org/10820 When creating client volfiles, xlator tier-dht will be loaded for each volume. So for services like nfs have one or more volumes . So for each volume in the graph a tier-dht xlator will be created. So the graph parser will fail because of the redundant node in graph. By this change tier-dht will be renamed as volname-tier-dht >Change-Id: I3c9b9c23ddcb853773a8a02be7fd8a5d09a7f972 >BUG: 1222840 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/10820 >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System >Reviewed-by: Kaushal M <kaushal@redhat.com> Change-Id: I5629d48d4d1dbec8790f75e2fee66729aa2f6eed BUG: 1226029 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10981 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Reviewed-by: Kaushal M <kaushal@redhat.com>
* tiering: Send both attach-tier and tier-start togetherMohammed Rafi KC2015-05-092-8/+77
| | | | | | | | | | | | | | | | | | | | | | Back port of http://review.gluster.org/10363 After attaching tier, we have to start tier rebalance process. This patch is to trigger tier start along with attch-tier. >Change-Id: I39380f95123f0087a82213ef263f9f33adcc5adc >BUG: 1214222 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/10363 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia27e97a9dc0772f3f56b991ce4019281ab7bf1b5 BUG: 1219848 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10711 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tiering/cli: Check replica count and bricks are proper or notMohammed Rafi KC2015-05-091-1/+2
| | | | | | | | | | | | | | | | | | | | | | | Back port of http://review.gluster.org/10428 Right now, attach-tier calls parsing function for add-brick. Add-brick does not have any check for brick count and replca count compatibility. >Change-Id: I44ec13eadffc003a9ebf8c4eb0193df559933a68 >BUG: 1215122 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/10428 >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: Ie86d5b61b044386a062bfc9782178853e3af0f11 BUG: 1219850 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10710 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cli/tiering: Enhance cli output for tieringMohammed Rafi KC2015-05-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | back port of http://review.gluster.org/10284 Fix for handling cli output for attach-tier and detach-tier >Change-Id: I4d17f4b09612754fe1b8cec6c2e14927029b9678 >BUG: 1211562 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/10284 >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System >Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: Ic30c8f0a104d89bf98f5d0069937a34674ee3f2d BUG: 1220052 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10713 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glusterd: support for tier volumes 'detach start' and 'detach commit'Dan Lambright2015-05-091-10/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Back port of http://review.gluster.org/10108 These commands work in a manner analagous to rebalancing when removing a brick. The existing migration daemon detects "detach start" and switches to moving data off the hot tier. While in this state all lookups are directed to the cold tier. gluster v detach-tier <vol> start gluster v detach-tier <vol> commit The status and stop cli commands shall be submitted separately. >Change-Id: I24fda5cc3ba74f5fb8aa9a3234ad51f18b80a8a0 >BUG: 1205540 >Signed-off-by: Dan Lambright <dlambrig@redhat.com> >Signed-off-by: root <root@localhost.localdomain> >Signed-off-by: Dan Lambright <dlambrig@redhat.com> >Reviewed-on: http://review.gluster.org/10108 >Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Change-Id: I212d748d077fb5870ee84b316c653acbafbea3f7 BUG: 1220047 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/10708 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ctr/xlator: Named lookup heal of pre-existing files, before ctr was ON.Joseph Fernandes2015-05-081-0/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The CTR xlator records file meta (heat/hardlinks) into the data. This works fine for files which are created after ctr xlator is switched ON. But for files which were created before CTR xlator is ON, CTR xlator is not able to record either of the meta i.e heat or hardlinks. Thus making those files immune to promotions/demotions. Solution: The solution that is implemented in this patch is do ctr-db heal of all those pre-existent files, using named lookup. For this purpose we use the inode-xlator context variable option in gluster. The inode-xlator context variable for ctr xlator will have the following, a. A Lock for the context variable b. A hardlink list: This list represents the successful looked up hardlinks. These are the scenarios when the hardlink list is updated: 1) Named-Lookup: Whenever a named lookup happens on a file, in the wind path we copy all required hardlink and inode information to ctr_db_record structure, which resides in the frame->local variable. We dont update the database in wind. During the unwind, we read the information from the ctr_db_record and , Check if the inode context variable is created, if not we create it. Check if the hard link is there in the hardlink list. If its not there we add it to the list and send a update to the database using libgfdb. Please note: The database transaction can fail(and we ignore) as there already might be a record in the db. This update to the db is to heal if its not there. If its there in the list we ignore it. 2) Inode Forget: Whenever an inode forget hits we clear the hardlink list in the inode context variable and delete the inode context variable. Please note: An inode forget may happen for two reason, a. when the inode is delete. b. the in-memory inode is evicted from the inode table due to cache limits. 3) create: whenever a create happens we create the inode context variable and add the hardlink. The database updation is done as usual by ctr. 4) link: whenever a hardlink is created for the inode, we create the inode context variable, if not present, and add the hardlink to the list. 5) unlink: whenever a unlink happens we delete the hardlink from the list. 6) mknod: same as create. 7) rename: whenever a rename happens we update the hardlink in list. if the hardlink was not present for updation, we add the hardlink to the list. What is pending: 1) This solution will only work for named lookups. 2) We dont track afr-self-heal/dht-rebalancer traffic for healing. > http://review.gluster.org/#/c/10370/ > Cherry picked from commit cb11dd91a6cc296e4a3808364077f4eacb810e48 > Change-Id: Ia4bbaf84128ad6ce8c3ddd70bcfa82894c79585f > BUG: 1212037 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/10370 > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I367aa46c3f4b8f912248fb8be75866507f2538df BUG: 1219075 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10370 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10615 Tested-by: Vijay Bellur <vbellur@redhat.com>
* Tests: use a portable way to flush kernel cacheEmmanuel Dreyfus2015-05-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Linux, kernel cache can be flushed using echo 3 > /proc/sys/vm/drop_caches This non-portable approach can be replaced by an on-purpose failed attempt to unmount: if the mount point is the current directory and umount is called, the kernel will flush inodes until it realize it cannot complete the operation because root of filesystem is busy: ( cd $M0 ; umount $M0 ) Unfortunately this does not flush everything. Entries may still be present in the kenrel FUSE cache. Using $GFS to mount the filesystem ensure --entry-timeout=0 and clears this problem. Some stall information may also remain in glusterfs caches, and that may have to be adressed by appropriate volume option. For instance tests/bugs/rpc/bug-954057.t needs to disable performance.stat-prefetch. Qtherwise, root's new credentials are not evaluated after root-quash is enabled. The test could also be done with performance.stat-prefetch enabled using various tricks: copying the file to read, creating a hard link on it, or just waiting long enough for metadata cache to expire. Backport of: I54929e899d55c04dcd9d947809133549f01fd0e1 BUG: 1212676 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Change-Id: I2849a27acaa0334ef30aae3b852019b5a6eeb419 Reviewed-on: http://review.gluster.org/10648 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* glusterd: Support distributed replicated volumes on hot tierDan Lambright2015-04-081-9/+13
| | | | | | | | | | | | | | We did not set up the graph properly for hot tiers with replicated subvolumes. Also add check that the file has not already been moved by another replicated brick on the same node. Change-Id: I9adef565ab60f6774810962d912168b77a6032fa BUG: 1206517 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10054 Reviewed-by: Joseph Fernandes <josferna@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* cluster/dht: fix tier.c problems found prior to feature freezeDan Lambright2015-04-061-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch resolves tiering translator issues taken from the list in bug 1203776. These issues have been selected to be fixed first. The rest will be fixed in a subsequent patch (or are not a problem). 3. Replace hardcoded #defines of promote/demote file names 6. Use loc_wipe() in migrate_using_query_file() 9. Only promote/demote files on the same node on which they reside. 14. Replace calloc with GF_CALLOC in tier.c and ensure freeing done properly. 15. Handle if parse_query_str fails 22. Only load gfdb library on server side, remove SQL references from client. Change-Id: I6563b11e58ab2e4c6b1ce44db755781ad6d930fb BUG: 1203776 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/9987 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* cluster/dht: Fix spurious failure in tier test.Dan Lambright2015-03-231-2/+11
| | | | | | | | | | | | Need to wait for a few seconds for rebalancing to complete before stopping volume. Change-Id: Ib81c02645240e7d74ebfb3e31ccbc612fc77b119 BUG: 1194753 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/9966 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* cluster/dht: Add tier translator.Dan Lambright2015-03-211-0/+116
The tier translator shares most of DHT's code. It differs in how subvolumes are chosen for I/Os, and how file migration (cache promotion and demotion) is managed. That different functionality is split to either DHT or tier logic according to the "tier_methods" structure. A cache promotion and demotion thread is created in a manner similar to the rebalance daemon. The thread operates a timing wheel which periodically checks for promotion and demotion candidates (files). Candidates are queued and then migrated. Candidates must exist on the same node as the daemon and meet other critera per caching policies. This patch has two authors (Dan Lambright and Joseph Fernandes). Dan did the DHT changes and Joe wrote the cache policies. The fix depends on DHT readidr changes and the database library which have been submitted separately. Header files in libglusterfs/src/gfdb should be reviewed in patch 9683. For more background and design see the feature page [1]. [1] http://www.gluster.org/community/documentation/index.php/Features/data-classification Change-Id: Icc26c517ccecf5c42aef039f5b9c6f7afe83e46c BUG: 1194753 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/9724 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2026-01-26 04:28:19 +0000