glusterfs.git, branch v6.3 doc: Added release notes for 6.3 2019-06-10T06:39:32+00:00 Hari Gowtham hgowtham@redhat.com 2019-06-08T14:58:10+00:00 9acd89087baea32b06142d51568df78e0ce57d54 Fixes: bz#1718555 Change-Id: Ibe540110156d70dc1bf308eccd08d49339f4ef7a Signed-off-by: Hari Gowtham <hgowtham@redhat.com> 
Fixes: bz#1718555

Change-Id: Ibe540110156d70dc1bf308eccd08d49339f4ef7a
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
cluster/ec: honor contention notifications for partially acquired locks 2019-06-03T04:08:06+00:00 Xavi Hernandez xhernandez@redhat.com 2019-05-09T09:07:18+00:00 7a387f97315f55e1c741d6ad749fb5621f067de0 EC was ignoring lock contention notifications received while a lock was being acquired. When a lock is partially acquired (some bricks have granted the lock but some others not yet) we can receive notifications from acquired bricks, which should be honored, since we may not receive more notifications after that. Since EC was ignoring them, once the lock was acquired, it was not released until the eager-lock timeout, causing unnecessary delays on other clients. This fix takes into consideration the notifications received before having completed the full lock acquisition. After that, the lock will be releaed as soon as possible. Backport of: > BUG: bz#1708156 > Change-Id: I2a306dbdb29fb557dcab7788a258bd75d826cc12 > Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> Fixes: bz#1714172 Change-Id: I2a306dbdb29fb557dcab7788a258bd75d826cc12 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> 
EC was ignoring lock contention notifications received while a lock was
being acquired. When a lock is partially acquired (some bricks have
granted the lock but some others not yet) we can receive notifications
from acquired bricks, which should be honored, since we may not receive
more notifications after that.

Since EC was ignoring them, once the lock was acquired, it was not
released until the eager-lock timeout, causing unnecessary delays on
other clients.

This fix takes into consideration the notifications received before
having completed the full lock acquisition. After that, the lock will
be releaed as soon as possible.

Backport of:
> BUG: bz#1708156
> Change-Id: I2a306dbdb29fb557dcab7788a258bd75d826cc12
> Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>

Fixes: bz#1714172
Change-Id: I2a306dbdb29fb557dcab7788a258bd75d826cc12
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
If bind-address is IPv6 return it successfully 2019-05-29T11:37:24+00:00 Amgad Saleh amgad.saleh@nokia.com 2019-05-24T01:27:03+00:00 8c0bb45c9f002461d5c4a63e35ae215958405fb5 >Change-Id: Ibd37b6ea82b781a1a266b95f7596874134f30079 >fixes: bz#1713730 >Signed-off-by: Amgad Saleh <amgad.saleh@nokia.com> fixes: bz#1715012 Change-Id: I9a9877b6c32a1ccbd119301a4225808ddaa02de4 Signed-off-by: Hari Gowtham <hgowtham@redhat.com> 
>Change-Id: Ibd37b6ea82b781a1a266b95f7596874134f30079
>fixes: bz#1713730
>Signed-off-by: Amgad Saleh <amgad.saleh@nokia.com>

fixes: bz#1715012
Change-Id: I9a9877b6c32a1ccbd119301a4225808ddaa02de4
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
doc: Added release notes for 6.2 2019-05-23T13:24:14+00:00 Hari Gowtham hgowtham@redhat.com 2019-05-23T11:43:04+00:00 630b896166fe7d718958bcf57faad4985572ea05 Fixes: bz#1701203 Change-Id: Id105192610726e370fa977df2c29723201b94695 Signed-off-by: Hari Gowtham <hgowtham@redhat.com> 
Fixes: bz#1701203

Change-Id: Id105192610726e370fa977df2c29723201b94695
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
geo-rep: Convert gfid conflict resolutiong logs into debug 2019-05-21T05:14:53+00:00 Kotresh HR khiremat@redhat.com 2019-05-14T05:35:45+00:00 fa8c8a3fa8bc217bc869fb3db09cb49c6457e402 The gfid conflict resolution code path is not supposed to hit in generic code path. But few of the heavy rename workload (BUG: 1694820) makes it a generic case. So logging the entries to be fixed as INFO floods the log in these particular workloads. Hence convert them to DEBUG. Backport of: > Patch: https://review.gluster.org/22720 > BUG: 1709653 > Change-Id: I4d5e102b87be5fe5b54f78f329e588882d72b9d9 > Signed-off-by: Kotresh HR <khiremat@redhat.com> fixes: bz#1712223 Change-Id: I4d5e102b87be5fe5b54f78f329e588882d72b9d9 Signed-off-by: Kotresh HR <khiremat@redhat.com> 
The gfid conflict resolution code path is not supposed
to hit in generic code path. But few of the heavy rename
workload (BUG: 1694820) makes it a generic case. So
logging the entries to be fixed as INFO floods the log
in these particular workloads. Hence convert them to DEBUG.

Backport of:
 > Patch: https://review.gluster.org/22720
 > BUG: 1709653
 > Change-Id: I4d5e102b87be5fe5b54f78f329e588882d72b9d9
 > Signed-off-by: Kotresh HR <khiremat@redhat.com>

fixes: bz#1712223
Change-Id: I4d5e102b87be5fe5b54f78f329e588882d72b9d9
Signed-off-by: Kotresh HR <khiremat@redhat.com>
geo-rep: Fix sync hang with tarssh 2019-05-21T05:14:53+00:00 Kotresh HR khiremat@redhat.com 2019-05-08T05:56:06+00:00 60df33ab0b7d57e3945d70ed933a5091c4d0b86c Problem: Geo-rep sync hangs when tarssh is used as sync engine at heavy workload. Analysis and Root cause: It's found out that the tar process was hung. When debugged further, it's found out that stderr buffer of tar process on master was full i.e., 64k. When the buffer was copied to a file from /proc/pid/fd/2, the hang is resolved. This can happen when files picked by tar process to sync doesn't exist on master anymore. If this count increases around 1k, the stderr buffer is filled up. Fix: The tar process is executed using Popen with stderr as PIPE. The final execution is something like below. tar | ssh <args> root@slave tar --overwrite -xf - -C <path> It was waiting on ssh process first using communicate() and then tar. Note that communicate() reads stdout and stderr. So when stderr of tar process is filled up, there is no one to read until untar via ssh is completed. This can't happen and leads to deadlock. Hence we should be waiting on both process parallely, so that stderr is read on both processes. Backport of: > Patch: https://review.gluster.org/22684/ > Change-Id: I609c7cc5c07e210c504771115b4d551a2e891adf > BUG: 1707728 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I609c7cc5c07e210c504771115b4d551a2e891adf fixes: bz#1709738 Signed-off-by: Kotresh HR <khiremat@redhat.com> 
Problem:
Geo-rep sync hangs when tarssh is used as sync
engine at heavy workload.

Analysis and Root cause:
It's found out that the tar process was hung.
When debugged further, it's found out that stderr
buffer of tar process on master was full i.e., 64k.
When the buffer was copied to a file from /proc/pid/fd/2,
the hang is resolved.

This can happen when files picked by tar process
to sync doesn't exist on master anymore. If this count
increases around 1k, the stderr buffer is filled up.

Fix:
The tar process is executed using Popen with stderr as PIPE.
The final execution is something like below.

tar | ssh <args> root@slave tar --overwrite -xf - -C <path>

It was waiting on ssh process first using communicate() and then tar.
Note that communicate() reads stdout and stderr. So when stderr of tar
process is filled up, there is no one to read until untar via ssh is
completed. This can't happen and leads to deadlock.
Hence we should be waiting on both process parallely, so that stderr is
read on both processes.

Backport of:
 > Patch: https://review.gluster.org/22684/
 > Change-Id: I609c7cc5c07e210c504771115b4d551a2e891adf
 > BUG: 1707728
 > Signed-off-by: Kotresh HR <khiremat@redhat.com>

Change-Id: I609c7cc5c07e210c504771115b4d551a2e891adf
fixes: bz#1709738
Signed-off-by: Kotresh HR <khiremat@redhat.com>
tests/geo-rep: Fix arequal checksum comparison 2019-05-21T05:14:53+00:00 Kotresh HR khiremat@redhat.com 2019-05-08T08:40:05+00:00 d6f523927bdd5d8914650aae7bd6e6f69e91b49f The arequal checkusm comparison was always returning as successful, eventhough, if it was not. Fixed the same. Backport of: > Patch: https://review.gluster.org/22682 > Change-Id: I5083da25c0954126e452d06311d2d376f8540555 > BUG: 1707742 > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 288cffd1ab7180cccfcdea36d0c469b9fa52108f) Change-Id: I5083da25c0954126e452d06311d2d376f8540555 fixes: bz#1712220 Signed-off-by: Kotresh HR <khiremat@redhat.com> 
The arequal checkusm comparison was always returning
as successful, eventhough, if it was not. Fixed the same.

Backport of:
> Patch: https://review.gluster.org/22682
> Change-Id: I5083da25c0954126e452d06311d2d376f8540555
> BUG: 1707742
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 288cffd1ab7180cccfcdea36d0c469b9fa52108f)

Change-Id: I5083da25c0954126e452d06311d2d376f8540555
fixes: bz#1712220
Signed-off-by: Kotresh HR <khiremat@redhat.com>
geo-rep: Fix sync-method config 2019-05-17T07:47:53+00:00 Kotresh HR khiremat@redhat.com 2019-05-08T05:26:31+00:00 072a21576a65b5b0b2597115280972376f076a91 Problem: When 'use_tarssh' is set to true, it exits with successful message but the default 'rsync' was used as sync-engine. The new config 'sync-method' is not allowed to set from cli. Analysis and Fix: The 'use_tarssh' config is deprecated with new config framework and 'sync-method' is the new config to choose sync-method i.e. tarssh or rsync. This patch fixes the 'sync-method' config. The allowed values are tarssh and rsync. Backport of: > Patch: https://review.gluster.org/22683 > Change-Id: I0edb0319cad0455b29e49f2f08a64ce324735e84 > BUG: 1707686 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I0edb0319cad0455b29e49f2f08a64ce324735e84 fixes: bz#1709737 Signed-off-by: Kotresh HR <khiremat@redhat.com> 
Problem:
When 'use_tarssh' is set to true, it exits with successful
message but the default 'rsync' was used as sync-engine.
The new config 'sync-method' is not allowed to set from cli.

Analysis and Fix:
The 'use_tarssh' config is deprecated with new
config framework and 'sync-method' is the new
config to choose sync-method i.e. tarssh or rsync.
This patch fixes the 'sync-method' config. The allowed
values are tarssh and rsync.

Backport of:
 > Patch: https://review.gluster.org/22683
 > Change-Id: I0edb0319cad0455b29e49f2f08a64ce324735e84
 > BUG: 1707686
 > Signed-off-by: Kotresh HR <khiremat@redhat.com>

Change-Id: I0edb0319cad0455b29e49f2f08a64ce324735e84
fixes: bz#1709737
Signed-off-by: Kotresh HR <khiremat@redhat.com>
geo-rep: Fix rename with existing destination with same gfid 2019-05-17T07:47:53+00:00 Sunny Kumar sunkumar@redhat.com 2019-04-02T07:08:09+00:00 219c9bc92c721d49de78fd5a4d98aca7d3c66ad4 Problem: Geo-rep fails to sync the rename properly if destination exists. It results in source to be remained on slave causing more number of files on slave. Also heavy rename workload like logrotate caused lot of ESTALE errors Cause: Geo-rep fails to sync rename if destination exists if creation of source file also falls into single batch of changelogs being processed. This is because, after fixing problematic gfids verifying from master, while re-processing original entries, CREATE also was re-processed causing more files on slave and rename to be failed. Solution: Entries need to be removed from retrial list after fixing problematic gfids on slave so that it's not re-created again on slave. Also treat ESTALE as EEXIST so that the error is properly handled verifying the op on master volume. Backport of: > Patch: https://review.gluster.org/22519/ > Change-Id: I50cf289e06b997adddff0552bf2466d9201dd1f9 > BUG: 1694820 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Signed-off-by: Sunny Kumar <sunkumar@redhat.com> Change-Id: I50cf289e06b997adddff0552bf2466d9201dd1f9 fixes: bz#1709734 Signed-off-by: Kotresh HR <khiremat@redhat.com> 
Problem:
   Geo-rep fails to sync the rename properly if destination exists.
It results in source to be remained on slave causing more number of
files on slave. Also heavy rename workload like logrotate caused
lot of ESTALE errors

Cause:
   Geo-rep fails to sync rename if destination exists if creation
of source file also falls into single batch of changelogs being
processed. This is because, after fixing problematic gfids verifying
from master, while re-processing original entries, CREATE also was
re-processed causing more files on slave and rename to be failed.

Solution:
   Entries need to be removed from retrial list after fixing
problematic gfids on slave so that it's not re-created again on slave.
   Also treat ESTALE as EEXIST so that the error is properly handled
verifying the op on master volume.

Backport of:
 > Patch: https://review.gluster.org/22519/
 > Change-Id: I50cf289e06b997adddff0552bf2466d9201dd1f9
 > BUG: 1694820
 > Signed-off-by: Kotresh HR <khiremat@redhat.com>
 > Signed-off-by: Sunny Kumar <sunkumar@redhat.com>

Change-Id: I50cf289e06b997adddff0552bf2466d9201dd1f9
fixes: bz#1709734
Signed-off-by: Kotresh HR <khiremat@redhat.com>
geo-rep: Fix entries and metadata counters in geo-rep status 2019-05-17T07:47:53+00:00 Kotresh HR khiremat@redhat.com 2019-04-23T05:15:25+00:00 4a4710b81000e330fc655d95f068ea1112f1f4fb Entries counter was incremented twice and decremented only once. And entries count was being used in place of metadata entries. This patch fixes both of them. Backport of: > Patch: https://review.gluster.org/22603 > BUG: 1512093 > Change-Id: I5601a5fe8d25c9d65b72eb529171e7117ebbb67f > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit e0a6941af6ed352911698012ada895d1296b549e) fixes: bz#1709685 Change-Id: I5601a5fe8d25c9d65b72eb529171e7117ebbb67f Signed-off-by: Kotresh HR <khiremat@redhat.com> 
Entries counter was incremented twice and decremented only
once. And entries count was being used in place of metadata
entries. This patch fixes both of them.

Backport of:
 > Patch: https://review.gluster.org/22603
 > BUG: 1512093
 > Change-Id: I5601a5fe8d25c9d65b72eb529171e7117ebbb67f
 > Signed-off-by: Kotresh HR <khiremat@redhat.com>
  (cherry picked from commit e0a6941af6ed352911698012ada895d1296b549e)

fixes: bz#1709685
Change-Id: I5601a5fe8d25c9d65b72eb529171e7117ebbb67f
Signed-off-by: Kotresh HR <khiremat@redhat.com>