glusterfs.git/xlators/cluster/ec/src, branch v3.10.0rc1

cluster/ec: Fallback to precompiled code

2017-02-21T13:43:23+00:00

When dynamic code generation fails for some reason, instead of causing
a failure in encode/decode, fallback to the precompiled version.

 >Change-Id: I4f8a97d3033aa5885779722b19c6e611caa4ffea
 >BUG: 1421955
 >Signed-off-by: Xavier Hernandez 
 >Reviewed-on: https://review.gluster.org/16614
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Niels de Vos 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Pranith Kumar Karampuri 
 >(cherry picked from commit 9f9d1482868e8e1044790c8358893f4421d89692)

Change-Id: I5e6221fce7a77782fd7d961e752a61dd59821a98
BUG: 1421956
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16697
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

posix: Fix creation of files with S_ISVTX on FreeBSD

2017-02-20T15:42:51+00:00

On FreeBSD the S_ISVTX flag is completely ignored when creating a
regular file. Since gluster needs to create files with this flag set,
specialy for DHT link files, it's necessary to force the flag.

This fix does this by calling fchmod() after creating a file that
must have this flag set.

> Change-Id: I51eecfe4642974df6106b9084a0b144835a4997a
> BUG: 1411228
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16417
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I2087516383bd132c59bbab98eda8f2243a2163fe
BUG: 1424973
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16686
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Change log level of messages to DEBUG

2017-02-16T14:46:22+00:00

Heal failed or passed should not be logged as info. These can be
observed from heal info if the heal is happening or not. If we require
to debug a case where heal is not happening, we can set the level to
DEBUG.

>Change-Id: I062668eadd145ef809b25e818e6bca1094f54cd6
>BUG: 1420619
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/16580
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Ashish Pandey 

Change-Id: I96517f9cf39340ae6883115e464b62339577fc5d
BUG: 1422766
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/16635
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

cluster/disperse: Do not log fop failed for lockless fops

2017-02-07T13:29:49+00:00

Problem: Operation failed messages are getting logged
based on the callbacks of lockless fop's. If a fop does
not take a lock, it is possible that it will get some
out of sync xattr, iatts. We can not depend on these
callback to psay that the fop has failed.

Solution: Print failed messages only for locked fops.
However, heal would still be triggered.

> Change-Id: I4427402c8c944c23f16073613caa03ea788bead3
> BUG: 1414287
> Signed-off-by: Ashish Pandey 
> Reviewed-on: http://review.gluster.org/16435
> Reviewed-by: Xavier Hernandez 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: I8728109d5cd93c315a5ada0a50b1f0f158493309
BUG: 1419824
Signed-off-by: Ashish Pandey 
Reviewed-on: https://review.gluster.org/16550
Tested-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

cluster/ec: Do not start heal on good file while IO is going on

2017-02-07T11:49:27+00:00

Problem:
Write on a file has been slowed down significantly after
http://review.gluster.org/#/c/13733/

RC : When update fop starts on a file, it sets dirty flag at
the start and remove it at the end which make an index entry
in indices/xattrop. During IO, SHD scans this and finds out
an index and starts heal even if all the fragments are healthy
and up tp date. This heal takes inodelk for different types of
heal. If the IO is for long time this will happen in every 60 seconds.
Due to this extra, unneccessary locking, IO gets slowed down.

Solution:
Before starting  any  type of heal check if file needs heal or not.

> Change-Id: Ib9519a43e7e4b2565d3f3153f9ca0fb92174fe51
> BUG: 1409191
> Signed-off-by: Ashish Pandey 
> Reviewed-on: http://review.gluster.org/16377
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> Reviewed-by: Xavier Hernandez 

Change-Id: I1a66aca626156164555a7a99a4f715c54c87e9a9
BUG: 1419825
Signed-off-by: Ashish Pandey 
Reviewed-on: https://review.gluster.org/16552
Tested-by: Xavier Hernandez 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: fix selinux issues with mmap()

2017-02-06T15:26:56+00:00

EC uses mmap() to create a memory area for the dynamic code. Since
the code is created on the fly and executed when needed, this region
of memory needs to have write and execution privileges.

This combination is not allowed by default by selinux. To solve the
problem a file is used as a backend storage for the dynamic code and
it's mapped into two distinct memory regions, one with write access
and the other one with execution access. This approach is the
recommended way to create dynamic code by a program in a more secure
way, and selinux allows it.

Additionally selinux requires that the backend file be stored in a
directory marked with type bin_t to be able to map it in an executable
area. To satisfy this condition, GLUSTERFS_LIBEXECDIR has been used.

This fix also changes the error check for mmap(), that was done
incorrectly (it checked against NULL instead of MAP_FAILED), and it
also correctly propagates the error codes and makes sure they aren't
silently ignored.

> Change-Id: I71c2f88be4e4d795b6cfff96ab3799c362c54291
> BUG: 1402661
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16405
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I5c2dd51b1161505316c8f78b73e9a585d0c115d0
BUG: 1418650
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16522
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Change level of messages to DEBUG

2017-01-30T14:14:28+00:00

Heal failed or passed should not be logged as warning.
These can be observed from heal info if the heal is
happening or not. If we require to debug a case where
heal is not happening, we can set the level to DEBUG.

>Change-Id: I347665c8c8b6223bb08a9f3dd5643a10ddc3b93e
>BUG: 1417050
>Signed-off-by: Ashish Pandey 
>Reviewed-on: https://review.gluster.org/16473
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Xavier Hernandez 
>CentOS-regression: Gluster Build System 
>Signed-off-by: Ashish Pandey 

Change-Id: I347665c8c8b6223bb08a9f3dd5643a10ddc3b93e
BUG: 1417135
Signed-off-by: Ashish Pandey 
Reviewed-on: https://review.gluster.org/16478
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Fixing log message

2017-01-09T07:34:40+00:00

Updating the warning message with details to improve
user understanding.

BUG: 1409202
Change-Id: I001f8d5c01c97fff1e4e1a3a84b62e17c025c520
Signed-off-by: Sunil Kumar H G 
Reviewed-on: http://review.gluster.org/16315
Tested-by: Sunil Kumar Acharya
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Do lookup on an existing file in link

2017-01-06T05:19:21+00:00

Problem:
In link fop lookup is happening on the new fop which doesn't exist so the iatt
ec serves parent xlators has size as zero which leads to 'cat' giving empty output

Fix:
Change code so that lookup happens on the existing link instead.

BUG: 1409730
Change-Id: I70eb02fe0633e61d1d110575589cc2dbe5235d76
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/16320
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Tested-by: Xavier Hernandez 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System