glusterfs.git/libglusterfs/src, branch v3.11dev

inode: Add per xlator ref count for inode

2017-01-18T09:29:45+00:00

Debugging inode ref leaks is very difficult as there is no info
except for the ref count on the inode. Hence this patch is first step
towards debugging inode ref leaks. With this patch, in the statedump
we get additional info that tells the ref count taken by each xlator
on the inode.

Change-Id: I7802f7e7b13c04eb4d41fdf52d5469fd2c2a185a
BUG: 1325531
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13736
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

libglusterfs: fix statvfs in FreeBSD

2017-01-10T17:08:00+00:00

FreeBSD interprets statvfs' f_bsize field in a different way than Linux.

This fix modifies the value returned by statvfs() on FreeBSD to match
the expected value by Gluster.

Change-Id: I930dab6e895671157238146d333e95874ea28a08
BUG: 1356076
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/16361
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

dht: At places needed use STACK_WIND_COOKIE

2017-01-09T18:37:02+00:00

Issue:
frame has a void * cookie pointer.
In case of STACK_WIND_COOKIE frame->cookie is assigned
to what is sent by the caller.
In case of STACK_WIND frame->cookie is assigned to point
point to the frame itself.

For ease of coding, at many places, the cookie in the cbk
is used to get the pointer to the next xl. This is
inconsistent when STACK_WIND_TAIL comes into picture.

Eg: dht_setxattr () {
    for (i = 0 ; i < conf->subvolume_cnt ; i++) {
       STACK_WIND (..dht_checking_pathinfo_cbk,
                   conf->subvolumes[i] ..);
    }

    dht_checking_pathinfo_cbk (...void *cookie...) {
        prev = cookie;
        ...
        for (i = 0; i < conf->subvolume_cnt; i++) {
            if (conf->subvolumes[i] == prev->this) {
                 ...
            }
        }
    }

    Consider the below graph:
    dht (parent)
    readdir-ahead  => Doesn't define setxattr and uses STACK_WIND_TAIL
    protocol-client

    With this graph, when dht_checking_pathinfo_cbk is called,
    cookie will have frame pointing to protocol-client.
    i.e. prev->this will be protocol-client. But dht was expecting
    it to be readdir-ahead as it has stored in conf->subvolumes[i]

Solution:
    Hence, as a thumb rule, if cbk is using cookie, then we explicitly
    call STACK_WIND_COOKIE.

Change-Id: I83aea1e24c809c5a91a0db7283e908e125471bd4
BUG: 1401812
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16039
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: Get maximum supported op-version in a cluster

2017-01-09T05:16:12+00:00

gluster volume get  cluster.opversion gives us the current
op-version on which the cluster is operating. There is no command
that lets the user know the maximum supported op-version that the
cluster can run on.

This patch adds a new global option cluster.max-op-version, that
can be used to retrieve the maximum supported op-version in a
cluster.

Usage:
	# gluster volume get all cluster.max-op-version

Example output:

Option                                  Value
------                                  -----
cluster.max-op-version                  30900

NOTE: The only way to test this feature for now is to set the
GD_OP_VERSION_MAX macro to different values (30800 for 3.8,30900
for 3.9, and so on) and rebuild glusterd. Since the regression test
framework currently doesn't have support to simulate these tests,
there are no accompanying regression tests for this feature. It
should be possible to add tests once glusto comes in and makes it
easier to run a heterogeneous cluster.

Change-Id: I547480ee5e7912664784643e436feb198b6d16d0
BUG: 1365822
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/16283
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

posix: make sure atime and mtime are set when calling lutimes()

2017-01-06T17:33:10+00:00

When overwriting an existing file with O_TRUNC, the 'atime' was set to
0, meaning the Epoch (01-Jan-1970 UTC). However, the 'mtime' gets
updated correcty.

In case 'atime' or 'mtime' is not passed in the 'struct iatt', the time
values passed to the systemcall are taken from the current values are
returned by lstat().

Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
BUG: 1401777
Reported-by: Eivind Sarto 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16034
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Bhat 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

ec: Invalidations in disperse volume should not update the stat

2017-01-06T05:12:18+00:00

Issue:
In disperse volume, the file is present across bricks, hence the stat
from one brick doesn't carry the valid size of the file. Therefore
the upcall from one brick updating the md-cache results in wrong size
being updated.

Fix:
If the notification is cache invalidation then, indicate md-cache that
the attributes is invalid.

BUG: 1410375
Change-Id: Id89d2283478e70b62b435a8891fffc86d2be8cb2
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16329
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: serialize init/reconfigure calls

2017-01-06T04:38:49+00:00

These functions do not generally "expect" to be called more than once
in parallel, and many are likely to misbehave in that case (one case
in DHT already).  Such parallel calls have not generally happened
because there are only a few places where we call these functions, and
those have been implicitly serialized until recently.  However, recent
changes in the epoll layer change that, as does brick multiplexing.
Therefore, the serialization is now explicit at the init/reconfigure
level.

It would be sufficient to serialize calls to a particular translator's
init and reconfigure functions, but that would require per-translator
locks and a bit more complexity in maintaining/using them.  Since
there's no clear reason why we would need or want to support a higher
level of parallelism, the simpler approach of a global lock should
suffice.

Change-Id: I26296c2826e91dc00b7f0c2061bcc2964ef90c4c
BUG: 1399134
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/16030
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

eventsapi: Use `getaddrinfo` instead of `gethostbyname`

2017-01-06T04:24:18+00:00

`gethostbyname` is not thread safe. Use `getaddrinfo` to avoid
any race or segfault while sending events

BUG: 1410313
Change-Id: I164af1f8eb72501fb0ed47445e68d896f7c3e908
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/16327
Reviewed-by: Atin Mukherjee 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

core: Disable the memory pooler in Gluster via a build flag

2017-01-04T11:42:40+00:00

Summary:
Passing --disable-mempool to configure will disable the mempool.

Change-Id: I60d5f70d54de507fe9f4695d7589f7ae1ba4bb0f
BUG: 1405165
Signed-off-by: Shreyas Siravara 
Reviewed-on: http://review.gluster.org/16148
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
Reviewed-by: Niels de Vos 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System 
Reviewed-by: Anoop C S 
NetBSD-regression: NetBSD Build System