glusterfs.git/xlators/cluster/dht/src/dht-common.c, branch v3.8rc2

cluster/dht: Perform NULL check on xdata before dict_get()

2016-05-06T14:15:02+00:00

        Backport of: http://review.gluster.org/14212

.. to prevent unnecessary logs from gf_msg_callingfn()


Change-Id: Ic2f21532f09af3ab7d36ce5f20c561fff5208fbb
BUG: 1333244
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14218
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Niels de Vos

dht/afr/client/posix: Fail mkdir without gfid-req

2016-04-29T13:57:39+00:00

Do not allow directory creations without gfids as
after the directories are created, operations
on them fail anyway. So it is better to fail mkdir.

BUG: 1317361
Change-Id: I8f8e3b38bbded1960b7215bac0432500f7e78038
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13690
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

libglusterfs: Add debug and trace logs for stack trace

2016-04-27T19:26:31+00:00

It has become very difficult to identify the xlator which returned
negative op_ret. Being able to just change the log level and
visualize the stack is helpful in such cases.

Change-Id: I6545b4802c1ab4d0d230d5e9e036afb2384882e1
BUG: 1330052
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/13448
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Vijay Bellur

cluster/distribute: detect stale layouts in entry fops

2016-04-22T17:28:54+00:00

dht_mkdir ()
{
      first-hashed-subvol = hashed-subvol for "bname" in in-memory
                            layout of "parent";
      inodelk (SETLKW, parent, "LAYOUT_HEAL_DOMAIN", "can be any
               subvol, but we choose first-hashed-subvol randomly");
      {
begin:
            hashed-subvol = hashed-subvol for "bname" in in-memory
                            layout of "parent";
            hash-range = extract hashe-range from layout of "parent";

            ret = mkdir (parent/bname, hashed-subvol, hash-range);
            if (ret == "hash-value doesn't fall into layout stored on
                       the brick (this error is returned by posix-mkdir)")
            {
                refresh_parent_layout ();
                goto begin;
            }

      }
      inodelk (UNLCK, parent, "LAYOUT_HEAL_DOMAIN",
               "first-hashed-subvol");

      proceed with other parts of dht_mkdir;
}

posix_mkdir (parent/bname, client-hash-range)
{

       disk-hash-range = getxattr (parent, "dht-layout-key");
       if (disk-hash-range != client-hash-range) {
              fail-with-error ("hash-value doesn't fall into layout
                                stored on the brick");
              return 0;
       }

       continue-with-posix-mkdir;
}

Similar changes need to be done for dentry operations like create,
symlink, link, unlink, rmdir, rename. These will be addressed in
subsequent patches. This patch addresses only mkdir codepath.

This change breaks stripe tests, as on some striped subvols dht layout
xattrs are not set for some reason. This results in failure of
mkdir. Since striped volumes are always created with dht, some tests
associated with stripe also fail. So, I am making following tests
changes (since stripe is out of maintainance):
* modify ./tests/basic/rpc-coverage.t to not to use striped volumes
* mark all (2) tests in tests/bugs/stripe/ as bad tests

Change-Id: Idd1ae879f24a48303dc743c1bb4d91f89a629e25
BUG: 1323040
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/13885
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran

quota: setting 'read-only' option in xdata to instruct DHT to not heal

2016-04-19T07:18:30+00:00

When quota is enabled the quota enforcer tries to get the size of the
source directory by sending nameless lookup to quotad. But if the rename
is successful even on one subvol or the source layout has anomalies then
this nameless lookup in quotad tries to heal the directory which requires
a lock on as many subvols as it can. But src is already locked as part of
rename. For rename to proceed in brick it needs to complete a cluster-wide
lookup. But cluster-wide lookup in quotad is blocked on locks held by rename,
hence a deadlock. To avoid this quota sends an option in xdata which instructs
DHT not to heal.

Change-Id: I792f9322331def0b1f4e16e88deef55d0c9f17f0
BUG: 1252244
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/13988
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

dht: add "nuke" functionality for efficient server-side deletion

2016-04-07T15:07:55+00:00

This turns a special xattr into an rmdir with flags set.  When that hits
the posix translator on the server side, that causes the file/directory
to be moved into the special "landfill" directory.  From there, the
posix janitor thread will take care of deleting it entirely on the
server side - traversing it recursively if necessary.  A couple of
secondary issues were fixed to make this effective.

 * FUSE now ensures that setxattr values are NUL terminated.

 * The janitor thread now gets woken up immediately when something is
   placed in 'landfill' instead of only when file descriptors need to be
   closed.

 * The default landfill-emptying interval was reduced to 10s.

To use the feature, issue a setxattr something like this:

   setfattr -n glusterfs.dht.nuke -v "" /mnt/glusterfs/vol/some_dir

The value doesn't actually matter; the mere receipt of a request with
this key is sufficient.  Some day it might be useful to allow setting a
required value as a sort of password, so that only those who know it can
access the underlying special functionality.

Change-Id: I8a343c2cdb40a76d5a06c707191fb67babb8514f
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/13878
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

dht: lock on subvols to prevent lookup vs rmdir race

2016-04-06T05:06:02+00:00

There is a possibility that while an rmdir is completed on
some non-hashed subvol and proceeding to others, a lookup
selfheal can recreate the same directory on those subvols
for which the rmdir had succeeded. Now the deletion of the
parent directory will fail with an ENOTEMPTY.

To fix this take blocking inodelk on the subvols before
starting rmdir. Selfheal must also take blocking inodelk
before creating the entry.

Change-Id: I168a195c35ac1230ba7124d3b0ca157755b3df96
BUG: 1245065
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/13528
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

dht: report constant directory size

2016-03-20T21:37:43+00:00

Directory size is meaningless.  Every filesystem has its own
unpredictable way of increasing or decreasing it, based on internal data
structures and even transient conditions.  Some filesystems (e.g. ext4)
never decrease it at all.  Others (e.g. btrfs) don't even report it.
Very few programs look at it, and those that do are broken.
Unfortunately, one such program is GNU tar, which will complain when it
sees different values because at different times we got the value from
different DHT subvolumes.  To avoid such problems, just report a
constant value.

Change-Id: Id64ce917c75b5f7ff50cb55b6e997f3b3556e7e3
BUG: 1302948
Original-author: Shyam 
Signed-off-by: Jeff Darcy 
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/13770
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

dht: mkdir must unwind with latest ctime

2016-02-26T11:14:32+00:00

Currently fops like mkdir used the the ctime it gets
after creating the directory entry. But setting layout
also updates the ctime of a directory. Hence DHT must
get the ctime after the setxattr call and unwind with
the latest ctime to avoid mismatch in time seen by
applications like tar.

Change-Id: Iecbbe3aac5244af5da9788b48ccf299ca56b4bae
BUG: 1302948
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/13352
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

cluster/dht: Skip subvols if no layout present

2016-02-15T04:33:39+00:00

Running "rm -rf" on a tiered volume sometimes caused
the client to crash because dht_readdirp_cbk referenced
a NULL layout for the hot tier subvol.
Now, entries are skipped if the layout is NULL. This
can cause "rm -rf" to fail with ENOTEMPTY rmdir failures.

Change-Id: Idd71a9d0f7ee712899cc7113bbf2cd3dcb25808b
BUG: 1307208
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/13440
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Raghavendra G