glusterfs.git/xlators/mount/fuse/src, branch v3.7.6

mount/fuse: use a queue instead of pipe to communicate with thread

2015-11-02T09:21:36+00:00

doing inode/entry invalidations.

Writing to pipe can block if pipe is full. This can lead to deadlocks
in some situations. Consider following situation:

1. Kernel sends a write on an inode. Client is waiting for a response
   to write from brick.
2. A lookup happens on behalf of different application/thread on the
   same inode. In response, mdc tries to invalidate the inode.
3. fuse_invalidate_inode is called. It writes a invalidation request
   to pipe. Another thread which reads from this pipe writes the
   request to /dev/fuse. The invalidate code in fuse-kernel-module,
   tries to acquire lock on all pages for the inode and is blocked as
   a write is in progress on same inode (step 1)
4. Now, poller thread is blocked in invalidate notification and cannot
   receive any messages from same socket (on which lookup response
   came). But client is expecting a response for write from same
   socket (again step1) and we've a deadlock.

The deadlock can be solved in two ways:
1. Use a queue (and a conditional variable for notifications) to pass
   invalidation requests from poller to invalidate thread. This is a
   variant of using non-blocking pipe, but doesn't have any limit on the
   amount of data (worst case we run out of memory and error out).

2. Allow events from sockets, immediately after we read one
   rpc-msg. Currently we disallow events till that rpc-msg is read from
   socket, processed and handled by higher layers. That way we won't run
   into these kind of issues. Also, it'll increase parallelism in way of
   reading from sockets.

This patch implements solution 1 above.

Change-Id: I8e8199fd7f4da9eab46a719d9292f35c039967e1
BUG: 1276550
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/12402
(cherry picked from commit 4f65f894ab1c19618383ba212dc0f0df48675823)
Reviewed-on: http://review.gluster.org/12466
Tested-by: Gluster Build System

features/shard: Support geo-rep for sharded volume

2015-10-29T08:40:02+00:00

Approach:
      Shard xlator on slave side is by passed for all the fops
to geo-rep mount. So each shard on master is considered as a
separate file for geo-rep and it syncs them separately on to
slave. The extended attribute in which shard maintains the
size is also synced from master and shard on slave doesn't
calculate by itself.

Pre-requisites:
      1. If master is sharded volume, slave also should be sharded.
      2. Slave's shard configurations should be same as master.
      3. Geo-rep config of xattr sync should not be disabled.

All other dependant patches:
      1. http://review.gluster.org/#/c/12205/
      2. http://review.gluster.org/#/c/12206/
      3. http://review.gluster.org/#/c/12225/
      4. http://review.gluster.org/#/c/12226/

BUG: 1275972
Change-Id: Ieba70e75ebaebd70851454e1b85c0fe86022ad8d
Reviewed-on: http://review.gluster.org/12228
Tested-by: NetBSD Build System 
Reviewed-by: Krutika Dhananjay 
Reviewed-by: Pranith Kumar Karampuri 
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12438
Tested-by: Gluster Build System

fuse: resolve complete path after a graph switch

2015-10-08T16:13:35+00:00

If a graph switch has happended as part of a attach-tier,
then there is a chance to hash fops to newly added brick
before fix-layout. This causes on going i/o to fail.

This patch will resolve a path, for graph switch by sending
recursive lookup to the parent directories. Those lookups
will help to heal the directory.

backport of>
>Change-Id: Ia2bb4b43a21e5cc6875ba1205628744c3f0ce4e5
>BUG: 1263549
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12184
>Tested-by: NetBSD Build System 
>Tested-by: Dan Lambright 
>Reviewed-by: Dan Lambright 

(cherry picked from commit d0edb6d555d687f76837515207b9408be0bdd55e)

Change-Id: Ie92cecd5e77178d227ef21242cd0e1af0fe9ee72
BUG: 1259081
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12319
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-28T09:51:08+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1246397
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11875
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

mount/fuse: Log ENODATA as DEBUG in {f}removexattr

2015-09-03T10:59:49+00:00

Logging ENODATA errors for {f}removexattr at a higher loglevel does not
add a lot of value and causes a log message flood as per multiple reports.

Added a new cbk, fuse_removexattr_cbk() to be used with removexattr fops.
ENODATA now gets logged at loglevel DEBUG in fuse_removexattr_cbk(). This also
prevents more conditional checks in the common fuse_err_cbk() callback.

Change-Id: I1585b4d627e0095022016c47d7fd212018a7194b
BUG: 1248941
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/12015
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/12041

libgfapi: send explicit lookups on inodes linked in readdirp

2015-07-06T15:14:02+00:00

          Backport of http://review.gluster.org/11236

If the inode is linked via readdirp, then the consuners of gfapi which are using
handles (got either in lookup or readdirp) might not send an explicit lookup on
that object again (ex: NFS, samba, USS). If there is a replicate volume where
the replicas of the object are not in sync, then readdirp followed by fops might
lead data being served from the subvolume which is not in sync with latest
data. And since lookup is needed to trigger self-heal on that object the
consumers might keep getting wrong data until an explicit lookup is not done.

Fuse handles this situation by sending an explicit lookup by itself (fuse
xlator) on those inodes which are linked via readdirp, whenever a fop comes on
that inode.

The same procedure is done in gfapi as well to address this situation.

Thanks to shyam(srangana@redhat.com) for valuable inputs

Change-Id: I4230fae8e0b01a95c056282b08ed30832d4804a7
BUG: 1240190
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/11545
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Niels de Vos

fuse: squash 64-bit inodes in readdirp when enable-ino32 is set

2015-06-04T23:17:26+00:00

The structures returned by readdirp contain the inode 2x. Only one of
them was squashed into 32-bits when enable-ino32 is enabled.

Backport of:
> Change-Id: I33a6d28fb118bb23971f918ffeb983d7f033106e
> BUG: 1223889
> Reviewed-on: http://review.gluster.org/10881
> Signed-off-by: Niels de Vos 
> Tested-by: Cyril Peponnet  [on release-3.5]

Change-Id: I33a6d28fb118bb23971f918ffeb983d7f033106e
BUG: 1223890
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10882
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

meta: implement fsync(dir)

2015-06-04T11:56:08+00:00

Change-Id: I707c608a9803fe6ef86860ca5578d4d3f63fd2aa
BUG: 1225859
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/10970
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

protocol: increase default group-cache-timeout to 300 seconds

2015-05-06T17:34:22+00:00

sssd uses 300 seconds by default too. There is no need to overload sssd
with requests that it would have cached.

Cherry picked from commit 34833364e9839f0036bccd58ec0a8a963e69263e:
> BUG: 1215187
> Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10371
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Vijay Bellur 

Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
BUG: 1215189
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10523
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

geo-rep: Fix ignoring geo-rep safe errors

2015-05-05T06:15:21+00:00

Fix ignoring geo-rep safe errors in fuse layer
and also ignore logging in client translator
for mknod. Though it is rare, to happen with
mknod, it might happen with history crawl on
overlapping changelogs replay.

BUG: 1217938
Change-Id: If06f7a6b6f86a315b4e033e294d6f6be67135cb8
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10422
Reviewed-on: http://review.gluster.org/10533
Reviewed-by: Aravinda VK 
Tested-by: NetBSD Build System
Tested-by: Gluster Build System