glusterfs.git, branch v5.2

doc: release notes for release 5.2

2018-12-13T02:15:52+00:00

Fixes: bz#1649895
Change-Id: Ic8f59df9d99efafd56c930e30040f046654a1f9e
Signed-off-by: ShyamsundarR

afr: thin-arbiter 2 domain locking and in-memory state

2018-12-12T14:26:42+00:00

2 domain locking + xattrop for write-txn failures:
--------------------------------------------------
- A post-op wound on TA takes AFR_TA_DOM_NOTIFY range lock and
AFR_TA_DOM_MODIFY full lock, does xattrop on TA and releases
AFR_TA_DOM_MODIFY lock and stores in-memory which brick is bad.

- All further write txn failures are handled based on this in-memory
value without querying the TA.

- When shd heals the files, it does so by requesting full lock on
AFR_TA_DOM_NOTIFY domain. Client uses this as a cue (via upcall),
releases AFR_TA_DOM_NOTIFY range lock and invalidates its in-memory
notion of which brick is bad. The next write txn failure is wound on TA
to again update the in-memory state.

- Any incomplete write txns before the AFR_TA_DOM_NOTIFY upcall release
request is got is completed before the lock is released.

- Any write txns got after the release request are maintained in a ta_waitq.

- After the release is complete, the ta_waitq elements are spliced to a
separate queue which is then processed one by one.

- For fops that come in parallel when the in-memory bad brick is still
unknown, only one is wound to TA on wire. The other ones are maintained
in a ta_onwireq which is then processed after we get the response from
TA.

Change-Id: I32c7b61a61776663601ab0040e2f0767eca1fd64
updates: bz#1648205
Signed-off-by: Ravishankar N 
Signed-off-by: Ashish Pandey

features/bitrot: compare the signature with proper length

2018-12-12T12:59:18+00:00

* The scrubber was comparing the checksum of the file that it
  calculated (by reading the file) with the on disk signature
  (stored via xattr) wrongly. It was using strlen to calculate
  the signature, while the actual length of the signature is
  given by the brick. Just use the actual length that the brick
  provides instead of trying to calculate the signature length via
  strlen API.

* In posix, gfid2path was using the same string that contains the
  list of all the xattrs of file to save the value of the gfid2path
  xattr as well. This causes confusion when gfid2path xattr is queried
  by scrubber for getting the actual path of a corrupted file. Use
  separate string to fetch the value of the xattr instead of the string
  that contains the list of xattrs.

Backport of:
 > Patch: https://review.gluster.org/21752/
 > BUG: 1654805
 > Change-Id: I2d664ab524d2b312233476cb35863dde3122e9a9
(cherry picked from commit f77fb6d568616592ab25501c402c140d15235ca9)
 
Change-Id: I2d664ab524d2b312233476cb35863dde3122e9a9
fixes: bz#1654370
Signed-off-by: Raghavendra Bhat

afr: assign gfid during name heal when no 'source' is present.

2018-12-12T12:57:59+00:00

Problem:
If parent dir is in split-brain or has dirty xattrs set, and the file
has gfid missing on one of the bricks, then name heal won't assign the
gfid.

Fix:
Use the brick we select the gfid from as the 'source'.

Note: Problem was found while trying to debug a split-brain issue on
Cynthia Zhou's setup.

fixes: bz#1655545
Change-Id: Id088d4f0fb017aa35122de426654194e581ed742
Reported-by: Cynthia Zhou 
Signed-off-by: Ravishankar N 
(cherry picked from commit 4d58730c0cd6ab5db39aec8a15276f7bd3371b04)

leases: Do not conflict with internal fops

2018-12-12T12:54:48+00:00

Internal fops (with frame->root->pid < 0) are used to heal
or move data and maintains data integrity. That is they do not
modify client data which holds the lease. Hence no need to recall
Lease for such fops.

Note: Like for locks, we would need rebalance and self-heal
daemon process to heal lease state as well.

Change-Id: I8988693fef8d00e17c19dcc842e2238f9eb5ab48
updates: bz#1651323
Signed-off-by: Soumya Koduri 
(cherry picked from commit 080aa5b9e9d998552e23f7c33aed3afb0ca93c34)

geo-rep: Fix traceback with symlink metadata sync

2018-12-12T12:54:37+00:00

While syncing metadata, 'os.chmod', 'os.chown',
'os.utime' should be used without de-reference.
But python supports only 'os.chown' without
de-reference. That's mostly because Linux
doesn't support 'chmod' on symlink file itself
but it does support 'chown'.

So while syncing metadata ops, if it's symlink
we should only sync 'chown' and not do 'chmod'
and 'utime'. It will lead to tracebacks with
errors like EROFS, EPERM, ACCESS, ENOENT.
All the three errors (EPERM, ACCESS, ENOENT)
were handled except EROFS. But the way it was
handled was not fool proof. The operation is
tried and failure was handled based on the errors.
All the errors with symlink file for 'chown',
'utime' had to be passed to safe errors list of
'errno_wrap'. This patch handles it better by
avoiding 'chmod' and 'utime' if it's symlink
file.
Backport of :
< Patch: https://review.gluster.org/21546/
> BUG: bz#1646104
> Change-Id: Ic354206455cdc7ab2a87d741d81f4efe1f19d77d
> Signed-off-by: Kotresh HR 
(cherry picked from commit 3c6cf9a4a1b46cab2dc53c1ee0afca0fe993102e)


fixes: bz#1654115
Change-Id: Ic354206455cdc7ab2a87d741d81f4efe1f19d77d
Signed-off-by: Kotresh HR

gfapi: Offload callback notifications to synctask

2018-12-12T12:53:23+00:00

Upcall notifications are received from server via epoll
and same thread is used to forward these notifications
to the application. This may lead to deadlock and hang
in the following scenario.

Consider if as part of handling these callbacks,
application has to do some operations which involve
sending I/Os to gfapi stack which inturn have to wait for
epoll threads to receive repsonse. Thus this may lead to
deadlock if all the epoll threads are waiting to complete
these callback notifications.

To address it, instead of using epoll thread itself,
make use of synctask to send those notificaitons to the
application.

Change-Id: If614e0d09246e4279b9d1f40d883a32a39c8fd90
updates: bz#1651323
Signed-off-by: Soumya Koduri 
(cherry picked from commit ad35193718a99494ab1b852ca4cbdf054f73de88)

lease: Treat unlk request as noop if lease not found

2018-12-12T12:53:10+00:00

When the glusterfs server recalls the lease, it expects
client to flush data and unlock the lease. If not it sets
a timer (starting from the time it sends RECALL request) and post
timeout, it revokes it.

Here we could have a race where in client did send UNLK
lease request but because of network delay it may have reached
after server revokes it. To handle such situations, treat
such requests as noop and return sucesss.

Change-Id: I166402d10273f4f115ff04030ecbc14676a01663
updates: bz#1651323
Signed-off-by: Soumya Koduri 
(cherry picked from commit c2e758b54d8a3f778e3e63db0000bb8b63de9b25)

geo-rep: Fix permissions with non-root setup

2018-11-29T15:35:41+00:00

Problem:
In non-root fail-over/fail-back(FO/FB), when slave is
promoted as master, the session goes to 'Faulty'

Cause:
The command 'gluster-mountbroker  '
is run as a pre-requisite on slave in non-root setup.
It modifies the permission and group of following required
directories and files recursively

  [1] /var/lib/glusterd/geo-replication
  [2] /var/log/glusterfs/geo-replication-slaves

In a normal setup, this is executed on slave node and hence
doing it recursively is not an issue on [1]. But when original
master becomes slave in non-root during FO/FB, it contains
ssh public keys and modifying permissions on them causes
geo-rep to fail with incorrect permissions.

Fix:
Don't do permission change recursively. Fix permissions for
required files.

Backport of:
 > Patch: https://review.gluster.org/#/c/glusterfs/+/21689/
 > BUG: bz#1651498
 > Change-Id: I68a744644842e3b00abc26c95c06f123aa78361d
 > Signed-off-by: Kotresh HR 
(cherry picked from commit b2776b1ec1ad845ba568c4439bca3b57cc4d2592)


fixes: bz#1654117
Change-Id: I68a744644842e3b00abc26c95c06f123aa78361d
Signed-off-by: Kotresh HR

leases: Fix incorrect inode_ref/unrefs

2018-11-29T15:34:57+00:00

From testing & code-reading, found couple of places where
we incorrectly unref the inode resulting in use_after_free
crash or ref leaks. This patch addresses couple of them.

a) When we try to grant the very first lease for a inode,
inode_ref is taken in __add_lease. This ref should be active
till all the leases granted to that inode are released (i.e,
till lease_cnt > 0). In addition even after lease_cnt becomes '0',
the inode should be active till all the blocked fops are resumed.

Hence release this ref, after resuming all those fops. To avoid
granting new leases while resuming those fops, defined a new boolean
(blocked_fops_resuming) to flag it in the lease_ctx.

b) 'new_lease_inode' which creates new lease_inode_entry and
takes ref on inode, is used while adding that entry to
client_list and recall_list.

Use its counter function '__destroy_lease_inode' which does unref
while removing those entries from those lists.

c) inode ref is also taken when added to timer->data. Unref the same
after processing timer->data.

Change-Id: Ie77c78ff4a971e0d9a66178597fb34faf39205fb
updates: bz#1651323
Signed-off-by: Soumya Koduri 
(cherry picked from commit b7aec05aa965202ab73120acf0da4c32fe0cf16c)