glusterfs.git/tests/basic/quota-anon-fd-nfs.t, branch v4.0dev1

Fixes quota aux mount failure

2017-05-08T06:15:55+00:00

The aux mount is created on the first limit/remove_limit/list command
and it remains until volume is stopped / deleted / (quota is disabled)
, where we do a lazy unmount. If the process is uncleanly terminated,
then the mount entry remains and we get (Transport disconnected) error
on subsequent attempts to run quota list/limit-usage/remove commands.

Second issue, There is also a risk of inadvertent rm -rf on the
/var/run/gluster causing data loss for the user. Ideally, /var/run is
a temp path for application use and should not cause any data loss to
persistent storage.

Solution:
1) unmount the aux mount after each use.
2) clean stale mount before mounting, if any.

One caveat with doing mount/unmount on each command is that we cannot
use same mount point for both list and limit commands.
The reason for this is that list command needs mount to be accessible
in cli after response from glusterd, So it could be unmounted by a
limit command if executed in parallel (had we used same mount point)
Hence we use separate mount points for list and limit commands.

Change-Id: I4f9e39da2ac2b65941399bffb6440db8a6ba59d0
BUG: 1433906
Signed-off-by: Sanoj Unnikrishnan 
Reviewed-on: https://review.gluster.org/16938
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Atin Mukherjee

tests: Added check for NFS export availability to quota-anon-fd-nfs.t

2017-02-22T11:20:20+00:00

Change-Id: I15a9441267c18bb1073d14db325c98fa497f2fb7
BUG: 1425515
Signed-off-by: Shyam 
Reviewed-on: https://review.gluster.org/16701
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: sanoj-unnikrishnan

libglusterfs: make memory pools more thread-friendly

2017-02-02T18:30:19+00:00

Early multiplexing tests revealed *massive* contention on certain
pools' global locks - especially for dictionaries and secondarily for
call stubs.  For the thread counts that multiplexing can create, a
more lock-free solution is clearly needed.  Also, the current mem-pool
implementation does a poor job releasing memory back to the system,
artificially inflating memory usage to match whatever the worst case
was since the process started.  This is bad in general, but especially
so for multiplexing where there are more pools and a major point of
the whole exercise is to reduce memory consumption.

The basic ideas for the new design are these

  There is one pool, globally, for each power-of-two size range.
  Every attempt to create a new pool within this range will instead
  add a reference to the existing pool.

  Instead of adding pools for each translator within each multiplexed
  brick (potentially infinite and quite possibly thousands), we
  allocate one set of size-based pools per *thread* (hundreds at
  worst).

  Each per-thread pool is divided into hot and cold lists.  Every
  allocation first attempts to use the hot list, then the cold list.
  When objects are freed, they always go on the hot list.

  There is one global "pool sweeper" thread, which periodically
  reclaims everything in each pool's cold list and then "demotes" the
  current hot list to be the new cold list.

  For normal allocation activity, only a per-thread lock need be
  taken, and even that only to guard against very rare contention from
  the pool sweeper.  When threads start and stop, a global lock must
  be taken to add them to the pool sweeper's list.  Lock contention is
  therefore extremely low, and the hot/cold lists also provide good
  locality.

A more complete explanation (of a similar earlier design) can be found
here:

 http://www.gluster.org/pipermail/gluster-devel/2016-October/051160.html

Change-Id: I5bc8a1ba57cfb553998f979a498886e0d006e665
BUG: 1385758
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/15645
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

glusterd: default value of nfs.disable, change from false to true

2016-04-27T07:59:30+00:00

Next step in eventual deprecation of glusterfs nfs server in favor
of ganesha.nfsd.

Also replace several open-coded strings with constant.

Change-Id: If52f5e880191a14fd38e69b70a32b0300dd93a50
BUG: 1092414
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13738
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Tested-by: Atin Mukherjee 
Reviewed-by: Niels de Vos

tests: Add mechanism for disabled tests

2016-03-10T07:19:40+00:00

Requirements:
Should be able to skip tests from run-tests.sh run.
Should be granular enough to disable on subset of OSes.

Solution:
Tests can have special comment lines with some comma separated values
within them.

Key names used to determine test status are
G_TESTDEF_TEST_STATUS_CENTOS6
G_TESTDEF_TEST_STATUS_NETBSD7
Some examples:
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
You can change status of test to enabled or delete the line only if all the
bugs are closed or modified or if the patch fixes it.

Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
BUG: 1295704
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/13393
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Kaushal M

Quota: fix testcases not to send parallel writes for accurate

2015-05-25T18:34:22+00:00

quota enforcement

Currently quota enforcer doesn't consider parallel writes
and allows quota to exceed limit where there are high rate
of parallel writes. Bug# 1223658 tracks the issue.

This patch fixes the spurious failures by not sending
parallel writes.
Using O_SYNC and O_APPEND flags and block size
not more that 256k (For higher block size NFS client
splits the block into 256k chinks and does parallel writes)

Change-Id: I297c164b030cecb87ce5b494c02b09e8b073b276
BUG: 1223798
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/10878
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

tests: Check aux umount is unmounted for quota tests

2015-05-04T08:23:28+00:00

Change-Id: If57d08f3446755ea41f66ca258efcc8ea5a89063
BUG: 1217701
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10480
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

test : fix for quota-anon-fd-nfs.t spurious failure

2015-04-01T06:42:51+00:00

quota with nfs sometimes fails to get parent
with nameless lookups. This is fixed with patch# 9478
Hence, adding nfs test-case back again.

Change-Id: I9ede7c7dae4604aa7fa03ce7c36f46a9879b7479
BUG: 1163543
Signed-off-by: Sachin Pandit 
Reviewed-on: http://review.gluster.org/9381
Tested-by: Gluster Build System 
Reviewed-by: Vijaikumar Mallikarjuna 
Reviewed-by: Vijay Bellur

test : removing quota-anon-fd-nfs.t for time being.

2015-01-07T06:19:45+00:00

I am removing quota-anon-fd-nfs.t for time being as it is causing
lots of regression failure. Meanwhile I will be working in parallel
to find out the root cause for this spurious failure in another patch
(http://review.gluster.org/#/c/9381/). I'll revert this patch once a proper fix is found.

Change-Id: I0ef205241219ed3210d58d2b487f1b00292115f0
BUG: 1163543
Signed-off-by: Sachin Pandit 
Reviewed-on: http://review.gluster.org/9390
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

quota: For a rename operation, do quota_check_limit only till the

2014-12-27T13:52:07+00:00

common ancestor of src and dst file

Example:
set quota limit set to 1GB on /
create a file /a1/b1/file1 of 600MB
mv /a1/b1/file1 /a1/b1/file2
This rename fails as it takes delta into account which sums up to 1.2BG.
Though we are not creating new file, we still get quota exceeded error.
So quota enforce should happen only till b1.

Similarly:
mv /a/b/c/file /a/b/x/y/file
quota enforce should happen only till dir 'b'

Change-Id: Ia1e5363da876c3d71bd424e67a8bb28b7ac1c7c1
BUG: 1153964
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/8940
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G