glusterfs.git/libglusterfs/src/mem-pool.c, branch v3.8.10

libglusterfs:Now mempool is added to ctx pool list under a lock

2016-11-28T09:10:12+00:00

mempool is added to ctx pool list without any lock. This can cause undefined
behaviour in case of multithreaded environment.

Fix: modify the list only under ctx->lock

> Reviewed-on: http://review.gluster.org/15842
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Poornima G 
> Reviewed-by: Niels de Vos 
(cherry picked from commit 277008a3a8583ef10cec9e4182960792e56c5c10)

Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba
Signed-off-by: Rajesh Joseph 
BUG: 1397663
Reviewed-on: http://review.gluster.org/15909
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

libglusterfs: add gf_get_mem_type()

2016-10-18T07:00:06+00:00

gfapi needs to provide a function towards applications to free memory
that it allocated. Depending on how the application is compiled/linked,
it could use a different memory allocator than Gluster itself. Therefore
it is not safe for gfapi to request applications to free memory with
'standard' free().

Examples for this are Gluster allocated structures with GF_CALLOC() when
memory accounting is enabled (the default). Some gfapi functions use
malloc() to allocate memory as a workaround, but the free() from the
jemalloc implementation should not be combined with the malloc() from
glibc.

Cherry picked from commit db4e26ed71a01e5f760fbc3c7051962426f102c9:
> Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
> BUG: 1344714
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15108
> Smoke: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 

This also includes commit da2aabe3 which fixes an unused variable
warning.

Change-Id: I626cd1a60abf8965f9263290f4045d1f69fc2093
BUG: 1347717
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15639
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

libglusterfs: fix glusterd statedump crash

2016-08-07T11:53:48+00:00

Backport of http://review.gluster.org/14987

commit 3c04a91 removed setting typeStr to NULL if num_allocs is set to 0, this
has caused this regression. Code has been put back like earlier and to avoid
statedump printing all the NULL values check is modified to see skip the records
if num_allocs is 0 instead of total_allocs

>Reviewed-on: http://review.gluster.org/14987
>NetBSD-regression: NetBSD Build System 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: N Balachandran 
>Reviewed-by: Prashanth Pai 
>Reviewed-by: Jeff Darcy 

Change-Id: Ib8bcc2fba908e88cf52b641c3f6bcba74f5e667c
BUG: 1364329
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15091
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Niels de Vos

statedump: Prevent (null) typestr to be printed

2016-04-24T13:44:44+00:00

Problem:
After the commits: 7e44c783ad731856956929f6614bbe045c26ea3a - lock: use
spinlock only on multicore systems
a6aecae2cd8171b8538bfe5d2800bdd157380b85 - nfs: fix lock variable type

we see a lot of "[global.glusterfs - usage-type (null) memusage]" in statedump
because lock status is not all-zeros after init, and the memcmp to check that
a datatype is never allocated is invalid.

Fix:
Changed if a datatype is allocated or not check based on total_allocs. Also
removed setting typestr to NULL on gf_free even when num_allocs is 0. Because
even that is leading to 'null' memusage string to be printed in statedump.

BUG: 1329870
Change-Id: If2b01a557cbdc787625db32e276e06cee3ac46ee
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14054
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

libglusterfs: coverity fix

2016-04-14T12:55:50+00:00

fix missing varargs cleanup

CID 1124856: string overflow

CID 1124656: NULL return

CID 1124374: constant expression

Change-Id: Iead530c599bdfef05a40c68b892215f4e4f02247
BUG: 789278
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/9630
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

mem-pool,stack,store,syncop,timer/libglusterfs : Porting to a new logging framework

2015-06-27T05:52:01+00:00

Change-Id: Idd3dcaf7eeea5207b3a5210676ce3df64153197f
BUG: 1194640
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/10827
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System

core: use reference counting for mem_acct structures

2015-05-09T13:28:13+00:00

When freeing memory, our memory-accounting code expects to be able to
dereference from the (previously) allocated block to its owning
translator.  However, as we have already found once in option
validation and twice in logging, that translator might itself have
been freed and the dereference attempt causes on of our daemons to
crash with SIGSEGV.  This patch attempts to fix that as follows:

 * We no longer embed a struct mem_acct directly in a struct xlator,
   but instead allocate it separately.

 * Allocated memory blocks now contain a pointer to the mem_acct
   instead of the xlator.

 * The mem_acct structure contains a reference count, manipulated in
   both the normal and translator allocate/free code using atomic
   increments and decrements.

 * Because it's now a separate structure, we can defer freeing the
   mem_acct until its reference count reaches zero (either way).

 * Some unit tests were disabled, because they embedded their own
   copies of the implementation for what they were supposedly testing.
   Life's too short to spend time fixing tests that seem designed to
   impede progress by requiring a certain implementation as well as
   behavior.

Change-Id: Id929b11387927136f78626901729296b6c0d0fd7
BUG: 1211749
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/10417
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Niels de Vos 
Reviewed-by: Pranith Kumar Karampuri

mem-pool: invalidate memory on GF_FREE to aid debugging

2015-03-30T12:37:07+00:00

Debugging where memory gets free'd with help from overwriting the memory
before it is free'd with some structures (repeatedly). The struct
mem_invalid starts with a magic value (0xdeadc0de), followed by a
pointer to the xlator, the mem-type. the size of the GF_?ALLOC()
requested area and the baseaddr pointer to what GF_?ALLOC() returned.

With these details, and the 'struct mem_header' that is placed when
calling GF_?ALLOC(), it is possible to identify overruns and possible
use-after-free. A memory dump (core) or running with a debugger is
needed to read the surrounding memory of corrupt structures.

This additional memory invalidation/poisoning needs to be enabled by
passing --enable-debug to ./configure.

Change-Id: I9f5f37dc4b5b59142adefc90897d32e89be67b82
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10019
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Reviewed-by: Emmanuel Dreyfus 
Reviewed-by: Kaleb KEITHLEY

testing: Switch to cmocka the successor of cmockery2

2015-03-05T21:51:37+00:00

This uses https://cmocka.org/ as the unit testing framework.

With this change, unit testing is made optional as well. We assume there
is no cmocka available while building. cmocka will be enabled by default
later on. For now, to build with cmocka run:

    $ ./configure --enable-cmocka

This change is based on the work of Andreas (replacing cmockery2 with
cmocka) and Kaleb (make cmockery2 an optional build dependency).

The only modifications I made, are additional #defines in unittest.h for
making sure the unit tests function as expected.

Change-Id: Iea4cbcdaf09996b49ffcf3680c76731459cb197e
BUG: 1067059
Merged-change: http://review.gluster.org/9762/
Signed-off-by: Andreas Schneider 
Signed-off-by: Kaleb S. KEITHLEY 
Signed-off-by: Niels de Vos 
Change-Id: Ia2e955481c102d5dce17695a9205395a6030e985
Reviewed-on: http://review.gluster.org/9738
Tested-by: Gluster Build System

glusterd: statedump support

2014-10-15T10:02:40+00:00

Although glusterd currently has statedump support but it doesn't dump its
context information. Implementing glusterd_dump_priv function to export per-node
glusterd information would be useful for debugging bugs. Once implemented, we
could enhance sos-report to fetch this information. This would potentially
reduce our time to root cause and data needed for debugability can be dumped
gradually.

Following is the main items of the dump list targeted in this patch :

    * Supported max/min op-version and current op-version
    * Information about peer list
    * Information about peer list involved while a transaction is going on
      (xaction_peers)
    * option dictionary in glusterd_conf_t
    * mgmt_v3_lock in glusterd_conf_t
    * List of connected clients
    * uuid of glusterd
    * A section of rpc related information like live connections and their
      statistics

There are couple of issues which were found during implementation and testing
phase:
 - xaction_peers of glusterd_conf_t was not initialized in init because of which
   traversing through this list head was crashing when there was no active
   transaction
 - gf_free was not setting the typestr to NULL if the the alloc count becomes 0
   for a mem-type earlier allocated.

Change-Id: Ic9bce2d57682fc1771cd2bc6af0b7316ecbc761f
BUG: 1139682
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/8665
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M