| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As part of a larger effort to consolidate hash functions
and use a faster, with less collissions, I've moved to use
xxhash as the hashing function for the dictionary.
Note: I'm using only 32 bits of the hashes, as the hash field
is currently a 32 bit variable. There's room (since the struct
is not packed) to move to use a 64bit variable. Donno how it'll
affect performance.
Only compile-tested!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ie244efd1368234e54dd5957ee1ae5bebd7d9409a
|
|
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In a previous patch (https://review.gluster.org/20769) we've
added the key length to be passed to dict_* funcs, to remove the need
to strlen() it. This patch moves some xlators to use it.
- It also adds dict_get_int32n which was missing.
- It also reduces the size of some key variables.
They were set to 1024b or PATH_MAX, where sometimes 64 bytes were
really enough.
Please review carefully:
1. That I did not reduce some the size of the key variables too much.
2. That I did not mix up some keys.
Compile-tested only!
Change-Id: Ic729baf179f40e8d02bc2350491d4bb9b6934266
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. We don't need a temp. string to strotol(), we can use data->data.
2. We don't need the gf_asprintf, when a regular asprintf would do.
3. No need to CALLOC the data, can MALLOC, since
we are going to populate most of the fields anyway.
4. Removed is_const member - it was not used.
5. Turned is_static to a gf_boolean_t
6. Turned another get_mem0 to get_mem (in dict_set_lk() func)
7. Remove dict_set_ptr - was exactly the same as dict_set_static_ptr
(apart from the 'static' bit) and no one was using it.
8. Removed is_static from the dictionary object. Did not seem to be
very useful.
Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I7b1f6320867b154be4a94033819407f7e51cf38c
|
|
|
|
|
|
| |
Fixes: bz#1625089
Change-Id: Ie56df0da46c242846a1ba51ccb9e011af118b119
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
We don't want to strlen NULL strings.
Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Id795998693ba410dbd80df962d48956a9e4164c4
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I suspect we've allocated and then set to '\0' an
additional byte for no reason: gf_asprintf() allocates
enough inc. the terminating null; data->len took that
additional byte into account; memcpy() it back
took care of that terminating null byte as well.
Also, unrelated, implemented str_to_data() via
strn_to_data().
Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I32c90e12974f51a0fbc0cfaebf9bd0fd722adc11
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
others.
They all take as a parameter the key length, instead of strlen() it.
In most cases, we know the key length, we just never bothered to save and pass it along.
(We most likely sprintf'ed it earlier and the return value could have been used).
A more interesting addition is dict_set_nstrn() [horrible name. Ideas are welcome].
It accepts both the string length and the key length and avoids strlen() both.
Some of it can be calculated on compile-time, btw.
For example:
dict_set_str (dict, "key", "all");
Should become:
dict_set_nstrn (dict, "key", sizeof ("key"), "all", sizeof ("all"));
Compile-tested only!
Change-Id: Ic2667f445f6c2e22e279505f5ad435788b4b668c
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
If possible, perform the key hash calculation not under
the dictionary lock.
Compile-tested only!
Change-Id: I88b6ed4eff6660d8334bc08ca4eebf61960e3ca1
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
| |
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I3f210ae236f778fd0e8e7a17d01da21f65cb4e4b
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the output of sprintf().
Also, constify and initialize parameters properly.
Only compile-tested!
Change-Id: I33870afbe14bc1292fb08115e878d48c00a43bee
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
possible
It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.
In some cases, also changed allocation size to be sizeof some
struct or type instead of a pointer - easier to read.
In some cases, removed redundant strlen() calls by saving the result
into a variable.
1. Only done for the straightforward cases. There's room for improvement.
2. Please review carefully, especially for string allocation, with the
terminating NULL string.
Only compile-tested!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Iea9098b9ec0a82a866fbc5836514b1b317daefa1
|
|
|
|
|
|
|
|
|
| |
The hash should be done outside of locks, whenever is possible.
Change-Id: I4f8f7455702e0489a57105cf79668c7fca90e1c0
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
| |
Change-Id: Ifda14a1cc01e0fa99ff20d710eccb43296e22955
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Fixing FORWARD_NULL coverify errors in libglusterfs.
CID: 1391407, 1391410
BUG: 789278
Change-Id: I3d20523005e4418759c8a72620edff7c977d2d00
updates: bz#789278
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...
Only compile-tested!
Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since setxattr and removexattr fops cbk do not carry poststat,
the stat cache was being invalidated in setxatr/remoxattr cbk.
Hence the further lookup wouldn't be served from cache.
To prevent this invalidation, md-cache is modified to get
the poststat in set/removexattr_cbk in dict.
Co-authored with Xavi Hernandez.
Change-Id: I6b946be2d20b807e2578825743c25ba5927a60b4
fixes: bz#1586018
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Function are_dicts_equal() had a bug when the first argument was NULL and
the second one wasn't NULL. In this case it incorrectly returned that the
dicts were different when they could be equal.
Fixes: bz#1566732
Change-Id: I0fc245c2e7d1395865a76405dbd05e5d34db3273
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: if a lookup is done on a newly added brick for a path on which limit
has been reached, the lookup fails to heal the directory tree due to quota.
Solution: Tag the lookup as an internal fop and ignore it in quota.
Since marking internal fop does not usually give enough contextual information.
Introducing new flags to pass the contextual info.
Adding dict_check_flag and dict_set_flag to aid flag operations.
A flag is a single bit in a bit array (currently limited to 256 bits).
Change-Id: Ifb6a68bcaffedd425dd0f01f7db24edd5394c095
fixes: bz#1505355
BUG: 1505355
Signed-off-by: Sanoj Unnikrishnan <sunnikri@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this patchset, some major things are changed in XDR, mainly:
* Naming: Instead of gfs3/gfs4 settle for gfx_ for xdr structures
* add iattx as a separate structure, and add conversion methods
* the *_rsp structure is now changed, and is also reduced in number
(ie, no need for different strucutes if it is similar to other response).
* use proper XDR methods for sending dict on wire.
Also, with the change of xdr structure, there are changes needed
outside of xlator protocol layer to handle these properly. Mainly
because the abstraction was broken to support 0-copy RDMA with payload
for write and read FOP. This made transport layer know about the xdr
payload, hence with the change of xdr payload structure, transport layer
needed to know about the change.
Updates #384
Change-Id: I1448fbe9deab0a1b06cb8351f2f37488cefe461f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This new type helps to avoid excessive logs. It should be
set only in case of
* volume graph building (graph.y)
* dict unserialize
(happens once a dictionary is received on wire in old protocol)
All other dict set and get should have proper check and warning
logs if there is a mismatch.
updates #220
Change-Id: I1cccb304a877aa80c07aaac95f10f5005e35b9c5
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Couple of instances doesn't pass enough number of parameters to the
function resulting compilation to fail.
Updates #203
Change-Id: Id8caa6fe7fc611645ad7ff11d81a2462e4ec6bab
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Printing the key name makes it easier for developers
to figure out which keys have dict data type mismatches.
Updates #337
Change-Id: I21d9a22488a4c5e5a8d991ca2d53f1e3039f7685
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Added 2 more types which are present in gluster codebase, mainly
IATT and UUID.
Updates #203
Change-Id: Ib6d6d6aefb88c3494fbf93dcbe08d9979484968f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch brings data type awareness to dictionary,
and also makes sure valid data is properly sent to the
other side of the wire using XDR.
Next step is to allow people to add more data types
(for example, Bool, UUID, iatt etc), and then make
it part of every fop signature in wire.
Fixes #203
Change-Id: Ie0eee2db847bea2bf7dad80dec89ce3e7c5917c1
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: In glusterfs code base we call mutex_lock/unlock to take
reference/dereference for a object.Sometime it could be
reason for lock contention also.
Solution: There is no need to use mutex to increase/decrease ref
counter, instead of using mutex use gcc builtin ATOMIC
operation.
Test: I have not observed yet how much performance gain after apply
this patch specific to glusterfs but i have tested same
with below small program(mutex and atomic both) and
get good difference.
static int numOuterLoops;
static void *
threadFunc(void *arg)
{
int j;
for (j = 0; j < numOuterLoops; j++) {
__atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL);
}
return NULL;
}
int
main(int argc, char *argv[])
{
int opt, s, j;
int numThreads;
pthread_t *thread;
int verbose;
int64_t n = 0;
if (argc < 2 ) {
printf(" Please provide 2 args Num of threads && Outer Loop\n");
exit (-1);
}
numThreads = atoi(argv[1]);
numOuterLoops = atoi (argv[2]);
if (1) {
printf("\tthreads: %d; outer loops: %d;\n",
numThreads, numOuterLoops);
}
thread = calloc(numThreads, sizeof(pthread_t));
if (thread == NULL) {
printf ("calloc error so exit\n");
exit (-1);
}
__atomic_store (&glob, &n, __ATOMIC_RELEASE);
for (j = 0; j < numThreads; j++) {
s = pthread_create(&thread[j], NULL, threadFunc, NULL);
if (s != 0) {
printf ("pthread_create failed so exit\n");
exit (-1);
}
}
for (j = 0; j < numThreads; j++) {
s = pthread_join(thread[j], NULL);
if (s != 0) {
printf ("pthread_join failed so exit\n");
exit (-1);
}
}
printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED));
exit(0);
}
time ./thr_count 800 800000
threads: 800; outer loops: 800000;
glob value is 640000000
real 1m10.288s
user 0m57.269s
sys 3m31.565s
time ./thr_count_atomic 800 800000
threads: 800; outer loops: 800000;
glob value is 640000000
real 0m20.313s
user 1m20.558s
sys 0m0.028
Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes issues 230,592,593,110,63 from [1]
[1] https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-10-30-9aa574a5/html/
Note: Resolve FORWARD_NULL coverity issue in glusterfs_ctx_new is
also fixed with this patch.
BUG: 789278
Change-Id: Ic4199a144a14cc9ead7366fb1c9699197141bc86
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Existing EC code updates the xattr on the subvolume
in a sequential pattern resulting in very poor performance.
With this fix EC now updates the xattr on the subvolume
in parallel which improves the xattr update performance.
BUG: 1445663
Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.
Fix: Use dict_get_with_ref to retrieve the file contents.
Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1484709
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/18115
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Issue:
The caller of glfs_setxattr sends a buffer to set as the value.
We create a dict in which the pointer to the value is set.
Underlying layers like md-cache take a ref on this dict to store
the value for a longer time. But the moment setxattr is complete,
the caller of glfs_setxattr can free the value memory.
Solution:
memcpy the setxattr value to the gluster buffer.
Change-Id: I58753fe702e8b7d0f6c4f058714c65d0ad5d7a0a
BUG: 1477488
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: https://review.gluster.org/17967
Reviewed-by: soumya k <skoduri@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Bulk xattr removal doesn't check if the xattrs that are coming in xdata
have gfid/volume-id xattrs, so there is potential for bulkremovexattr
removing gfid/volume-id.
I also observed that bulkremovexattr is not available for fremovexattr.
Fix:
Do proper checks in bulk removexattr to remove gfid/volume-id.
Refactor [f]removexattr to reduce the differences.
BUG: 1470489
Change-Id: Ia845b31846a149500111c0996646e648f72cdce6
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17765
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
when you take the 'statedump', it shows the output like below
-----
[dict]
max-number-of-dict-pairs=13
total-pairs-used=41613
total-dict-used=12629
average-pairs-per-dict=3
------
Updates #220
Change-Id: I71a7eda3a3cd23edf4483234f22f983923bbb081
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: https://review.gluster.org/4035
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates #220
Change-Id: I03b1d2fac2dfcdd21bdf4e4fff19d49425699931
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: https://review.gluster.org/6450
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The dict_rename_key() function will be used for converting the
"security.selinux" xattr to "trusted.gluster.selinux" in the upcoming
SELinux xlator.
BUG: 1318100
Change-Id: Ic5d0b9127e2c360d355f02e200a820597e83fa2c
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
[ndevos: split from change Id8916bd8e064ccf74ba86225ead95f86dc5a1a25]
Reviewed-on: https://review.gluster.org/16616
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and statedump too. Also "const char *" (versus just "char *") for the
fmt param.
Change-Id: Ic63734a673208a2cd49aebccce7659816e6179e3
BUG: 1399196
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: https://review.gluster.org/15881
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Issue:
md-cache caches a specified list of xattrs, and when cache invalidation
is enabled, it makes sense to recieve invalidation only when those xattrs
are modified by other clients. But the current implementation of upcall
is that, it will send invalidation when any of the on-disk xattrs is modified.
Solution:
md-cache sends a list of xattrs that it is interested in, to upcall by
issuing an ipc(). The challenge here is to make sure everytime a brick
goes offline and comes back up, the ipc() needs to be issued to the
bricks. Hence ipc() is sent from md-cache every time there is a
CHILD_UP/CHILD_MODIFIED event.
TODO:
There will be patches following, in cluster xlators, to implement ipc fop.
Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d
BUG: 1211863
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/15002
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Once dynstr is set into a dict by function dict_set_dynstr, its free
operation will be called by this dict when the dict is destroyed.
Signed-off-by: Zhou Zhengping <johnzzpcrystal@gmail.com>
Change-Id: Idd2bd19a041bcb477e1c897428ca1740fb75c5f3
BUG: 1354141
Reviewed-on: http://review.gluster.org/14882
Tested-by: Zhou Zhengping <johnzzpcrystal@gmail.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fix missing varargs cleanup
CID 1124856: string overflow
CID 1124656: NULL return
CID 1124374: constant expression
Change-Id: Iead530c599bdfef05a40c68b892215f4e4f02247
BUG: 789278
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/9630
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Earlier, when the database was queried we used to save
all the queried records in an ASCII format in the query file.
This caused issues like filename having ASCII delimiter and used
to take a lot of space. The tier.c file also had a lot of parsing code.
Here we changed the format of the query file to binary.
All the logic of serialization and formating of query record is done
by libgfdb. Libgfdb provides API,
gfdb_write_query_record() and gfdb_read_query_record(),
which the user i.e tier migrator and CTR xlator can use to
write to and read from query file.
With this binary format we save on disk space i.e reduce to 50% atleast
as we are saving GFID's in binary format 16 bytes and not the string format
which takes 36 bytes + We are not saving path of the file + we are also saving on
ASCII delimiters.
The on disk format of query record is as follows,
+---------------------------------------------------------------------------+
| Length of serialized query record | Serialized Query Record |
+---------------------------------------------------------------------------+
4 bytes Length of serialized query record
|
|
-------------------------------------------------|
|
|
V
Serialized Query Record Format:
+---------------------------------------------------------------------------+
| GFID | Link count | <LINK INFO> |..... | FOOTER |
+---------------------------------------------------------------------------+
16 B 4 B Link Length 4 B
| |
| |
-----------------------------| |
| |
| |
V |
Each <Link Info> will be serialized as |
+-----------------------------------------------+ |
| PGID | BASE_NAME_LENGTH | BASE_NAME | |
+-----------------------------------------------+ |
16 B 4 B BASE_NAME_LENGTH |
|
|
------------------------------------------------------------------------|
|
|
V
FOOTER is a magic number 0xBAADF00D indicating the end of the record.
This also serves as a serialized schema validator.
Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a
BUG: 1272207
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/12354
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dict_set_bin() is handling the pointer that it passed inconsistently.
Depending on the errors that can occur, the pointer passed to the dict
can be free'd, but there is no guarantee.
It is cleaner to have the caller free the pointer that allocated it and
dict_set_bin() returned an error. When dict_set_bin() returned success,
the given pointer will be free'd when dict_unref() calls data_destroy().
Many callers of dict_set_bin() already take care of free'ing the pointer
on error. The ones that did not, are corrected with this change too.
Change-Id: I39a4f7ebc0cae6d403baba99307d7ce408f25966
BUG: 1242280
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11638
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Copy/pasting complete functions for setting one boolean is rather ugly,
this can easily be solved with a helper function that gets the boolean
as an extra parameter.
Also removing dict_set_dynmstr() because there are no callers, and using
GF_MALLOC() or GF_CALLOC() to allocate memory for strings is much more
preferred.
And, as Kaleb pointed out, prefixing functions with an underscore '_' is
reserved for libc internals. Renaming the _dict_*() functions for that
reason.
Change-Id: I8e681954502f17e59c8f5e8a294d3db238d3e5f3
BUG: 1232304
Reported-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11716
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch saves cpu cyles by deleting duplicated code in most frequently
used functions 'dict_set_static_bin' & 'dict_set_bin' which is already
repeated/executed in function 'bin_to_data'
Change-Id: I8e18b3f2b321e06bb6bd5a67e1782cb159182dd2
BUG: 1232304
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/11255
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
logging framework
Change-Id: Ie7d180e0ab2fed1270d66504606d1b2522884020
BUG: 1194640
Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com>
Reviewed-on: http://review.gluster.org/10828
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not. value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.
geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.
Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1207712
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/10078
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the "Signer" -- responsible for signing files with their
checksums upon last file descriptor close (last release()).
The event notification facility provided by the changelog xlator
is made use of.
Moreover, checksums are as of now SHA256 hash of the object data
and is the only available hash at this point of time. Therefore,
there is no special "what hash to use" type check, although it's
does not take much to add various hashing algorithms to sign
objects with. Signatures are stored in extended attributes of the
objects along with the the type of hashing used to calculate the
signature. This makes thing future proof when other hash types
are added. The signature infrastructure is provided by bitrot
stub: a little piece of code that sits over the POSIX xlator
providing interfaces to "get or set" objects signature and it's
staleness.
Since objects are signed upon receiving release() notification,
pre-existing data which are "never" modified would never be
signed. To counter this, an initial crawler thread is spawned
The crawler scans the entire brick for objects that are unsigned
or "missed" signing due to the server going offline (node reboots,
crashes, etc..) and triggers an explicit sign. This would also
sign objects when bit-rot is enabled for a volume and/or after
upgrade.
Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b
BUG: 1170075
Original-Author: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9711
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The tier translator shares most of DHT's code. It differs in how
subvolumes are chosen for I/Os, and how file migration (cache promotion
and demotion) is managed. That different functionality is split to either
DHT or tier logic according to the "tier_methods" structure.
A cache promotion and demotion thread is created in a manner
similar to the rebalance daemon. The thread operates a timing
wheel which periodically checks for promotion and demotion candidates
(files). Candidates are queued and then migrated. Candidates must exist on
the same node as the daemon and meet other critera per caching policies.
This patch has two authors (Dan Lambright and Joseph Fernandes). Dan
did the DHT changes and Joe wrote the cache policies. The fix depends on
DHT readidr changes and the database library which have been submitted
separately. Header files in libglusterfs/src/gfdb should be reviewed in
patch 9683.
For more background and design see the feature page [1].
[1]
http://www.gluster.org/community/documentation/index.php/Features/data-classification
Change-Id: Icc26c517ccecf5c42aef039f5b9c6f7afe83e46c
BUG: 1194753
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/9724
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
This enables trusted.ec.dirty to be tracked in index
Change-Id: Ief1619110859f6f9ccee3da229f0688b73e2124b
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9602
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently the dict is created with hash size 1, i.e. there is
only one hash bucket and the calculation of hash decomes redundant.
Change-Id: Id70aea0d798902494ebb6d82955d97d591bc73d2
BUG: 789278
Signed-off-by: Poornima Gurusiddaiah <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/8211
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Iaa3454f7f3b6516660b1976bea63e39ea7795f8f
BUG: 1164051
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9121
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|