glusterfs.git/libglusterfs/src/gfdb, branch v3.7.4

cluster/tier: fix 64 bit issue with sql query using times

2015-08-13T16:08:43+00:00

This is a backport of 11885.

> We overflowed when converting seconds to usecs in preperation for
> sql queries. The fix uses uint64_t throughout including subexpressions.

> Change-Id: I59bdb742197400dede97f54735b52030920b0d19
> BUG: 1231268
> Signed-off-by: Dan Lambright 

Change-Id: Iaef7083dac7a162080f4bc23e29dedfe700fa342
BUG: 1252903
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/11913
Tested-by: NetBSD Build System 
Reviewed-by: Joseph Fernandes

libgfdb/ctr: Fixing rename or update of link

2015-08-12T13:46:53+00:00

Backport of http://review.gluster.org/11584 to 3.7.2

Fixing rename or updating of link in
1) libgfdb : handling return from updatelink
2) ctr xlator: correcting freeing of oldbase name in
               hardlink list in the update link

>Change-Id: I1789c492b81876123cd456051a9ae495e876ad46
>BUG: 1241054
>Signed-off-by: Joseph Fernandes 
>Reviewed-on: http://review.gluster.org/11584
>Tested-by: Joseph Fernandes
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 
>Tested-by: Gluster Build System 
Signed-off-by: Joseph Fernandes 

Change-Id: I6da86c06e390f62fe8a44ade7b1efda5771e9ca8
BUG: 1242536
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/11649
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

libgfdb/sql: Fixing broken query of find_unchanged

2015-07-18T13:46:05+00:00

 The find_unchanged query should be

          "write_heat <= defined_heat" AND "read_heat <= defined_heat"
   and not
          "write_heat <= defined_heat" OR "read_heat <= defined_heat"

>>Backport of http://review.gluster.org/#/c/11577/
>>Change-Id: Ie82e02aafbb7ea14563007307de3350ea022049a
>>BUG: 1240970
>>Signed-off-by: Joseph Fernandes 

Change-Id: Ie5d06fa2e3a3552741d2e6bd76807a69808c3d65
BUG: 1241776
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/11608
Reviewed-by: Dan Lambright 
Reviewed-by: Pamela Ousley 
Tested-by: Gluster Build System 
Tested-by: Dan Lambright

tier/ctr: Ignore creation of T file and Ctr Lookup heal improvememnts

2015-06-28T01:11:00+00:00

This is a back port of 11334

1) Ignore creation of T file in ctr_mknod
2) Ignore lookup for T file in ctr_lookup
3) Ctr_lookup:
    a. If the gfid and pgfid in empty dont record
    b. Decreased log level for multiple heal attempts
    c. Inode/File heal happens after an expiry period, which is configurable.
    d. Hardlink heal happens after an expiry period, which is configurable.

> Change-Id: Id8eb5092e78beaec22d05f5283645081619e2452
> BUG: 1235269
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/11334
> Tested-by: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

Change-Id: Ia28a5cf975e41d318906f707deca447aaa35630f
BUG: 1236288
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/11446
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
Tested-by: Gluster Build System

gfdb/libglusterfs : Port log messages to a new framework

2015-06-26T12:58:08+00:00

        Backport of http://review.gluster.org/#/c/10819/

Cherry picked from 2847b03a38515a625e36d52c7e0a1d37107d72ba
>Change-Id: I61874561fdf2c175d2b3eec0c85c25f12dc60552
>BUG: 1194640
>Signed-off-by: Mohamed Ashiq 
>Reviewed-on: http://review.gluster.org/10819
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Joseph Fernandes

Change-Id: I61874561fdf2c175d2b3eec0c85c25f12dc60552
BUG: 1217722
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/11284
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/tier.t: Fixing tier.t crash in regression runs

2015-05-30T15:07:35+00:00

1) If the database file exists
   a. Dont try re-creating the db schema
   b. Dont try re-configuring the db.

2) Dont assert in fini_db () when connection is NULL

>> Signed-off-by: Joseph Fernandes 
>> Reviewed-on: http://review.gluster.org/10870
>> Tested-by: NetBSD Build System
>> Tested-by: Gluster Build System 
>> Reviewed-by: Vijay Bellur 
>> Signed-off-by: Joseph Fernandes 
>>Change-Id: Idd2833f55caaa6b3a77d935d877d6c4d2994da6a

Change-Id: Ia2e044fce4e41bccc8fdada1cb21f240fdbd55df
BUG: 1225077
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/10992
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

CTR/Libgfdb: Log typo fix

2015-05-10T09:30:18+00:00

Log typo fix for CTR Xlator and Libgfdb

Change-Id: Ia39069a5ce9c48bbee937f1b5c5d749a30c9ac56
BUG: 1220100
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/10742
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System

ctr/xlator: Named lookup heal of pre-existing files, before ctr was ON.

2015-05-08T07:34:30+00:00

Problem: The CTR xlator records file meta (heat/hardlinks)
into the data. This works fine for files which are created
after ctr xlator is switched ON. But for files which were
created before CTR xlator is ON, CTR xlator is not able to
record either of the meta i.e heat or hardlinks. Thus making
those files immune to promotions/demotions.

Solution: The solution that is implemented in this patch is
do ctr-db heal of all those pre-existent files, using named lookup.
For this purpose we use the inode-xlator context variable option
in gluster.
The inode-xlator context variable for ctr xlator will have the
following,
    a. A Lock for the context variable
    b. A hardlink list: This list represents the successful looked
       up hardlinks.
These are the scenarios when the hardlink list is updated:
1) Named-Lookup: Whenever a named lookup happens on a file, in the
   wind path we copy all required hardlink and inode information to
   ctr_db_record structure, which resides in the frame->local variable.
   We dont update the database in wind. During the unwind, we read the
   information from the ctr_db_record and ,
   Check if the inode context variable is created, if not we create it.
   Check if the hard link is there in the hardlink list.
      If its not there we add it to the list and send a update to the
      database using libgfdb.
      Please note: The database transaction can fail(and we ignore) as there
      already might be a record in the db. This update to the db is to heal
      if its not there.
      If its there in the list we ignore it.
2) Inode Forget: Whenever an inode forget hits we clear the hardlink list in
   the inode context variable and delete the inode context variable.
   Please note: An inode forget may happen for two reason,
   a. when the inode is delete.
   b. the in-memory inode is evicted from the inode table due to cache limits.
3) create: whenever a create happens we create the inode context variable and
   add the hardlink. The database updation is done as usual by ctr.
4) link: whenever a hardlink is created for the inode, we create the inode context
   variable, if not present, and add the hardlink to the list.
5) unlink: whenever a unlink happens we delete the hardlink from the list.
6) mknod: same as create.
7) rename: whenever a rename happens we update the hardlink in list. if the hardlink
   was not present for updation, we add the hardlink to the list.

What is pending:
1) This solution will only work for named lookups.
2) We dont track afr-self-heal/dht-rebalancer traffic for healing.

> http://review.gluster.org/#/c/10370/
> Cherry picked from commit cb11dd91a6cc296e4a3808364077f4eacb810e48
> Change-Id: Ia4bbaf84128ad6ce8c3ddd70bcfa82894c79585f
> BUG: 1212037
> Signed-off-by: Joseph Fernandes 
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/10370
> Reviewed-by: Jeff Darcy 
> Tested-by: Gluster Build System 
> Tested-by: NetBSD Build System
> Reviewed-by: Vijay Bellur 

Change-Id: I367aa46c3f4b8f912248fb8be75866507f2538df
BUG: 1219075
Signed-off-by: Joseph Fernandes 
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/10370
Reviewed-by: Jeff Darcy 
Reviewed-by: Vijay Bellur 
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/10615
Tested-by: Vijay Bellur

ctr/libgfdb: Performance enhancer for unlink/create/rename/link fops

2015-05-07T13:49:34+00:00

1) ctr_link_consistency option for ctr xaltor is provided so that
   the user can choose to switch it on or off.
   /* For link consistency we do a double update i.e mark the link
   * during the wind and during the unwind we update/delete the link.
   * This has a performance hit. We give a choice here whether we need
   * link consistency to be spoton or not using link_consistency flag.
   * This will have only one link update */

2) In delete the wind time recording is moved to unwind path.
   /* Special performance case:
   * Updating wind time in unwind for delete. This is done here
   * as in the wind path we will not know whether its the last
   * link or not. For a last link there is not use to update any
   * wind or unwind time!*/

>   http://review.gluster.org/#/c/10170/
>   Cherry picked from commit 606d9734543208542afcf9df982bf2d560235ef6
>   Change-Id: I209472fb816f939db4a868b97ba053b028f17ea6
>   BUG: 1217786
>   Signed-off-by: Joseph Fernandes 
>   Reviewed-on: http://review.gluster.org/10170
>   Reviewed-by: Dan Lambright 
>   Tested-by: Gluster Build System 
>   Signed-off-by: Joseph Fernandes 

Change-Id: I4a89ef80875f36cff91520f712e1f47fde258a63
BUG: 1219066
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/10170
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/10614
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

build: make contrib/uuid dependency optional

2015-04-10T11:39:52+00:00

On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.

libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.

Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.

Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur