glusterfs.git/rpc/rpc-lib/src/protocol-common.h, branch v3.7.16

glusterd: try to connect on GF_PMAP_PORT_FOREIGN aswell

2016-05-02T03:53:04+00:00

This patch fix couple of things mentioned below:

1. previously we use to try to connect on only GF_PMAP_PORT_FREE
in the pmap_registry_alloc(), it could happen that some foreign process
would have freed the port by this time ?, hence it is worth giving a try on
GF_PMAP_PORT_FOREIGN ports as well instead of wasting them all.

2. fix pmap_registry_remove() to mark the port asGF_PMAP_PORT_FREE

3. added useful comments on gf_pmap_port_type enum members

Backport of:
> Change-Id: Id2aa7ad55e76ae3fdece21bed15792525ae33fe1
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14080
> Tested-by: Prasanna Kumar Kalever 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

Change-Id: Ib7852aa1c55611e81c78341aace4d374d516f439
BUG: 1323564
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14117
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

afr: add mtime based split-brain resolution to CLI

2016-04-04T17:33:56+00:00

Backport of http://review.gluster.org/#/c/13828/

Extended the CLI to include support for split-brain resolution based on
mtime. The command syntax is:

$:gluster volume heal  split-brain latest-mtime 

where  can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file.

Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f
BUG: 1321748
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13838
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Anuradha Talur 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterd: cli command implementation for bitrot scrub status

2015-11-23T03:53:58+00:00

This patch is backport of: http://review.gluster.org/10231

CLI command  for bitrot scrub status will be :

gluster volume bitrot  scrub status

Above command will show the statistics of bitrot scrubber.

Upon execution of this command it will show some common
scrubber tunable value of volume  followed by
statistics of scrubber statistics of individual nodes.

sample ouput for single node:

Volume name : 

State of scrub: Active

Scrub frequency: biweekly

Bitrot error log location: /var/log/glusterfs/bitd.log

Scrubber error log location: /var/log/glusterfs/scrub.log

=========================================================

Node name:

Number of Scrubbed files:

Number of Unsigned files:

Last completed scrub time:

Duration of last scrub:

Error count:

=========================================================

This is just infrastructure. list of bad file, last scrub
time, error count value will be taken care by
http://review.gluster.org/#/c/12503/  and
http://review.gluster.org/#/c/12654/ patches.

    >> Change-Id: I3ed3c7057c9d0c894233f4079a7f185d90c202d1
    >> BUG: 1207627
    >> Signed-off-by: Gaurav Kumar Garg 
    >> Reviewed-on: http://review.gluster.org/10231
    >> Reviewed-by: Atin Mukherjee 
    >> Tested-by: NetBSD Build System 
    >> Tested-by: Gluster Build System 

Change-Id: I45ed94e5e0e78a1e007c30eb0b252f74cf3c9187
BUG: 1283881
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/12704
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/tier: add gluster v tier

2015-09-10T17:59:11+00:00

This is a backport of 11984.

> Currently the tier feature piggy backs off the rebalance command
> syntax to obtain status and this is clumsy. Introduce a new
> tier command that can do tier specific operations, starting
> with volume status to display counters.

> Old commands:
> gluster volume attach-tier  [replica count] {bricklist..}
> gluster volume detach-tier  {start|stop|commit}

> New commands:
> gluster volume tier  attach [replica count] {bricklist} |
>                           detach {start|stop|commit} |
>                           status

> Change-Id: Ic07b3c6260588162de7d34380f8cbd3d8a7f35d3
> BUG: 1255693
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/11984
> Reviewed-by: Gaurav Kumar Garg 
> Tested-by: Gluster Build System 
Signed-off-by: Dan Lambright 

Change-Id: Id45bd0fa6b8606dd47863de83a694908da393229
BUG: 1261664
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12143
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg

Upcall: Process each of the upcall events separately

2015-05-07T11:19:16+00:00

As suggested during the code-review of Bug1200262, have modified
GF_CBK_UPCALL to be exlusively GF_CBK_CACHE_INVALIDATION.
Thus, for any new upcall event, a new CBK procedure will be added.

Also made changes to store upcall data separately based on the
upcall event type received.

BUG: 1217711
Change-Id: I0f5e53d6f5ece16aecb514a0a426dca40fa1c755
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/10049
Reviewed-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.org/10562
Tested-by: NetBSD Build System
Tested-by: Gluster Build System

geo-rep: Status Enhancements

2015-05-06T16:12:35+00:00

Discussion in gluster-devel
http://www.gluster.org/pipermail/gluster-devel/2015-April/044301.html

MASTER NODE - Master Volume Node
MASTER VOL - Master Volume name
MASTER BRICK - Master Volume Brick
SLAVE USER - Slave User to which Geo-rep session is established
SLAVE - :: used in Geo-rep Create command
SLAVE NODE - Slave Node to which Master worker is connected
STATUS - Worker Status(Created, Initializing, Active, Passive, Faulty,
         Paused, Stopped)
CRAWL STATUS - Crawl type(Hybrid Crawl, History Crawl, Changelog Crawl)
LAST_SYNCED - Last Synced Time(Local Time in CLI output and UTC in XML output)
ENTRY - Number of entry Operations pending.(Resets on worker restart)
DATA - Number of Data operations pending(Resets on worker restart)
META - Number of Meta operations pending(Resets on worker restart)
FAILURES - Number of Failures
CHECKPOINT TIME - Checkpoint set Time(Local Time in CLI output and UTC
                  in XML output)
CHECKPOINT COMPLETED - Yes/No or N/A
CHECKPOINT COMPLETION TIME - Checkpoint Completed Time(Local Time in CLI
                             output and UTC in XML output)

XML output:

cliOutput>
  geoRep>
      volume>
        name>
        sessions>
          session>
             session_slave>
             pair>
                master_node>
                master_brick>
                slave_user>
                slave/>
                slave_node>
                status>
                crawl_status>
                entry>
                data>
                meta>
                failures>
                checkpoint_completed>
                master_node_uuid>
                last_synced>
                checkpoint_time>
                checkpoint_completion_time>

BUG: 1218586
Change-Id: I944a6c3c67f1e6d6baf9670b474233bec8f61ea3
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/10121
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/10574
Tested-by: Gluster Build System

glusterd: CLI commands to create and manage tiered volumes.

2015-03-19T13:32:28+00:00

A tiered volume is a normal volume with some number of new bricks
representing "hot" storage. The "hot" bricks can be attached or
detached dynamically to a normal volume. When this happens, a new graph
is constructed. The root of the new graph is an instance of the tier
translator. One subvolume of the tier translator leads to the old volume,
and another leads to the new hot bricks.

attach-tier  [ ]  ... [force]

volume detach-tier  [replica ] 
... 

gluster volume rebalance  tier start

gluster volume rebalance  tier stop

gluster volume rebalance  tier status

The "tier start" CLI command starts a server side daemon. The daemon
initiates file level migration based on caching policies. The daemon's
status can be monitored and stopped.

Note development on the "tier status" command is incomplete. It will be
added in a subsequent patch.

When the "hot" storage is detached, the tier translator is removed
from the graph and the tiered volume reverts to its original state as
described in the volume's info file.

For more background and design see the feature page [1].

[1]
http://www.gluster.org/community/documentation/index.php/Features/data-classification

Change-Id: Ic8042ce37327b850b9e199236e5be3dae95d2472
BUG: 1194753
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9753
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

cli/glusterd: cli command implementation for bitrot features

2015-03-19T01:34:07+00:00

CLI command for bitrot features.

volume bitrot  enable|disable

Above command will enable/disable bitrot feature for particular volume.

BUG: 1170075
Change-Id: Ie84002ef7f479a285688fdae99c7afa3e91b8b99
Signed-off-by: Gaurav Kumar Garg     
Signed-off-by: Anand nekkunti        
Signed-off-by: Dominic P Geevarghese 
Reviewed-on: http://review.gluster.org/9866
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

CLI : GLobal option for NFS-Ganesha

2015-03-18T11:33:13+00:00

A new global CLI option has been introduced for NFS-Ganesha.
gluster features.ganesha enable/disable.
This option is persistent and shall be inherited
by new volumes created after this option is set.

gluster features.ganesha enable

It carries out the following functions:
1. Disables gluster-nfs across the cluster
2. Starts NFS-Ganesha server on a subset of nodes and exports  '/'.
3. Creates the HA cluster for NFS-Ganesha.
4. Writes the option into the global config file.

gluster features.ganesha disable

1. Stops NFS-Ganesha server.
2. Tears down the HA cluster for NFS-Ganesha

With this change the older volume set
options with keys "nfs-ganesha.host"
and "nfs-ganesha.enable" will no longer
be supported. This commit has only has the
CLI related changes. Another patch will
be submitted to support this feature entirely.

Change-Id: Ie4b66a16c23b33b795738654b9a68f8e2c34efe3
BUG: 1188184
Signed-off-by: Meghana Madhusudhan 
Reviewed-on: http://review.gluster.org/9538
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

every/where: add GF_FOP_IPC for inter-translator communication

2015-03-17T14:02:15+00:00

Several features - e.g. encryption, erasure codes, or NSR - involve
multiple cooperating translators which sometimes need a "private" means
of communication amongst themselves.  Historically we've used virtual or
synthetic xattrs, but that's not very elegant and clutters up the
getxattr/setxattr path which must also handle real xattr requests.  This
new fop should address that.

The only argument is an int32_t "op" which should be recognized by the
target translator.  It is recommended that translators using these
feature follow some convention regarding the ops that they define, to
avoid conflicts.  Using a hash of the target translator's type string as
a base for a series of ops would probably be a good start.  Any other
information can be passed in both directions using xdata.

The default behavior for this fop, as with any other, is to pass through
to FIRST_CHILD.  That makes use of this fop "transparent" to other
translators that were written before it existed, but it also means that
it only really works with pass-through translators.  If a routing
translator (such as DHT) or a fan-out translator (such as AFR) is
involved, the IPC might not reach its intended destination unless those
translators are modified to forward IPC fops along all paths.

If an IPC gets all the way to storage/posix it is considered an error,
much like an uncaught exception.  We don't actually *do* anything in
that case, but we do log it send back an EOPNOTSUPP error.  This makes
the "unrecognized opcode" condition distinguishable from the "no IPC
support" condition (which would yield an RPC error instead) so clients
can probe for the presence of a handler for their own favorite opcode
and either use that or use old-school xattrs depending on the result.

BUG: 1158628
Signed-off-by: Venky Shankar 
Signed-off-by: Jeff Darcy 
Change-Id: I84af1b17babe5b30ec03ecf027ae37d09b873968
Reviewed-on: http://review.gluster.org/8812
Reviewed-by: Vijay Bellur