glusterfs.git/tools/glusterfind, branch release-4.0

build: add --without-server option

2018-02-21T18:08:14+00:00

With Gluster 4.0 we will not provide the server components for EL6 and
older. At one point Gluster 4.x will get GlusterD2, which requires
Golang tools in the distribution. EL6 does not contain these at the
moment.

With this change, it is possible to `./configure --without-server` which
prevents building glusterd and the xlators for the bricks. Building RPMs
can pass `--without server` and the glusterfs-server sub-package will
not be created.

Change-Id: I97f5ccf9f2c76e60d9af83915fc59fae57ad6d25
BUG: 1547635
Signed-off-by: Niels de Vos

glusterfind: Speed up gfid lookup 100x by using an SQL index

2017-12-30T20:37:57+00:00

Fixes #1529883.

This fixes some bits of `glusterfind`'s horrible performance,
making it 100x faster.

Until now, glusterfind was, for each line in each CHANGELOG.* file,
linearly reading the entire contents of the sqlite database in
4096-bytes-sized pread64() syscalls when executing the

  SELECT COUNT(1) FROM %s WHERE 1=1 AND gfid = ?

query through the code path:

  get_changes()
    parse_changelog_to_db()
      when_data_meta()
        gfidpath_exists()
          _exists()

In a quick benchmark on my laptop, doing one such `SELECT` query
took ~75ms on a 10MB-sized sqlite DB, while doing the same query
with an index took < 1ms.

Change-Id: I8e7fe60f1f45a06c102f56b54d2ead9e0377794e
BUG: 1529883
Signed-off-by: Niklas Hambüchen

tools/glusterfind: Fix encoding to encode only space,newline and percent chars

2017-07-21T08:41:13+00:00

libgfchangelog was encoding path using spec rfc3986, but encoding only
required for SPACE, NEWLINE and PERCENT chars since the NEWLINE char is
used as record separator and SPACE as field separator in the parsed
changelogs output.

Changed the encoding function to encode only SPACE, NEWLINE and PERCENT chars

BUG: 1451724
Change-Id: Ic1dea824d23493dedcf3db45f353f90572f4e046
Signed-off-by: Aravinda VK 
Reviewed-on: https://review.gluster.org/17788
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Milind Changire

tools/glusterfind: unquote DELETE path before further processing

2017-06-30T07:40:51+00:00

Problem:
DELETE path is quoted before it reaches glusterfind. This wasn't handled
in the glusterfind code leading to double quoting of path separator
'%2F' to '%252F' i.e. the '%' character in '%2F' itself was quoted to
'%25'

Solution:
unquote the the deleted path before further processing

Change-Id: I2dfbbd7792dc0f9da5c8e02093b0f1c031ff344a
BUG: 1465024
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17629
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

tools/glusterfind: initialize variable 'end'

2017-06-19T10:10:32+00:00

Problem:
syntax error due to uninitialized variable 'end'

Solution:
set end = -1

Change-Id: Ia8b500e9dd5304d0832e5f594abca0b5b2ab357e
BUG: 1462241
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17557
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

tools/glusterfind: add --end-time option

2017-06-12T05:14:08+00:00

Add optional --end-time argument to be used with --since-time
when using the "query" command.

"start" and "end" times are passed in the command-line to
changelog.py only if --full has not been specified on command-line.
-1 is passed to changelog.py as end time if user has not supplied
--end-time on command-line.

brickfind.py:
Remove unused "start" command-line argument.

Also:
Minor indentation changes to keep flake8-2.7 happy.

Change-Id: I063ef5459916f711503881ade5c4fc32374edad0
BUG: 1453151
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17439
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

tools/glusterfind: add --field-separator option

2017-06-12T04:48:55+00:00

Problem:
Default field separator is a space character.
This gets in the way if the file name itself has embedded spaces.

Solution:
Add --field-separator option to "pre" and "query" commands.
The field separator string will be used to separate strings in the
output lines that get written to the output file.

eg.
old output:
NEW file1.txt
RENAME file2 Copy.txt file3.txt

with --field-separator as "==="
new output:
NEW===file1.txt
RENAME===file2 Copy.txt===file3.txt

Change-Id: I71e878fed58ba1113d97044ac9f6404ee66227c7
BUG: 1453151
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17481
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:18:07+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430841
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Joe Julian 
Reviewed-by: Shyamsundar Ranganathan

tools/glusterfind: delete temporary folder

2017-01-30T05:27:25+00:00

Problem:
Run specific temporary folder created under
/usr/var/lib/misc/glusterfsd/glusterfind///
remains undeleted.

Solution:
Delete the temporary folder from all nodes.

Change-Id: I0edaf868aebb01b15c489434bbb26fe853351384
BUG: 1413526
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/16416
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

tools/glusterfind: avoid deleting keys directory

2016-12-08T06:05:48+00:00

Problem:
gluster volume delete mistakenly deletes the .keys directory under
/var/lib/glusterd/glusterfind.

Solution:
Check for ".keys" directory and avoid deleting it.

Change-Id: Ia595c8bf3f423c1ad5d6faa183a29598c07a11f9
BUG: 1402369
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/16052
Reviewed-by: Aravinda VK 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System