summaryrefslogtreecommitdiffstats
path: root/tools/glusterfind/src/brickfind.py
Commit message (Collapse)AuthorAgeFilesLines
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* tools/glusterfind: add --end-time optionMilind Changire2017-06-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Add optional --end-time argument to be used with --since-time when using the "query" command. "start" and "end" times are passed in the command-line to changelog.py only if --full has not been specified on command-line. -1 is passed to changelog.py as end time if user has not supplied --end-time on command-line. brickfind.py: Remove unused "start" command-line argument. Also: Minor indentation changes to keep flake8-2.7 happy. Change-Id: I063ef5459916f711503881ade5c4fc32374edad0 BUG: 1453151 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: https://review.gluster.org/17439 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* tools/glusterfind: add --field-separator optionMilind Changire2017-06-121-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Default field separator is a space character. This gets in the way if the file name itself has embedded spaces. Solution: Add --field-separator option to "pre" and "query" commands. The field separator string will be used to separate strings in the output lines that get written to the output file. eg. old output: NEW file1.txt RENAME file2 Copy.txt file3.txt with --field-separator as "===" new output: NEW===file1.txt RENAME===file2 Copy.txt===file3.txt Change-Id: I71e878fed58ba1113d97044ac9f6404ee66227c7 BUG: 1453151 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: https://review.gluster.org/17481 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* tools/glusterfind: add --full option to query commandMilind Changire2016-03-301-3/+7
| | | | | | | | | | | | | | | | | The --full option will use brickfind.py to list all files in the volume. The output file will contain url-encoded file names prefixed with the tag string to indicate that all files should be considered as new. The default tag string for the --full option is "NEW". This can be changed with the --tag-for-full-find command-line option. Change-Id: Ic85ba5db062e19df13ae9dc2de8a08eacb5c9792 BUG: 1286279 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/12779 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* tools/glusterfind: New option --no-encodeAravinda VK2016-02-261-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | New option added to skip encoding path in output file. Also handled Unicode strings. File paths can have newline characters, to differentiate between each path patch is encoded according to RFC3986(https://www.ietf.org/rfc/rfc3986.txt). Due to this consumer applications have to decode the path before consuming it. With this option Paths are not encoded, can be directly consumed by applications. Unicode encoding is handled automatically BUG: 1310080 Change-Id: I83d59831997dbd1264b48e9b1aa732c7dfc700b5 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13477 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* tools/glusterfind: GFID to Path conversion using ChangelogAravinda VK2015-05-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Records fop information collected from Changelogs in sqlite database. This is only working database, not required after processing. After post processing, output file is generated by reading these database files. This is applicable only in incremental run, When a changelog is parsed, all the details are saved in Db. GFID to Path is converted to those files for which information is available in Changelogs. For all the failed cases, it tries to convert to Path using Pgfid, if not found GFID to Path is done using find. BUG: 1201284 Change-Id: I53f168860dae15a0149004835e67f97aebd822be Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10463 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tools/glusterfind: Partial FindAravinda VK2015-05-081-24/+15
| | | | | | | | | | | | | | | | | | | | | | | | Enabled by default, if one node fails Glusterfind will not fail to return list of files from other nodes. This behavior can be changed using --disable-partial Now session is maintained in each nodes as well as in initiator node. Every pre command will pick the status file from respective node and start collecting list of changes happened after the status time. --reset-session-time, new option to force reset the session time. Next incremental run will start from this time. Change-detector argument is removed since Changelog mode is required to detect deletes and Renames. Change-Id: I0b83bc7c0e1b30b13de772b2d21fe968db4ff964 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1201289 Reviewed-on: http://review.gluster.org/10320 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tools/glusterfind: Ignore .trashcan dir during Brick CrawlAravinda VK2015-03-301-2/+6
| | | | | | | | | | | BUG: 1203650 Change-Id: Icf81726ad34bd5cfec6d8e06763b02a159e84c2e Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10010 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* feature/glusterfind: A tool to find incremental changesAravinda VK2015-03-181-0/+97
Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2025-06-23 23:06:36 +0000