summaryrefslogtreecommitdiffstats
path: root/tools/glusterfind/src/changelog.py
Commit message (Collapse)AuthorAgeFilesLines
* tools/glusterfind: Handling Unicode file namesAravinda VK2016-03-301-1/+2
| | | | | | | | | | | | | | | | | | | | | | Unicode filenames handled cleanly with this patch. Changelog files and output files are opened with utf-8 encoding using codecs.open. urllib.quote_plus and unquote_plus will not handle Unicode so, encode Unicode to 8-bit string version before calling unquote. urllib.quote_plus requires 8-bit string itself so do not decode to Unicode if we need to use quote_plus(when --no-encode=false). Decode to unicode in --no-encode is set. BUG: 1322431 Change-Id: If5561c749ab5529445650d322c831eb4da22b65a Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13798 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 48a0a38fadf9c5164869a908dcff8a951aa21b4b) Reviewed-on: http://review.gluster.org/13856
* tools/glusterfind: New option --no-encodeAravinda VK2016-03-081-7/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | New option added to skip encoding path in output file. Also handled Unicode strings. File paths can have newline characters, to differentiate between each path patch is encoded according to RFC3986(https://www.ietf.org/rfc/rfc3986.txt). Due to this consumer applications have to decode the path before consuming it. With this option Paths are not encoded, can be directly consumed by applications. Unicode encoding is handled automatically BUG: 1313310 Change-Id: I83d59831997dbd1264b48e9b1aa732c7dfc700b5 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13477 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 1b897c39ba8c0f1bf180316637cc2d87e6920800) Reviewed-on: http://review.gluster.org/13559
* tools/glusterfind: add query command to list filesMilind Changire2015-12-111-6/+12
| | | | | | | | | | | | | | | | | | | | | | | | | When session information is maintained outside Gluster, there needs to be some mechanism to list files starting from a time-stamp. This patch implements the feature via the "query" command-line option. The only caveat is that the first time the query command is run for the volume, it will likely report that "historical changelogs are not available". This is due to the fact that changelogs had not been turned on for the volume so far. So the volume options need to be turned on outside glusterfind or the since_time need to be greater than the current time when the query command is run for the very first time for the volume. The query command turns on the required volume options for collecting changelogs. Change-Id: I6cb7a57a5ecd166210e2eb4deede06d40ccfa996 BUG: 1272007 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/12362 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit a56e32e19703c0fbe2cedebcaf5edc8a6307d5a1) Reviewed-on: http://review.gluster.org/12742
* tools/glusterfind: Prepend prefix in case of deleteSaravanakumar Arumugam2015-11-241-14/+4
| | | | | | | | | | | | | | | | In case of delete operation alone, output prefix adding was not handled earlier. Output prefix is added now. Change-Id: Ia91444dddbff501b26a864f6185ca4c0aaf4c802 Reviewed-on: http://review.gluster.org/11712/ BUG: 1257141 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12016 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* tools/glusterfind: ignoring deleted filesMilind Changire2015-06-121-24/+36
| | | | | | | | | | | | | | | | | | | | OSError and IOError exceptions were being thrown if files were deleted after session was created and a subsequent glusterfind pre was attmepted. glusterfind now detects this scenario and safely ignores these changes to the file-system. We also avoid recording deleted file paths into database in the case where gfid to path resolution cannot be performed for deleted files. Also, we now turn on volume option to capture delete paths. Change-Id: Iaf3883463f5e64643497bd77229ad80a9b2d5da5 BUG: 1230783 Reviewed-on: http://review.gluster.org/#/c/11194/ Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/11199 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* tools/glusterfind: Fix GFID to Path conversion for dirAravinda VK2015-06-121-1/+1
| | | | | | | | | | | | | For Directories, GFID to Path conversion was fine, but wrong filter was used while updating to db. Change-Id: Id588adbb25ff5fd357f8cc2d5485bd2eb4a52b65 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/#/c/11158/ BUG: 1230712 Reviewed-on: http://review.gluster.org/11184 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tools/glusterfind: GFID to Path conversion using ChangelogAravinda VK2015-05-091-171/+223
| | | | | | | | | | | | | | | | | | | | | | | | Records fop information collected from Changelogs in sqlite database. This is only working database, not required after processing. After post processing, output file is generated by reading these database files. This is applicable only in incremental run, when a changelog is parsed, all the details are saved in DB. GFID to Path is converted to those files for which information is available in Changelogs. For all the failed cases, it tries to convert to Path using Pgfid, if not found GFID to Path is done using find. BUG: 1219475 Change-Id: Iaccc9d063b1187d973137e592f024979b724e85c Reviewed-On: http://review.gluster.org/#/c/10463/ Original-Author: Aravinda VK <avishwan@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10640 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* tools/glusterfind: Partial FindAravinda VK2015-05-091-7/+33
| | | | | | | | | | | | | | | | | | | | | | | | | This is optional and enabled by default, if one node fails Glusterfind will not fail to return list of files from other nodes. This behavior can be changed using --disable-partial Now session is maintained in each nodes as well as in initiator node. Every pre command will pick the status file from respective node and start collecting list of changes happened after the status time. --reset-session-time, new option to force reset the session time. Next incremental run will start from this time. Change-detector argument is removed since Changelog mode is required to detect deletes and Renames. BUG: 1219467 Change-Id: I1d0a0629facc3d26780200ccdf39b221ab4037c4 Original-Author: Aravinda VK <avishwan@redhat.com> Reviewed-On: http://review.gluster.org/#/c/10320 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/10639 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tools/glusterfind: Changelog_init before changelog_registerAravinda VK2015-03-301-0/+1
| | | | | | | | | | | | | | With the new Changelog RPC based approach, Changelog_init needs to be called before changelog_register BUG: 1206127 Change-Id: I817b84016d3f9ffdf2ea7a4e177953fe14ccc323 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10009 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* tools/glusterfind: Ignore .trashcan dir during Brick CrawlAravinda VK2015-03-301-2/+9
| | | | | | | | | | | BUG: 1203650 Change-Id: Icf81726ad34bd5cfec6d8e06763b02a159e84c2e Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/10010 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* feature/glusterfind: A tool to find incremental changesAravinda VK2015-03-181-0/+309
Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-25 00:17:32 +0000