diff options
| author | Vijay Bellur <vijay@gluster.com> | 2012-04-09 23:11:52 +0530 | 
|---|---|---|
| committer | Anand Avati <avati@redhat.com> | 2012-04-11 10:25:56 -0700 | 
| commit | 076830c068fb39bbc3e863c89a4253cbea36357e (patch) | |
| tree | 842884d8db9a40d5a53e5171c852a84daa8e0f65 /doc/hacker-guide | |
| parent | df8e2f53b70f4f49af70df308010dddfe5ca35ec (diff) | |
doc: Move outdated documentation to legacy
Change-Id: I0ceba9a993e8b1cdef4ff6a784bfd69c08107d88
BUG: 811311
Signed-off-by: Vijay Bellur <vijay@gluster.com>
Reviewed-on: http://review.gluster.com/3116
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Diffstat (limited to 'doc/hacker-guide')
| -rw-r--r-- | doc/hacker-guide/Makefile.am | 8 | ||||
| -rw-r--r-- | doc/hacker-guide/adding-fops.txt | 33 | ||||
| -rw-r--r-- | doc/hacker-guide/bdb.txt | 70 | ||||
| -rw-r--r-- | doc/hacker-guide/call-stub.txt | 1033 | ||||
| -rw-r--r-- | doc/hacker-guide/hacker-guide.tex | 309 | ||||
| -rw-r--r-- | doc/hacker-guide/lock-ahead.txt | 80 | ||||
| -rw-r--r-- | doc/hacker-guide/posix.txt | 59 | ||||
| -rw-r--r-- | doc/hacker-guide/replicate.txt | 206 | ||||
| -rw-r--r-- | doc/hacker-guide/write-behind.txt | 45 | 
9 files changed, 0 insertions, 1843 deletions
diff --git a/doc/hacker-guide/Makefile.am b/doc/hacker-guide/Makefile.am deleted file mode 100644 index 65c92ac235e..00000000000 --- a/doc/hacker-guide/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -EXTRA_DIST = replicate.txt bdb.txt posix.txt call-stub.txt write-behind.txt - -#EXTRA_DIST = hacker-guide.tex afr.txt bdb.txt posix.txt call-stub.txt write-behind.txt -#hacker_guidedir = $(docdir) -#hacker_guide_DATA = hacker-guide.pdf - -#hacker-guide.pdf: $(EXTRA_DIST) -#	pdflatex $(srcdir)/hacker-guide.tex diff --git a/doc/hacker-guide/adding-fops.txt b/doc/hacker-guide/adding-fops.txt deleted file mode 100644 index e70dbbdc829..00000000000 --- a/doc/hacker-guide/adding-fops.txt +++ /dev/null @@ -1,33 +0,0 @@ -		  HOW TO ADD A NEW FOP TO GlusterFS -		  ================================= - -Steps to be followed when adding a new FOP to GlusterFS: - -1. Edit glusterfs.h and add a GF_FOP_* constant. - -2. Edit xlator.[ch] and: -   2a. add the new prototype for fop and callback. -   2b. edit xlator_fops structure. - -3. Edit xlator.c and add to fill_defaults. - -4. Edit protocol.h and add struct necessary for the new FOP. - -5. Edit defaults.[ch] and provide default implementation. - -6. Edit call-stub.[ch] and provide stub implementation. - -7. Edit common-utils.c and add to gf_global_variable_init(). - -8. Edit client-protocol and add your FOP. - -9. Edit server-protocol and add your FOP. - -10. Implement your FOP in any translator for which the default implementation -    is not sufficient. - -========================================== -Last updated: Mon Oct 27 21:35:49 IST 2008 - -Author: Vikas Gorur <vikas@gluster.com> -========================================== diff --git a/doc/hacker-guide/bdb.txt b/doc/hacker-guide/bdb.txt deleted file mode 100644 index fd0bd365206..00000000000 --- a/doc/hacker-guide/bdb.txt +++ /dev/null @@ -1,70 +0,0 @@ - -* How does file translates to key/value pair? ---------------------------------------------- - -  in bdb a file is identified by key (obtained by taking basename() of the path of -the file) and file contents are stored as value corresponding to the key in database -file (defaults to glusterfs_storage.db under dirname() directory). - -* symlinks, directories ------------------------ - -  symlinks and directories are stored as is. - -* db (database) files ---------------------- - -  every directory, including root directory, contains a database file called -glusterfs_storage.db. all the regular files contained in the directory are stored -as key/value pair inside the glusterfs_storage.db. - -* internal data cache ---------------------- - -  db does not provide a way to find out the size of the value corresponding to a key.  -so, bdb makes DB->get() call for key and takes the length of the value returned.  -since DB->get() also returns file contents for key, bdb maintains an internal cache and -stores the file contents in the cache. -  every directory maintains a seperate cache. -   -* inode number transformation ------------------------------ - -  bdb allocates a inode number to each file and directory on its own. bdb maintains a  -global counter and increments it after allocating inode number for each file -(regular, symlink or directory). NOTE: bdb does not guarantee persistent inode numbers. - -* checkpoint thread -------------------- - -  bdb creates a checkpoint thread at the time of init(). checkpoint thread does a  -periodic checkpoint on the DB_ENV. checkpoint is the mechanism, provided by db, to  -forcefully commit the logged transactions to the storage. - -NOTES ABOUT FOPS: ------------------ - -lookup() - - 1> do lstat() on the path, if lstat fails, we assume that the file being looked up -    is either a regular file or doesn't exist. - 2> lookup in the DB of parent directory for key corresponding to path. if key exists, -    return key, with.  -    NOTE: 'struct stat' stat()ed from DB file is used as a container for 'struct stat' -           of the regular file. st_ino, st_size, st_blocks are updated with file's values. - -readv() - - 1> do a lookup in bctx cache. if successful, return the requested data from cache. - 2> if cache missed, do a DB->get() the entire file content and insert to cache. - -writev(): - 1> flush any cached content of this file. - 2> do a DB->put(), with DB_DBT_PARTIAL flag.  -    NOTE: DB_DBT_PARTIAL is used to do partial update of a value in DB. - -readdir(): - 1> regular readdir() in a loop, and vomit all DB_ENV log files and DB files that -    we encounter. - 2> if the readdir() buffer still has space, open a DB cursor and do a sequential -    DBC->get() to fill the reaadir buffer. - - diff --git a/doc/hacker-guide/call-stub.txt b/doc/hacker-guide/call-stub.txt deleted file mode 100644 index bca1579b22d..00000000000 --- a/doc/hacker-guide/call-stub.txt +++ /dev/null @@ -1,1033 +0,0 @@ -creating a call stub and pausing a call ---------------------------------------- -libglusterfs provides seperate API to pause each of the fop. parameters to each API is -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -       NOTE: @fn should exactly take the same type and number of parameters that  -             the corresponding regular fop takes. -rest will be the regular parameters to corresponding fop. - -NOTE: @frame can never be NULL. fop_<operation>_stub() fails with errno -      set to EINVAL, if @frame is NULL. also wherever @loc is applicable, -      @loc cannot be NULL. - -refer to individual stub creation API to know about call-stub creation's behaviour with -specific parameters. - -here is the list of stub creation APIs for xlator fops. - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@loc       - pointer to location structure. -             NOTE: @loc will be copied to a different location, with inode_ref() to -	           @loc->inode and @loc->parent, if not NULL. also @loc->path will be -		   copied to a different location. -@need_xattr - flag to specify if xattr should be returned or not. -call_stub_t * -fop_lookup_stub (call_frame_t *frame, -		 fop_lookup_t fn, -		 loc_t *loc, -		 int32_t need_xattr); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -call_stub_t * -fop_stat_stub (call_frame_t *frame, -	       fop_stat_t fn, -	       loc_t *loc); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to lk fop. -         NOTE: @fd is stored with a fd_ref(). -call_stub_t * -fop_fstat_stub (call_frame_t *frame, -		fop_fstat_t fn, -		fd_t *fd); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to @loc->inode and -	       @loc->parent, if not NULL. also @loc->path will be copied to a different location. -@mode  - mode parameter to chmod. -call_stub_t * -fop_chmod_stub (call_frame_t *frame, -		fop_chmod_t fn, -		loc_t *loc, -		mode_t mode); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to lk fop. -         NOTE: @fd is stored with a fd_ref(). -@mode  - mode parameter for fchmod fop. -call_stub_t * -fop_fchmod_stub (call_frame_t *frame, -		 fop_fchmod_t fn, -		 fd_t *fd, -		 mode_t mode); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to @loc->inode and -	       @loc->parent, if not NULL. also @loc->path will be copied to a different location. -@uid   - uid parameter to chown. -@gid   - gid parameter to chown. -call_stub_t * -fop_chown_stub (call_frame_t *frame, -		fop_chown_t fn, -		loc_t *loc, -		uid_t uid, -		gid_t gid); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to lk fop. -         NOTE: @fd is stored with a fd_ref(). -@uid   - uid parameter to fchown. -@gid   - gid parameter to fchown. -call_stub_t * -fop_fchown_stub (call_frame_t *frame, -		 fop_fchown_t fn, -		 fd_t *fd, -		 uid_t uid, -		 gid_t gid); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location, if not NULL. -@off   - offset parameter to truncate fop. -call_stub_t * -fop_truncate_stub (call_frame_t *frame, -		   fop_truncate_t fn, -		   loc_t *loc, -		   off_t off); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to lk fop. -         NOTE: @fd is stored with a fd_ref(). -@off   - offset parameter to ftruncate fop. -call_stub_t * -fop_ftruncate_stub (call_frame_t *frame, -		    fop_ftruncate_t fn, -		    fd_t *fd, -		    off_t off); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@tv    - tv parameter to utimens fop. -call_stub_t * -fop_utimens_stub (call_frame_t *frame, -		  fop_utimens_t fn, -		  loc_t *loc, -		  struct timespec tv[2]); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@mask  - mask parameter for access fop. -call_stub_t * -fop_access_stub (call_frame_t *frame, -		 fop_access_t fn, -		 loc_t *loc, -		 int32_t mask); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@size  - size parameter to readlink fop. -call_stub_t * -fop_readlink_stub (call_frame_t *frame, -		   fop_readlink_t fn, -		   loc_t *loc, -		   size_t size); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@mode  - mode parameter to mknod fop. -@rdev  - rdev parameter to mknod fop. -call_stub_t * -fop_mknod_stub (call_frame_t *frame, -		fop_mknod_t fn, -		loc_t *loc, -		mode_t mode, -		dev_t rdev); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@mode  - mode parameter to mkdir fop. -call_stub_t * -fop_mkdir_stub (call_frame_t *frame, -		fop_mkdir_t fn, -		loc_t *loc, -		mode_t mode); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -call_stub_t * -fop_unlink_stub (call_frame_t *frame, -		 fop_unlink_t fn, -		 loc_t *loc); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -call_stub_t * -fop_rmdir_stub (call_frame_t *frame, -		fop_rmdir_t fn, -		loc_t *loc); - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@linkname - linkname parameter to symlink fop. -@loc      - pointer to location structure. -            NOTE: @loc will be copied to a different location, with inode_ref() to -	          @loc->inode and @loc->parent, if not NULL. also @loc->path will be -		  copied to a different location. -call_stub_t * -fop_symlink_stub (call_frame_t *frame, -		  fop_symlink_t fn, -		  const char *linkname, -		  loc_t *loc); - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@oldloc   - pointer to location structure. -            NOTE: @oldloc will be copied to a different location, with inode_ref() to  -	          @oldloc->inode and @oldloc->parent, if not NULL. also @oldloc->path will  -		  be copied to a different location, if not NULL. -@newloc   - pointer to location structure. -            NOTE: @newloc will be copied to a different location, with inode_ref() to -	          @newloc->inode and @newloc->parent, if not NULL. also @newloc->path will -		  be copied to a different location, if not NULL. -call_stub_t * -fop_rename_stub (call_frame_t *frame, -		 fop_rename_t fn, -		 loc_t *oldloc, -		 loc_t *newloc); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc     - pointer to location structure. -           NOTE: @loc will be copied to a different location, with inode_ref() to -	         @loc->inode and @loc->parent, if not NULL. also @loc->path will be -		 copied to a different location. -@newpath - newpath parameter to link fop. -call_stub_t * -fop_link_stub (call_frame_t *frame, -	       fop_link_t fn, -	       loc_t *oldloc, -	       const char *newpath); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@flags - flags parameter to create fop. -@mode  - mode parameter to create fop. -@fd    - file descriptor parameter to create fop. -         NOTE: @fd is stored with a fd_ref(). -call_stub_t * -fop_create_stub (call_frame_t *frame, -		 fop_create_t fn, -		 loc_t *loc, -		 int32_t flags, -		 mode_t mode, fd_t *fd); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@flags - flags parameter to open fop. -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -call_stub_t * -fop_open_stub (call_frame_t *frame, -	       fop_open_t fn, -	       loc_t *loc, -	       int32_t flags, -	       fd_t *fd); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to lk fop. -         NOTE: @fd is stored with a fd_ref(). -@size  - size parameter to readv fop. -@off   - offset parameter to readv fop. -call_stub_t * -fop_readv_stub (call_frame_t *frame, -		fop_readv_t fn, -		fd_t *fd, -		size_t size, -		off_t off); - -@frame  - call frame which has to be used to resume the call at call_resume(). -@fn     - procedure to call during call_resume().  -@fd     - file descriptor parameter to lk fop. -          NOTE: @fd is stored with a fd_ref(). -@vector - vector parameter to writev fop.  -	  NOTE: @vector is iov_dup()ed while creating stub. and frame->root->req_refs -                dictionary is dict_ref()ed. -@count  - count parameter to writev fop. -@off    - off parameter to writev fop. -call_stub_t * -fop_writev_stub (call_frame_t *frame, -		 fop_writev_t fn, -		 fd_t *fd, -		 struct iovec *vector, -		 int32_t count, -		 off_t off); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to flush fop. -         NOTE: @fd is stored with a fd_ref(). -call_stub_t * -fop_flush_stub (call_frame_t *frame, -		fop_flush_t fn, -		fd_t *fd); - - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@fd       - file descriptor parameter to lk fop. -            NOTE: @fd is stored with a fd_ref(). -@datasync - datasync parameter to fsync fop. -call_stub_t * -fop_fsync_stub (call_frame_t *frame, -		fop_fsync_t fn, -		fd_t *fd, -		int32_t datasync); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to @loc->inode and -	       @loc->parent, if not NULL. also @loc->path will be copied to a different location. -@fd    - file descriptor parameter to opendir fop. -         NOTE: @fd is stored with a fd_ref(). -call_stub_t * -fop_opendir_stub (call_frame_t *frame, -		  fop_opendir_t fn, -		  loc_t *loc,  -		  fd_t *fd); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to getdents fop. -         NOTE: @fd is stored with a fd_ref(). -@size  - size parameter to getdents fop. -@off   - off parameter to getdents fop. -@flags - flags parameter to getdents fop. -call_stub_t * -fop_getdents_stub (call_frame_t *frame, -		   fop_getdents_t fn, -		   fd_t *fd, -		   size_t size, -		   off_t off, -		   int32_t flag); - -@frame   - call frame which has to be used to resume the call at call_resume(). -@fn      - procedure to call during call_resume().  -@fd      - file descriptor parameter to setdents fop. -           NOTE: @fd is stored with a fd_ref(). -@flags   - flags parameter to setdents fop. -@entries - entries parameter to setdents fop. -call_stub_t * -fop_setdents_stub (call_frame_t *frame, -		   fop_setdents_t fn, -		   fd_t *fd, -		   int32_t flags, -		   dir_entry_t *entries, -		   int32_t count); - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@fd       - file descriptor parameter to setdents fop. -            NOTE: @fd is stored with a fd_ref(). -@datasync - datasync parameter to fsyncdir fop. -call_stub_t * -fop_fsyncdir_stub (call_frame_t *frame, -		   fop_fsyncdir_t fn, -		   fd_t *fd, -		   int32_t datasync); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -call_stub_t * -fop_statfs_stub (call_frame_t *frame, -		 fop_statfs_t fn, -		 loc_t *loc); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to  -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@dict  - dict parameter to setxattr fop. -         NOTE: stub creation procedure stores @dict pointer with dict_ref() to it. -call_stub_t * -fop_setxattr_stub (call_frame_t *frame, -		   fop_setxattr_t fn, -		   loc_t *loc, -		   dict_t *dict, -		   int32_t flags); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@name  - name parameter to getxattr fop. -call_stub_t * -fop_getxattr_stub (call_frame_t *frame, -		   fop_getxattr_t fn, -		   loc_t *loc, -		   const char *name); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@name  - name parameter to removexattr fop. -         NOTE: name string will be copied to a different location while creating stub. -call_stub_t * -fop_removexattr_stub (call_frame_t *frame, -		      fop_removexattr_t fn, -		      loc_t *loc, -		      const char *name); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to lk fop. -         NOTE: @fd is stored with a fd_ref(). -@cmd   - command parameter to lk fop. -@lock  - lock parameter to lk fop. -         NOTE: lock will be copied to a different location while creating stub. -call_stub_t * -fop_lk_stub (call_frame_t *frame, -	     fop_lk_t fn, -	     fd_t *fd, -	     int32_t cmd, -	     struct flock *lock); - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@fd       - fd parameter to gf_lk fop. -	    NOTE: @fd is fd_ref()ed while creating stub, if not NULL. -@cmd      - cmd parameter to gf_lk fop. -@lock     - lock paramater to gf_lk fop. -	    NOTE: @lock is copied to a different memory location while creating -	          stub.  -call_stub_t * -fop_gf_lk_stub (call_frame_t *frame, -		fop_gf_lk_t fn, -		fd_t *fd, -		int32_t cmd, -		struct flock *lock); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@fd    - file descriptor parameter to readdir fop. -         NOTE: @fd is stored with a fd_ref(). -@size  - size parameter to readdir fop. -@off   - offset parameter to readdir fop. -call_stub_t * -fop_readdir_stub (call_frame_t *frame, -		  fop_readdir_t fn, -		  fd_t *fd, -		  size_t size, -		  off_t off); - -@frame - call frame which has to be used to resume the call at call_resume(). -@fn    - procedure to call during call_resume().  -@loc   - pointer to location structure. -         NOTE: @loc will be copied to a different location, with inode_ref() to -	       @loc->inode and @loc->parent, if not NULL. also @loc->path will be -	       copied to a different location. -@flags - flags parameter to checksum fop. -call_stub_t * -fop_checksum_stub (call_frame_t *frame, -		   fop_checksum_t fn, -		   loc_t *loc, -		   int32_t flags); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@inode     - inode parameter to @fn. -	     NOTE: @inode pointer is stored with a inode_ref(). -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -@dict      - dict parameter to @fn. -	     NOTE: @dict pointer is stored with dict_ref(). -call_stub_t * -fop_lookup_cbk_stub (call_frame_t *frame, -		     fop_lookup_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     inode_t *inode, -		     struct stat *buf, -		     dict_t *dict); -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_stat_cbk_stub (call_frame_t *frame, -		   fop_stat_cbk_t fn, -		   int32_t op_ret, -		   int32_t op_errno, -		   struct stat *buf); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_fstat_cbk_stub (call_frame_t *frame, -		    fop_fstat_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    struct stat *buf); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_chmod_cbk_stub (call_frame_t *frame, -		    fop_chmod_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    struct stat *buf); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_fchmod_cbk_stub (call_frame_t *frame, -		     fop_fchmod_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     struct stat *buf); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_chown_cbk_stub (call_frame_t *frame, -		    fop_chown_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_fchown_cbk_stub (call_frame_t *frame, -		     fop_fchown_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_truncate_cbk_stub (call_frame_t *frame, -		       fop_truncate_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno, -		       struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_ftruncate_cbk_stub (call_frame_t *frame, -			fop_ftruncate_cbk_t fn, -			int32_t op_ret, -			int32_t op_errno, -			struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_utimens_cbk_stub (call_frame_t *frame, -		      fop_utimens_cbk_t fn, -		      int32_t op_ret, -		      int32_t op_errno, -		      struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_access_cbk_stub (call_frame_t *frame, -		     fop_access_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@path      - path parameter to @fn. -	     NOTE: @path is copied to a different memory location, if not NULL. -call_stub_t * -fop_readlink_cbk_stub (call_frame_t *frame, -		       fop_readlink_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno, -		       const char *path); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@inode     - inode parameter to @fn. -	     NOTE: @inode pointer is stored with a inode_ref(). -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_mknod_cbk_stub (call_frame_t *frame, -		    fop_mknod_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    inode_t *inode, -		    struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@inode     - inode parameter to @fn. -	     NOTE: @inode pointer is stored with a inode_ref(). -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_mkdir_cbk_stub (call_frame_t *frame, -		    fop_mkdir_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    inode_t *inode, -		    struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_unlink_cbk_stub (call_frame_t *frame, -		     fop_unlink_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_rmdir_cbk_stub (call_frame_t *frame, -		    fop_rmdir_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@inode     - inode parameter to @fn. -	     NOTE: @inode pointer is stored with a inode_ref(). -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_symlink_cbk_stub (call_frame_t *frame, -		      fop_symlink_cbk_t fn, -		      int32_t op_ret, -		      int32_t op_errno, -		      inode_t *inode, -		      struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_rename_cbk_stub (call_frame_t *frame, -		     fop_rename_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@inode     - inode parameter to @fn. -	     NOTE: @inode pointer is stored with a inode_ref(). -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_link_cbk_stub (call_frame_t *frame, -		   fop_link_cbk_t fn, -		   int32_t op_ret, -		   int32_t op_errno, -		   inode_t *inode, -		   struct stat *buf); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@fd        - fd parameter to @fn. -	     NOTE: @fd pointer is stored with a fd_ref(). -@inode     - inode parameter to @fn. -	     NOTE: @inode pointer is stored with a inode_ref(). -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_create_cbk_stub (call_frame_t *frame, -		     fop_create_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     fd_t *fd, -		     inode_t *inode, -		     struct stat *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@fd        - fd parameter to @fn. -	     NOTE: @fd pointer is stored with a fd_ref(). -call_stub_t * -fop_open_cbk_stub (call_frame_t *frame, -		   fop_open_cbk_t fn, -		   int32_t op_ret, -		   int32_t op_errno, -		   fd_t *fd); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@vector    - vector parameter to @fn.	 -	     NOTE: @vector is copied to a different memory location, if not NULL. also -	           frame->root->rsp_refs is dict_ref()ed. -@stbuf     - stbuf parameter to @fn. -	     NOTE: @stbuf is copied to a different memory location, if not NULL. -call_stub_t * -fop_readv_cbk_stub (call_frame_t *frame, -		    fop_readv_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    struct iovec *vector, -		    int32_t count, -		    struct stat *stbuf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@stbuf     - stbuf parameter to @fn. -	     NOTE: @stbuf is copied to a different memory location, if not NULL. -call_stub_t * -fop_writev_cbk_stub (call_frame_t *frame, -		     fop_writev_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     struct stat *stbuf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_flush_cbk_stub (call_frame_t *frame, -		    fop_flush_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_fsync_cbk_stub (call_frame_t *frame, -		    fop_fsync_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@fd        - fd parameter to @fn. -	     NOTE: @fd pointer is stored with a fd_ref(). -call_stub_t * -fop_opendir_cbk_stub (call_frame_t *frame, -		      fop_opendir_cbk_t fn, -		      int32_t op_ret, -		      int32_t op_errno, -		      fd_t *fd); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@entries   - entries parameter to @fn. -@count     - count parameter to @fn. -call_stub_t * -fop_getdents_cbk_stub (call_frame_t *frame, -		      fop_getdents_cbk_t fn, -		      int32_t op_ret, -		      int32_t op_errno, -		      dir_entry_t *entries, -		      int32_t count); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_setdents_cbk_stub (call_frame_t *frame, -		       fop_setdents_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno); - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_fsyncdir_cbk_stub (call_frame_t *frame, -		       fop_fsyncdir_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@buf       - buf parameter to @fn. -	     NOTE: @buf is copied to a different memory location, if not NULL. -call_stub_t * -fop_statfs_cbk_stub (call_frame_t *frame, -		     fop_statfs_cbk_t fn, -		     int32_t op_ret, -		     int32_t op_errno, -		     struct statvfs *buf); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_setxattr_cbk_stub (call_frame_t *frame, -		       fop_setxattr_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -@value     - value dictionary parameter to @fn. -	     NOTE: @value pointer is stored with a dict_ref(). -call_stub_t * -fop_getxattr_cbk_stub (call_frame_t *frame, -		       fop_getxattr_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno, -		       dict_t *value); - - -@frame     - call frame which has to be used to resume the call at call_resume(). -@fn        - procedure to call during call_resume().  -@op_ret    - op_ret parameter to @fn. -@op_errno  - op_errno parameter to @fn. -call_stub_t * -fop_removexattr_cbk_stub (call_frame_t *frame, -			  fop_removexattr_cbk_t fn, -			  int32_t op_ret, -			  int32_t op_errno); - - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@op_ret   - op_ret parameter to @fn. -@op_errno - op_errno parameter to @fn. -@lock     - lock parameter to @fn. -	    NOTE: @lock is copied to a different memory location while creating -	          stub.  -call_stub_t * -fop_lk_cbk_stub (call_frame_t *frame, -		 fop_lk_cbk_t fn, -		 int32_t op_ret, -		 int32_t op_errno, -		 struct flock *lock); - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@op_ret   - op_ret parameter to @fn. -@op_errno - op_errno parameter to @fn. -@lock     - lock parameter to @fn. -	    NOTE: @lock is copied to a different memory location while creating -	          stub.  -call_stub_t * -fop_gf_lk_cbk_stub (call_frame_t *frame, -		    fop_gf_lk_cbk_t fn, -		    int32_t op_ret, -		    int32_t op_errno, -		    struct flock *lock); - - -@frame    - call frame which has to be used to resume the call at call_resume(). -@fn       - procedure to call during call_resume().  -@op_ret   - op_ret parameter to @fn. -@op_errno - op_errno parameter to @fn. -@entries  - entries parameter to @fn. -call_stub_t * -fop_readdir_cbk_stub (call_frame_t *frame, -		      fop_readdir_cbk_t fn, -		      int32_t op_ret, -		      int32_t op_errno, -		      gf_dirent_t *entries); - - -@frame         - call frame which has to be used to resume the call at call_resume(). -@fn            - procedure to call during call_resume().  -@op_ret        - op_ret parameter to @fn. -@op_errno      - op_errno parameter to @fn. -@file_checksum - file_checksum parameter to @fn. -                 NOTE: file_checksum will be copied to a different memory location  -		       while creating stub. -@dir_checksum  - dir_checksum parameter to @fn. -                 NOTE: file_checksum will be copied to a different memory location  -		       while creating stub. -call_stub_t * -fop_checksum_cbk_stub (call_frame_t *frame, -		       fop_checksum_cbk_t fn, -		       int32_t op_ret, -		       int32_t op_errno, -		       uint8_t *file_checksum, -		       uint8_t *dir_checksum); - -resuming a call: ---------------- -  call can be resumed using call stub through call_resume API. - -  void call_resume (call_stub_t *stub); - -  stub - call stub created during pausing a call. - -  NOTE: call_resume() will decrease reference count of any fd_t, dict_t and inode_t that it finds -        in  stub->args.<operation>.<fd_t-or-inode_t-or-dict_t>. so, if any fd_t, dict_t or -	inode_t pointers are assigned at stub->args.<operation>.<fd_t-or-inode_t-or-dict_t> after -	fop_<operation>_stub() call, they must be <fd_t-or-inode_t-or-dict_t>_ref()ed. -	 -	call_resume does not STACK_DESTROY() for any fop. -	 -  if stub->fn is NULL, call_resume does STACK_WIND() or STACK_UNWIND() using the stub->frame. - -  return - call resume fails only if stub is NULL. call resume fails with errno set to EINVAL. diff --git a/doc/hacker-guide/hacker-guide.tex b/doc/hacker-guide/hacker-guide.tex deleted file mode 100644 index c2d7255d7df..00000000000 --- a/doc/hacker-guide/hacker-guide.tex +++ /dev/null @@ -1,309 +0,0 @@ -\documentclass{book}[12pt] -\usepackage{graphicx} -% \usepackage{fancyhdr} - -% \pagestyle{fancy} -\begin{document} - -% \headheight 117pt -% \rhead{\includegraphics{zr-logo.eps}} - -\author{Gluster} -\title{GlusterFS 1.3 Hacker's Guide} -\date{June 1, 2007} - -\maketitle -\frontmatter -\tableofcontents - -\mainmatter -\chapter{Introduction} - -\section{Coding guidelines} -GlusterFS uses Git for version control. To get the latest source do: -\begin{verbatim} -  $ git clone git://git.gluster.com/glusterfs.git glusterfs -\end{verbatim} -\noindent -GlusterFS follows the GNU coding -standards\footnote{http://www.gnu.org/prep/standards\_toc.html} for the -most part. - -\chapter{Major components} -\section{libglusterfs} -\texttt{libglusterfs} contains supporting code used by all the other components.  -The important files here are: - -\texttt{dict.c}: This is an implementation of a serializable dictionary type. It is -used by the protocol code to send requests and replies. It is also used to pass options -to translators. - -\texttt{logging.c}: This is a thread-safe logging library. The log messages go to a -file (default \texttt{/usr/local/var/log/glusterfs/*}). - -\texttt{protocol.c}: This file implements the GlusterFS on-the-wire -protocol. The protocol itself is a simple ASCII protocol, designed to -be easy to parse and be human readable. - -A sample GlusterFS protocol block looks like this: -\begin{verbatim} -  Block Start                            header -  0000000000000023                       callid -  00000001                               type -  00000016                               op -  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx       human-readable name -  00000000000000000000000000000ac3       block size -  <...>                                  block -  Block End -\end{verbatim} - -\texttt{stack.h}: This file defines the \texttt{STACK\_WIND} and -\texttt{STACK\_UNWIND} macros which are used to implement the parallel -stack that is maintained for inter-xlator calls. See the \textsl{Taking control -of the stack} section below for more details. - -\texttt{spec.y}: This contains the Yacc grammar for the GlusterFS -specification file, and the parsing code. - - -Draw diagrams of trees -Two rules: -(1) directory structure is same -(2) file can exist only on one node - -\section{glusterfs-fuse} -\section{glusterfsd} -\section{transport} -\section{scheduler} -\section{xlator} - -\chapter{xlators} -\section{Taking control of the stack} -One can think of STACK\_WIND/UNWIND as a very specific RPC mechanism. - -% \includegraphics{stack.eps} - -\section{Overview of xlators} - -\flushleft{\LARGE\texttt{cluster/}} -\vskip 2ex -\flushleft{\Large\texttt{afr}} -\vskip 2ex -\flushleft{\Large\texttt{stripe}} -\vskip 2ex -\flushleft{\Large\texttt{unify}} - -\vskip 4ex -\flushleft{\LARGE\texttt{debug/}} -\vskip 2ex -\flushleft{\Large\texttt{trace}} -\vskip 2ex -The trace xlator simply logs all fops and mops, and passes them through to its child. - -\vskip 4ex -\flushleft{\LARGE\texttt{features/}} -\flushleft{\Large\texttt{posix-locks}} -\vskip 2ex -This xlator implements \textsc{posix} record locking semantics over -any kind of storage. - -\vskip 4ex -\flushleft{\LARGE\texttt{performance/}} - -\flushleft{\Large\texttt{io-threads}} -\vskip 2ex -\flushleft{\Large\texttt{read-ahead}} -\vskip 2ex -\flushleft{\Large\texttt{stat-prefetch}} -\vskip 2ex -\flushleft{\Large\texttt{write-behind}} -\vskip 2ex - -\vskip 4ex -\flushleft{\LARGE\texttt{protocol/}} -\vskip 2ex - -\flushleft{\Large\texttt{client}} -\vskip 2ex - -\flushleft{\Large\texttt{server}} -\vskip 2ex - -\vskip 4ex -\flushleft{\LARGE\texttt{storage/}} -\flushleft{\Large\texttt{posix}} -\vskip 2ex -The \texttt{posix} xlator is the one which actually makes calls to the -on-disk filesystem. Currently this is the only storage xlator available. However, -plans to develop other storage xlators, such as one for Amazon's S3 service, are -on the roadmap. - -\chapter{Writing a simple xlator} -\noindent -In this section we're going to write a rot13 xlator. ``Rot13'' is a -simple substitution cipher which obscures a text by replacing each -letter with the letter thirteen places down the alphabet. So `a' (0) -would become `n' (12), `b' would be 'm', and so on.  Rot13 applied to -a piece of ciphertext yields the plaintext again, because rot13 is its -own inverse, since: - -\[ -x_c = x + 13\; (mod\; 26) -\] -\[ -x_c + 13\; (mod\; 26) = x + 13 + 13\; (mod\; 26) = x -\] - -First we include the requisite headers. - -\begin{verbatim} -#include <ctype.h> -#include <sys/uio.h> - -#include "glusterfs.h" -#include "xlator.h" -#include "logging.h" - -/* - * This is a rot13 ``encryption'' xlator. It rot13's data when  - * writing to disk and rot13's it back when reading it.  - * This xlator is meant as an example, not for production - *  use ;) (hence no error-checking) - */ - -\end{verbatim} - -Then we write the rot13 function itself. For simplicity, we only transform lower case -letters. Any other byte is passed through as it is. - -\begin{verbatim} -/* We only handle lower case letters for simplicity */ -static void  -rot13 (char *buf, int len) -{ -  int i; -  for (i = 0; i < len; i++) { -    if (isalpha (buf[i])) -      buf[i] = (buf[i] - 'a' + 13) % 26; -    else if (buf[i] <= 26) -      buf[i] = (buf[i] + 13) % 26 + 'a'; -  } -} -\end{verbatim} - -Next comes a utility function whose purpose will be clear after looking at the code -below. - -\begin{verbatim} -static void -rot13_iovec (struct iovec *vector, int count) -{ -  int i; -  for (i = 0; i < count; i++) { -    rot13 (vector[i].iov_base, vector[i].iov_len); -  } -} -\end{verbatim} - -\begin{verbatim} -static int32_t -rot13_readv_cbk (call_frame_t *frame, -                 call_frame_t *prev_frame, -                 xlator_t *this, -                 int32_t op_ret, -                 int32_t op_errno, -                 struct iovec *vector, -                 int32_t count) -{ -  rot13_iovec (vector, count); - -  STACK_UNWIND (frame, op_ret, op_errno, vector, count); -  return 0; -} - -static int32_t -rot13_readv (call_frame_t *frame, -             xlator_t *this, -             dict_t *ctx, -             size_t size, -             off_t offset) -{ -  STACK_WIND (frame, -              rot13_readv_cbk, -              FIRST_CHILD (this), -              FIRST_CHILD (this)->fops->readv, -              ctx, size, offset); -  return 0; -} - -static int32_t -rot13_writev_cbk (call_frame_t *frame, -                  call_frame_t *prev_frame, -                  xlator_t *this, -                  int32_t op_ret, -                  int32_t op_errno) -{ -  STACK_UNWIND (frame, op_ret, op_errno); -  return 0; -} - -static int32_t -rot13_writev (call_frame_t *frame, -              xlator_t *this, -              dict_t *ctx, -              struct iovec *vector, -              int32_t count,  -              off_t offset) -{ -  rot13_iovec (vector, count); - -  STACK_WIND (frame,  -              rot13_writev_cbk, -              FIRST_CHILD (this), -              FIRST_CHILD (this)->fops->writev, -              ctx, vector, count, offset); -  return 0; -} - -\end{verbatim} - -Every xlator must define two functions and two external symbols. The functions are  -\texttt{init} and \texttt{fini}, and the symbols are \texttt{fops} and \texttt{mops}. -The \texttt{init} function is called when the xlator is loaded by GlusterFS, and  -contains code for the xlator to initialize itself. Note that if an xlator is present -multiple times in the spec tree, the \texttt{init} function will be called each time -the xlator is loaded. - -\begin{verbatim} -int32_t -init (xlator_t *this) -{ -  if (!this->children) { -    gf_log ("rot13", GF_LOG_ERROR,  -            "FATAL: rot13 should have exactly one child"); -    return -1; -  } - -  gf_log ("rot13", GF_LOG_DEBUG, "rot13 xlator loaded"); -  return 0; -} -\end{verbatim} - -\begin{verbatim} - -void  -fini (xlator_t *this) -{ -  return; -} - -struct xlator_fops fops = { -  .readv        = rot13_readv, -  .writev       = rot13_writev -}; - - -\end{verbatim} - -\end{document} - diff --git a/doc/hacker-guide/lock-ahead.txt b/doc/hacker-guide/lock-ahead.txt deleted file mode 100644 index 63392b7fa2c..00000000000 --- a/doc/hacker-guide/lock-ahead.txt +++ /dev/null @@ -1,80 +0,0 @@ -			Lock-ahead translator -                        --------------------- - -The objective of the lock-ahead translator is to speculatively -hold locks (inodelk and entrylk) on the universal set (0 - infinity -in case of inodelk and all basenames in case of entrylk) even -when a lock is requested only on a subset, in anticipation that -further locks will be requested within the same universal set. - -So, for example, when cluster/replicate locks a region before -writing to it, lock-ahead would instead lock the entire file. -On further writes, lock-ahead can immediately return success for  -the lock requests, since the entire file has been previously locked. - -To avoid starvation of other clients/mountpoints, we employ a -notify mechanism, described below. - -typedef struct { -        struct list_head subset_locks; -} la_universal_lock_t; - -Universal lock structure is stored in the inode context. - -typedef struct { -	enum {LOCK_AHEAD_ENTRYLK, LOCK_AHEAD_FENTRYLK, -              LOCK_AHEAD_INODELK, LOCK_AHEAD_FINODELK}; - -        union { -	      fd_t *fd; -              loc_t loc; -	}; - -	off_t l_start; -        off_t l_len; - -	const char *basename; - -	struct list_head universal_lock; -} la_subset_lock_t; - - -fops implemented: - -* inodelk/finodelk/entrylk/fentrylk: - -lock: -  if universal lock held: -    add subset to it (save loc_t or fd) and return success -  else: -    send lock-notify fop -    hold universal lock and return -      (set inode context, add subset to it, save loc_t or fd) -     -    if this fails: -      forward the lock request - -unlock: -  if subset exists in universal lock: -    delete subset lock from list -  else: -    forward it - -* release: -    hold subset locks (each subset lock using the saved loc_t or fd) -    and release universal lock - -* lock-notify (on unwind) (new fop) -    hold subset locks and release universal lock - - -lock-notify in locks translator: - -if a subset lock in entrylk/inodelk cannot be satisfied -because of a universal lock held by someone else: -    unwind the lock-notify fop - -============================================== -$ Last updated: Tue Feb 17 11:31:18 IST 2009 $ -$ Author: Vikas Gorur <vikas@gluster.com>  $ -============================================== diff --git a/doc/hacker-guide/posix.txt b/doc/hacker-guide/posix.txt deleted file mode 100644 index d0132abfedc..00000000000 --- a/doc/hacker-guide/posix.txt +++ /dev/null @@ -1,59 +0,0 @@ ---------------- -* storage/posix ---------------- - -- SET_FS_ID - -  This is so that all filesystem checks are done with the user's  -  uid/gid and not GlusterFS's uid/gid. - -- MAKE_REAL_PATH -  -  This macro concatenates the base directory of the posix volume -  ('option directory') with the given path. - -- need_xattr in lookup - -  If this flag is passed, lookup returns a xattr dictionary that contains -  the file's create time, the file's contents, and the version number -  of the file. - -  This is a hack to increase small file performance. If an application  -  wants to read a small file, it can finish its job with just a lookup  -  call instead of a lookup followed by read. - -- getdents/setdents - -  These are used by unify to set and get directory entries. - -- ALIGN_BUF -   -  Macro to align an address to a page boundary (4K). - -- priv->export_statfs - -  In some cases, two exported volumes may reside on the same -  partition on the server. Sending statvfs info for both -  the volumes will lead to erroneous df output at the client, -  since free space on the partition will be counted twice. - -  In such cases, user can disable exporting statvfs info -  on one of the volumes by setting this option. - -- xattrop - -  This fop is used by replicate to set version numbers on files.  - -- getxattr/setxattr hack to read/write files - -  A key, GLUSTERFS_FILE_CONTENT_STRING, is handled in a special way by -  getxattr/setxattr. A getxattr with the key will return the entire -  content of the file as the value. A setxattr with the key will write -  the value as the entire content of the file. - -- posix_checksum -   -  This calculates a simple XOR checksum on all entry names in a -  directory that is used by unify to compare directory contents. - - diff --git a/doc/hacker-guide/replicate.txt b/doc/hacker-guide/replicate.txt deleted file mode 100644 index fd1ef2747f0..00000000000 --- a/doc/hacker-guide/replicate.txt +++ /dev/null @@ -1,206 +0,0 @@ ---------------- -* cluster/replicate ---------------- - -Before understanding replicate, one must understand two internal FOPs: - -GF_FILE_LK: -  This is exactly like fcntl(2) locking, except the locks are in a  -  separate domain from locks held by applications. - -GF_DIR_LK (loc_t *loc, char *basename): -  This allows one to lock a name under a directory. For example, -  to lock /mnt/glusterfs/foo, one would use the call: - -  GF_DIR_LK ({loc_t for "/mnt/glusterfs"}, "foo") - -  If one wishes to lock *all* the names under a particular directory, -  supply the basename argument as NULL. - -  The locks can either be read locks or write locks; consult the  -  function prototype for more details. - -Both these operations are implemented by the features/locks (earlier -known as posix-locks) translator. - --------------- -* Basic design --------------- - -All FOPs can be classified into four major groups: - - - inode-read -   Operations that read an inode's data (file contents) or metadata (perms, etc.). - -   access, getxattr, fstat, readlink, readv, stat. - - - inode-write -   Operations that modify an inode's data or metadata. - -   chmod, chown, truncate, writev, utimens. - - - dir-read -   Operations that read a directory's contents or metadata. - -   readdir, getdents, checksum. - - - dir-write -   Operations that modify a directory's contents or metadata. - -   create, link, mkdir, mknod, rename, rmdir, symlink, unlink. - -   Some of these make a subgroup in that they modify *two* different entries: -        link, rename, symlink. - - - Others -   Other operations. - -   flush, lookup, open, opendir, statfs. - ------------- -* Algorithms ------------- - -Each of the four major groups has its own algorithm: - - ---------------------- - - inode-read, dir-read - ---------------------- - - = Send a request to the first child that is up: -   - if it fails: -       try the next available child -   - if we have exhausted all children: -       return failure - - ------------- - - inode-write - ------------- - - All operations are done in parallel unless specified otherwise. - - (1) Send a GF_FILE_LK request on all children for a write lock on  -     the appropriate region -            (for metadata operations: entire file (0, 0) -             for writev: (offset, offset+size of buffer)) - -     - If a lock request fails on a child: -         unlock all children -         try to acquire a blocking lock (F_SETLKW) on each child, serially. -	  -	 If this fails (due to ENOTCONN or EINVAL): -           Consider this child as dead for rest of transaction. - - (2) Mark all children as "pending" on all (alive) children  -     (see below for meaning of "pending"). - -     - If it fails on any child: -         mark it as dead (in transaction local state). - - (3) Perform operation on all (alive) children. - -     - If it fails on any child: -         mark it as dead (in transaction local state). - - (4) Unmark all successful children as not "pending" on all nodes. - - (5) Unlock region on all (alive) children. -      - ----------- - - dir-write - ----------- - - The algorithm for dir-write is same as above except instead of holding - GF_FILE_LK locks we hold a GF_DIR_LK lock on the name being operated upon. - In case of link-type calls, we hold locks on both the operand names. - ------------ -* "pending" ------------ - - The "pending" number is like a journal entry. A pending entry is an - array of 32-bit integers stored in network byte-order as the extended - attribute of an inode (which can be a directory as well). -  - There are three keys corresponding to three types of pending operations: - - - AFR_METADATA_PENDING -     There are some metadata operations pending on this inode (perms, ctime/mtime,  -     xattr, etc.). - - - AFR_DATA_PENDING -     There is some data pending on this inode (writev). - - - AFR_ENTRY_PENDING -     There are some directory operations pending on this directory -     (create, unlink, etc.). -     ------------ -* Self heal ------------ - - - On lookup, gather extended attribute data: -   - If entry is a regular file: -     - If an entry is present on one child and not on others: -       - create entry on others. -     - If entries exist but have different metadata (perms, etc.): -       - consider the entry with the highest AFR_METADATA_PENDING number as -         definitive and replicate its attributes on children. - -   - If entry is a directory: -     - Consider the entry with the higest AFR_ENTRY_PENDING number as -       definitive and replicate its contents on all children. - -   - If any two entries have non-matching types (i.e., one is file and -     other is directory): -     - Announce to the user via log that a split-brain situation has been -       detected, and do nothing. -  - - On open, gather extended attribute data: -   - Consider the file with the highest AFR_DATA_PENDING number as -     the definitive one and replicate its contents on all other -     children. - - During all self heal operations, appropriate locks must be held on all - regions/entries being affected. - ---------------- -* Inode scaling ---------------- - -Inode scaling is necessary because if a situation arises where: -  - An inode number is returned for a directory (by lookup) which was -    previously the inode number of a file (as per FUSE's table), then -    FUSE gets horribly confused (consult a FUSE expert for more details). - -To avoid such a situation, we distribute the 64-bit inode space equally -among all children of replicate. - -To illustrate: - -If c1, c2, c3 are children of replicate, they each get 1/3 of the available -inode space: - -Child:        c1   c2   c3   c1   c2   c3   c1   c2   c3   c1   c2 ... -Inode number: 1    2    3    4    5    6    7    8    9    10   11 ... - -Thus, if lookup on c1 returns an inode number "2", it is scaled to "4" -(which is the second inode number in c1's space). - -This way we ensure that there is never a collision of inode numbers from -two different children. - -This reduction of inode space doesn't really reduce the usability of  -replicate since even if we assume replicate has 1024 children (which would be a -highly unusual scenario), each child still has a 54-bit inode space. - -2^54 ~ 1.8 * 10^16 - -which is much larger than any real world requirement. - - -============================================== -$ Last updated: Sun Oct 12 23:17:01 IST 2008 $ -$ Author: Vikas Gorur <vikas@gluster.com>  $ -============================================== - diff --git a/doc/hacker-guide/write-behind.txt b/doc/hacker-guide/write-behind.txt deleted file mode 100644 index a6e9a8890e4..00000000000 --- a/doc/hacker-guide/write-behind.txt +++ /dev/null @@ -1,45 +0,0 @@ -basic working --------------- - -  write behind is basically a translator to lie to the application that the write-requests are finished, even before it is actually finished. - -  on a regular translator tree without write-behind, control flow is like this: -   -  1. application makes a write() system call. -  2. VFS ==> FUSE ==> /dev/fuse. -  3. fuse-bridge initiates a glusterfs writev() call. -  4. writev() is STACK_WIND()ed upto client-protocol or storage translator. -  5. client-protocol, on receiving reply from server, starts STACK_UNWIND() towards the fuse-bridge. - -  on a translator tree with write-behind, control flow is like this: -   -  1. application makes a write() system call. -  2. VFS ==> FUSE ==> /dev/fuse. -  3. fuse-bridge initiates a glusterfs writev() call. -  4. writev() is STACK_WIND()ed upto write-behind translator. -  5. write-behind adds the write buffer to its internal queue and does a STACK_UNWIND() towards the fuse-bridge. -   -  write call is completed in application's percepective. after STACK_UNWIND()ing towards the fuse-bridge, write-behind initiates a fresh writev() call to its child translator, whose replies will be consumed by write-behind itself. write-behind _doesn't_ cache the write buffer, unless 'option flush-behind on' is specified in volume specification file. - -windowing ---------- - -  write respect to write-behind, each write-buffer has three flags: 'stack_wound', 'write_behind' and 'got_reply'. - -  stack_wound: if set, indicates that write-behind has initiated STACK_WIND() towards child translator.  - -  write_behind: if set, indicates that write-behind has done STACK_UNWIND() towards fuse-bridge. - -  got_reply: if set, indicates that write-behind has received reply from child translator for a writev() STACK_WIND(). a request will be destroyed by write-behind only if this flag is set. - -  currently pending write requests = aggregate size of requests with write_behind = 1 and got_reply = 0. -   -  window size limits the aggregate size of currently pending write requests. once the pending requests' size has reached the window size, write-behind blocks  writev() calls from fuse-bridge.  -  blocking is only from application's perspective. write-behind does STACK_WIND() to child translator straight-away, but hold behind the STACK_UNWIND() towards fuse-bridge. STACK_UNWIND() is done only once write-behind gets enough replies to accomodate for currently blocked request. -   -flush behind ------------- - -  if 'option flush-behind on' is specified in volume specification file, then write-behind sends aggregate write requests to child translator, instead of regular per request STACK_WIND()s. -   -  | 
