Kissdx svn hook

From Kissdx-wiki

1 Kissdx auto information to developers on svn update

Kissdx auto information to developers on svn update

General configuration of svn

# ll /kiss2/kissdx/svn
drwxr-xr-x  7 www-data svn  4096 2006-11-10 22:25 kissdx        ==> Official svn
drwxr-xr-x  7 www-data svn  4096 2006-10-29 12:33 kissdx_study  ==> Testing area

# ll kissdx    ==> user www-data is readeable to enable web2svn access
drwxr-xr-x  2 www-data svn 4096 2006-11-10 22:25 conf
drwxr-xr-x  2 www-data svn 4096 2006-11-12 13:01 dav
drwxr-sr-x  5 www-data svn 4096 2008-03-11 23:40 db
-r--r--r--  1 www-data svn    2 2006-11-10 22:25 format
drwxr-xr-x  2 www-data svn 4096 2007-02-18 22:16 hooks
drwxr-xr-x  2 www-data svn 4096 2006-11-10 22:25 locks

# ll hooks/   ==> only file without extension is used by svn ==> post-commit is the only one.
-rwxr-xr-x  1 www-data svn   1406 2007-02-18 22:16 post-commit
-rwxr-xr-x  1 okahn    okahn 1544 2006-12-06 22:39 post-commit.save.2007.02.18
-rw-r--r--  1 www-data svn   1377 2006-12-06 07:46 post-commit.tmpl
-rw-r--r--  1 www-data svn   1508 2006-11-10 22:25 post-revprop-change.tmpl
-rw-r--r--  1 www-data svn   2363 2006-11-10 22:25 pre-commit.tmpl
-rw-r--r--  1 www-data svn   1952 2006-11-10 22:25 pre-revprop-change.tmpl
-rw-r--r--  1 www-data svn   1533 2006-11-10 22:25 start-commit.tmpl

[edit]

Configuration of svn hook

First try were to send a email and create a dump full repo to sent to backup server with ftp transfer. Experience shows that each commit user action waits for the end of the server hook action to finish the commit process on client side. This creates a 4 to 10 sec wait for each user commit. Actually frustrating for the user.

Second try were :

Each user commit send an email for developer information, and set a flag "Candidate for full dump"

Each hour, a full dump is triggered and send it on backup server if the flag is set

cat post-commit
#!/bin/sh
# POST-COMMIT HOOK receiving following parameters
#   [1] REPOS-PATH   (the path to this repository)
#   [2] REV          (the number of the revision just committed)
#
# This shell is ran under the user "www-data" due to the activation remotly via Apache2/dav system
# Tips : This file should be Executable "x" for the user www-data 
#  (careful, default save via samba is  not "x"
# Tips : Take care to have any resources (log, path...) available for user "www-data"

cd /kiss2/kissdx/svn_commit_log/
REPOS="$1"
REV="$2"
AUTHOR=$(svnlook author --revision $REV $REPOS)
WHOAMI=$(whoami)
############################################################
/usr/bin/logger "[Kissdx] SvnPostCommit: === START - POST COMMIT === running as ($WHOAMI) ====="
/usr/bin/logger "[Kissdx] SvnPostCommit: = CommitFlag set: author[$AUTHOR] repo[$REPOS] rev[$REV]" 
/usr/lib/subversion/hook-scripts/commit-email.pl "$REPOS" "$REV" --from "admin@kissdx.com" \
      -s "[$REPOS] $AUTHOR -" s.kahn@free.fr

# 2007.02.18-OKahn - Avoid a full dump consuming 4sec on each end user commit (too long)
#/kiss2/kissdx/kissdx_s.svnadmin.dump.sh 1>> /kiss2/kissdx/svn_commit_log/kissdx_study.save.log 2>&1

touch /kiss2/kissdx/svn_commit_log/kissdx_commit_flag

/usr/bin/logger "[$REPOS] SvnPostCommit: === END of POST COMMIT ========================"
##############################################################

[edit]

Configuration of save dump on backup server

Crontab

0 * * * * /kiss2/kissdx/kissdx.svnadmin.dump.sh 1>> /kiss2/kissdx/kissdx.save.log 2>&1

/kiss2/kissdx/kissdx.svnadmin.dump.sh
#!/bin/sh
# subversion repository full dump and store with ftp
# Tips : This shell is executed under user "www-data" (comes from apache2/dav)
# (c) 2006 Olivier Kahn under terms of GNU GPL v.2 or later
# History
# 2007.02.18 Initialy launched form a post commit hook in svn, (resp.time:4sec is not affordable)
# 2007.02.18 New architecture : each commit create a flag. This full dump shell is crontab each hour
#             Testing the existence of flag  /kiss2/kissdx/svn_commit_log/kissdx_study_commit_flag
#             Repository may be a parameter (based on svn standard)

#/usr/bin/logger "DEBUG ==== SvnAdmin Dump start"
# set path due to security reason of svn that create an empty environement
export PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11
#/usr/bin/logger "DEBUG  == SvnAdmin Path [$PATH]"

REPO=kissdx
SVNROOT=/kiss2/kissdx/svn
COMMIT_FLAG=/kiss2/kissdx/svn_commit_log/kissdx_commit_flag
STORE=/kiss2/kissdx/dump

#GPGCRYPT=n
#EMAIL=your@email.here

REPOPATH=$SVNROOT/$REPO;
REPOSAVE=dump_kissdx_`date +%Y.%m.%d-%H.%M.%S`;
YOUNGEST=`/usr/bin/svnlook youngest $REPOPATH`;
STARTTIME=`/bin/date -u +%Y.%m.%d-%H.%M`;

# -- Test of flag (if not, no need to full dump) ----------------
if [ -e $COMMIT_FLAG ]
then
 echo "[$STARTTIME][$REPO] Commit_Flag found: $COMMIT_FLAG"
 /usr/bin/logger "[$REPO] Commit_Flag found: $COMMIT_FLAG"
else
 echo "[$STARTTIME][$REPO] Commit_Flag not found: $COMMIT_FLAG"
 /usr/bin/logger "[$REPO] No flag, No need to launch SVN Full Dump"
 exit 0 
fi

#/usr/bin/logger "DEBUG creation of a full dump repository named : $REPOSAVE"
#echo ===== Begin Dump of kissdx svn [$STARTTIME]=======
#echo YOUNGEST=$YOUNGEST;
#echo REPO=$REPO;
#echo REPOPATH to dump =$REPOPATH;
#echo REPOSAVE to store=$REPOSAVE;
#echo === svnadmin dump ...
/usr/bin/svnadmin dump $REPOPATH >$STORE/$REPOSAVE 
/usr/bin/logger "[$REPO] Svnadmin Full Dump .... $STORE/$REPOSAVE" 

#echo === tar dump in $STORE/$REPOSAVE.tar.bz2 ...
/bin/tar  -cjf $STORE/$REPOSAVE.tar.bz2 $STORE/$REPOSAVE
if [ $? -eq 0 ]; then
       #echo "tar successful and full dump source removed"
       rm -f $STORE/$REPOSAVE
else
       #echo "tar of svn dump failed"
       exit 1
fi
LS=`ls -al $STORE/$REPOSAVE.tar.bz2`;
/usr/bin/logger "[Kissdx] $LS"

echo === ftp on ctj2ee.free.fr
cd $STORE
/usr/bin/ftp -n ftpperso.free.fr  <<!
quote user MYUSER
quote pass MYPASS
binary
cd Public
cd kissdx-save
put $REPOSAVE.tar.bz2
quit
!
echo === end ftp ctj2ee.free.fr

# Store on local NAS (disk in miror)
#echo === ftp on 10.39.10.16
#cd $STORE
#ftp -n 10.39.10.16  <<!
#quote user admin
#quote pass admin
#binary
#cd Public
#cd kissdx-save
#put $REPOSAVE.tar.bz2
#quit
#!
#echo === end ftp 10.39.10.16

echo ===== End Dump of kissdx svn =======

#/usr/bin/logger SvnAdmin Dump $STORE/$REPOSAVE.tar.bz2
/usr/bin/logger "[$REPO] SvnAdmin FTP ctj2ee.free.fr $REPOSAVE.tar.bz2"

# part of code found to focus on a incremental backup (not used for kissdx)
#if [ `ls $STORE | wc -w` -gt 0 ]; then
#       BACKUP=repodiff_`date -u +%Y%m%d%H%M%S`.tar.bz2
#       ATTACH=$STORE/../$BACKUP
#       tar  -C $STORE -cjf $ATTACH .
#       if [ $GPGCRYPT = y ]; then
#               gpg -e -r $EMAIL $ATTACH
#               ATTACH=$ATTACH.gpg
#       fi
#       echo "." | mutt -c $EMAIL -a $ATTACH -s "repository incremental backup"
#       rm -f $ATTACH
#fi
#rm -rf $STORE

# Remove the commit flag 
rm -f $COMMIT_FLAG

[edit]