Results 1 to 2 of 2

Thread: script to tar and backup directories to a remote ftp site

  1. #1
    Ramblin is offline Sugar Community Member
    Join Date
    May 2010
    Posts
    199

    Default script to tar and backup directories to a remote ftp site

    You can either use the Webmin>System>Filesystem Backup module OR use the script I have created (see below) to backup directories to an ftp site. Do ONLY one or the other, NOT both.

    I did the scripts since setting up the Webmin module for each directory got a bit tiring, especially if I was doing multiple servers/systems. Your choice.

    To setup the file backups with the script in this posting:
    Create a new directory in /root called /backup-scripts
    Create a new file in /root/backup-scripts called backup_crm.sh (it does more than backup the crm if you want it to)
    Change the permissions of the new backup_crm.sh file to allow execution by the root user (I changed my permissions to 755; I have the defualt at 644)
    Copy the following contents into the backup_crm.sh file
    Code:
    - (I put the contents into the next posting since a single posting exceeded the character count of this forum)
    Edit the ftp parameters (near the top of the backup_crm.sh file) to put your ftp parameters in - the remote site to which you want to upload your files
    - Host url
    - UserID
    - Password
    Edit the name of the server (near the top of the backup_crm.sh file) from which the files are being uploaded (just a text description for you to identify it)
    Edit the list of directories (near the top of the backup_crm.sh file) to be the list you want backed up
    Add to the crontab file in /etc the following new entry
    Code:
    # Run the file backup program to copy key files to remote ftp site
    7 3 * * * root /root/backup-scripts/backup_crm.sh >/dev/null 2>&1
    That's it! The directories you listed will be backed up each night of the week, with next Monday replacing this Monday's files, next Tuesday replacing this Tuesday's files, ... and additionally once a month for longer-term archiving.
    - The directories you listed will be stored locally and then uploaded to the ftp site from the local store.
    - You an restore from the ftp if you have a catastrophic failure (hard drive crash requiring entire system recreation) or you can restore from the files stored locally if there is just a local issue.

  2. #2
    Ramblin is offline Sugar Community Member
    Join Date
    May 2010
    Posts
    199

    Default Re: script to tar and backup directories to a remote ftp site

    The code for the backup_crm.sh file is:
    Code:
    #!/bin/bash
    # 
    # This program is open source, free for use by anyone, subject to GPLv2 licensing
    # All caveats regarding use of open-source programs apply; 
    # I did my best but it is your responsibility to understand the program and check its suitability for you
    # Authored by Richard Cantin, Ayuda, www.ayuda.ca
    # 
    # This program backs up the directories in the list shown below 
    # - to a $Target_Directory directory predefined as tmp/backups (creates one if it does not already exist)
    # - to a remote ftp site (you specify)
    #
    # The program creates a backup every night and stores 
    # - a rolling week's worth of backups (Monday through Sunday each day has its own backup)
    # - a monthly backup with each month kept until it is replaced by next year's equivalent monthly backup
    # 
    # If you want the same setup I have, the program will work as is, without any editing, other than
    # - declaring the name of the server (text field for you use only)
    # - declaring the directories you want backed up
    # - declaring the ftp parameters 
    # 
    # To install and configure this program (assuming you want to stay with my settings (below)
    # Create a /backup-scripts directory in the /root directory and put this file, named backup_crm.sh, with executable permission,into the /root/backup-scripts directory 
    # Add to crontab the schedule for this program 
    # (with these settings, the Program runs every night at 3:07AM)
    #    7 3 * * * root /root/backup-scripts/backup_crm.sh >/dev/null 2>&1
    #
    # If there is an error in either the tar process or the ftp transfer the program sends an email to the root user of the server
    # So if you have root aliased to an admin account or if you want to change the MailTo below
    # you will get notified of any errors.
    #
    #####################################################
    #
    # Your ftp site parameters - you need to specify your ftp site
    # (Note: ftp_host is the full url to your ftp site)
    #
    
    ftp_host="put_your_host_url_in_here_between_the_quotes"
    ftp_userid=" put_your_ftp_userid_in_here_between_the_quotes"
    ftp_password=" put_your_ftp_password_in_here_between_the_quotes"
    
    #####################################################
    #
    # The Server Name - text entry to describe which server is being backed up (Useful for admins who look after more than one server)
    #
    
    Server_Name="put_a_(short)_text_description_of_your_server_here"
    
    # The list of Directories to be backed up
    # You can add to this list of Directories for each additional Directory you want to back up
    #   - add a new Array entry with a number next in sequence (sequence starts at 1)
    #   - You do NOT need to start or end the Directory name with a / 
    # correct sample: Source_Directory[1]="var/www/html"
    # incorrect sample: Source_Directory[1]="/var/www/html/"
    #
    
    Source_Directory[1]="put_your_first_directory_here"
    Source_Directory[2]=" put_your_second_directory_here "
    Source_Directory[3]=" add_as_many_as_you_like "
    Source_Directory[4]=" delete_these_samples "
    
    #####################################################
    #
    # This defines variables and parameters used in the program
    # You should NOT be making changes to these parameters unless you plan to re-write the program
    #
    
    date_for_monthly_backup="03"
    
    this_date=$(date +"%d")
    
    this_month=$(date +"%m-%b")
    this_weekday=$(date +"%u-%a")
    
    File_Prefix="prod"
    File_Suffix=".tar.gz"
    
    Target_Directory="/tmp/backups/"
    
    # Set parameters used to e-mail errors
    Mailer="/bin/mail"
    MailTo="root"
    
    #####################################################
    #
    # Do NOT edit anythihng below here - this is the core program
    #
    
    #Create the names (Daily and Monthly) for each Directory backup file
    Num_Backup_Directories=${#Source_Directory[@]}
    Dir_Count=1
    while [ ${Dir_Count} -le ${Num_Backup_Directories} ]
    	do
    		# Strip any spaces from the Source Directory Name
    		Dir_Name=$(echo ${Source_Directory[${Dir_Count}]} | tr -d [:space:] )
    		# Test if the Source Directory name has a leading / and if so, strip it off
    		First_Char=${Dir_Name:0:1}
    		if [ ${First_Char} == "/" ]
    			then
    				New_Dir_Name=${Dir_Name:1}
    			else
    				New_Dir_Name=${Dir_Name}
    		fi
    		# Test if the Source Directory name has a trailing / and if so, strip it off
    		Num_Char=${#New_Dir_Name}
    		let Char_Pointer=${Num_Char}-1
    		Last_Char=${New_Dir_Name:${Char_Pointer}}
    		if [ ${Last_Char} == "/" ]
    			then
    				Final_Dir_Name=${New_Dir_Name:0:${Char_Pointer}}
    			else
    				Final_Dir_Name=${New_Dir_Name}
    		fi
    		Source_Directory[${Dir_Count}]=${Final_Dir_Name}
    		# Replace any remaining / with - for the name to use for the backup file
    		Path_as_Name=$(echo ${Final_Dir_Name} | tr "\/" "-")
    		# Create File Names for Daily and Monthly backup files
    		Backup_Daily_to_Filename[${Dir_Count}]=${File_Prefix}"_"${Path_as_Name}"_"${this_weekday}${File_Suffix}
    		Backup_Monthly_to_Filename[${Dir_Count}]=${File_Prefix}"_"${Path_as_Name}"_m_"${this_month}${File_Suffix}
    		let Dir_Count=Dir_Count+1
    	done
    
    # Check to see if the Directory specified in "Target_Directory" already exists
    # If not, create it
    if [ ! -d "${Target_Directory}" ]
    	then
    		mkdir "${Target_Directory}"
    		echo "New Directory created: "${Target_Directory}
    fi
    
    # Check to see if this is the date to do the Monthly backup as well as the Daily backup
    # If so, set the counter to enable two backups per File - One for Daily and one for Monthly
    if [ ${this_date} -eq ${date_for_monthly_backup} ]
    	then
    		# It is the date to do the Monthly backup as well as the Daily Backup
    		Num_Backups_per_File=2
    	else
    		# It is NOT the date to do the Monthly backup as well as the Daily Backup
    		Num_Backups_per_File=1
    fi
    
    # Create Daily (and if the right date, Monthly) backup tar.gz files and transfer them to the designated ftp site
    Backup_Type[1]="Daily"
    Backup_Type[2]="Monthly"
    cd ${Target_Directory}
    Dir_Count=1
    while [ ${Dir_Count} -le ${Num_Backup_Directories} ]
    	do
    		#
    		# Create Daily (and if the right date, Monthly) backup tar.gz files (with error check if tar does not work)
    		tar -cvzf ${Backup_Daily_to_Filename[${Dir_Count}]} -C / ${Source_Directory[${Dir_Count}]} 2>error-message-daily.txt
    		Error_State=$?
    		wait
    		if [ ${Num_Backups_per_File} -eq 2 ]
    			then
    				cp ${Backup_Daily_to_Filename[${Dir_Count}]} ${Backup_Monthly_to_Filename[${Dir_Count}]}
    				wait
    		fi
    		Backup_Count=1
    		while [ ${Backup_Count} -le ${Num_Backups_per_File} ]
    			do
    				if [ ${Error_State} -eq 0 ]
    					then
    						Comp_Result="The compressed file was created for the "${Backup_Type[${Backup_Count}]}" backup of Directory: "${Source_Directory[${Dir_Count}]}" on Server: ""${Server_Name}"
    						echo ${Comp_Result}
    						# Send the Daily/Monthly backup tar.gz file to the designated ftp site (with error check if transfer does not work)
    						if [ ${Backup_Count} -eq 1 ]
    							then
    								File_to_Transfer=${Backup_Daily_to_Filename[${Dir_Count}]}
    							else
    								File_to_Transfer=${Backup_Monthly_to_Filename[${Dir_Count}]}
    						fi
    						ftp -inv ${ftp_host} >/tmp/backups/ftp.log 2>&1 << end-ftp-session
    							user ${ftp_userid} ${ftp_password}
    							binary
    							lcd ${Target_Directory}
    							put ${File_to_Transfer}
    							bye
    end-ftp-session
    						wait
    						# Check for successful transfer or report errors
    						Message=""
    						if [ $(grep -ic "226 Transfer complete" /tmp/backups/ftp.log) -ge 1 ]
    							then
    								Message="ftp for file ${File_to_Transfer} was successful"
    								echo ${Message}
    							else
    								if [ $(grep -ic "not connected" /tmp/backups/ftp.log) -ge 1 ]
    									then
    										Message="ftp for file ${File_to_Transfer} failed - no connection made to remote site"
    										echo ${Message}
    								fi
    								if [ $(grep -ic "No such file or directory" /tmp/backups/ftp.log) -ge 1 ]
    									then
    										Message="ftp for file ${File_to_Transfer} failed - file does not exist"
    										echo ${Message}
    								fi
    								if [ "${Message}" == "" ]
    									then
    										Message="ftp for file ${File_to_Transfer} failed for an unknown reason"
    										echo ${Message}
    								fi
    								Subject="Error! - ftp for backup of file ${File_to_Transfer} on ${Server_Name} failed"
    								echo "${Message}" | ${Mailer} -s "${Subject}" ${MailTo}
    						fi
    					else
    						Comp_Result="Error! - the compressed file was NOT properly created for the "${Backup_Type[${Backup_Count}]}" backup of Directory: "${Source_Directory[${Dir_Count}]}" on Server: ""${Server_Name}"
    						echo ${Comp_Result}
    						if [ ${Backup_Count} -eq 1 ]
    							then
    								Message=$(cat error-message-daily.txt)
    							else
    								Message="The Monthly backup file of of Directory:"${Source_Directory[${Dir_Count}]}" on Server: ""${Server_Name} failed because the Daily backup was never created.  Fix the Daily backup."
    						fi
    						Subject="Error! - the compressed file was NOT properly created for the "${Backup_Type[${Backup_Count}]}" backup of Directory: "${Source_Directory[${Dir_Count}]}" on Server: ""${Server_Name}"
    						echo "${Message}" | ${Mailer} -s "${Subject}" ${MailTo}
    				fi
    				let Backup_Count=Backup_Count+1
    			done
    		let Dir_Count=Dir_Count+1
    	done
    
    # If get to here, exit with no error flags
    exit 0
    Last edited by Ramblin; 2012-11-19 at 11:15 AM. Reason: To handle cases where leading or trailing / are in directory names

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Remote Backup to Cloud Storage Amazon S3
    By EJH in forum General Discussion
    Replies: 2
    Last Post: 2009-02-02, 06:44 AM
  2. add contacts from a remote script
    By blesone in forum Help
    Replies: 1
    Last Post: 2006-08-20, 08:12 PM
  3. Remote backup
    By joris in forum General Discussion
    Replies: 4
    Last Post: 2006-02-15, 05:29 PM
  4. remote backup
    By joris in forum Feature Requests
    Replies: 0
    Last Post: 2006-02-13, 03:36 PM
  5. Replies: 0
    Last Post: 2005-07-27, 11:47 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •