In this post I wanted to highlight how I back up individual WordPress installations on the Linux server handling ePortfolios and store these on a shared folder on a Windows Server 2008 machine. From there they are part of the backup/archiving process which is beyond my remit.

All was going well until we had to replace our Linux backup machine which I used SCP to copy to. We shifted the backups to a Windows 2008 server copying to it via a Samba share. As this Munin chart indicates that caused a few issues. The blue area represents running processes, the orange are un-interruptible processes and the yellow stars are server reboots. The graph covers March 2014 – April 2015 so you can tell we had a long stable period of reliable backups until November 2014 when the server was replaced.

There are hundreds of individual sites to backup so using one massive backup file wouldn’t be appropriate. I use a shell script to construct individual ePortfolio zip backups, shift them to the backup location, then delete the source zip file and move onto the next one. The database tables match the folder names so there’s a simple method to get a list of file folders and database tables to be zipped.

I use a Linux shell script a bit like this (Where you see ${File} this text is replaced by the folder name in the running script):

#!/bin/bash
#find out todays date in the format dd_mm_yy
Today="`date +%d_%m_%y`" 
for File in `ls /var/www/eportfolios`; do

    #dump the database tables for the current ePortfolio
    mysqldump -uyourdbuser  -pyourdbpass -C -Q -e --create-options  eportfolios
    ${File}_commentmeta ${File}_comments ${File}_links ${File}_options
    ${File}_postmeta   > /var/tempbackup/${File}.sql
 
    #zip the wordpress file folders
    zip -r  /var/tempbackup/eportfolio_${File}_$Today.zip  /var/www/eportfolios/${File}/wp-config.php /var/www/eportfolios/${File}/wp-content 
    #drop the database dump file into the zip file
    zip -gj /var/tempbackup/eportfolio_${File}_$Today.zip /var/tempbackup/${File}.sql

    #COPY ZIP FILE TO BACKUP SERVER HERE!

    #Delete the local files and move to the next one
    rm /var/tempbackup/eportfolio_${File}_$Today.zip
    rm /var/tempbackup/${File}.sql   
 
done;

Notice that I don’t backup the whole WordPress folder – only the wp-config.php file and the wp-content folder. That makes the backup dramatically smaller because the rest of the WordPress files are the same in every ePortfolio. If we need to restore an ePortfolio we expand the standard set of WordPress files and introduce the config and content from the backup.

Now it’s time to fill in the line which does the copy

#COPY ZIP FILE TO BACKUP SERVER HERE!

I have used three different methods in the past. I use the eportfolio name joebloggs for the examples.

1. SCP

Originally the backup target was another Linux machine so this was quite straightforward. Following the instructions on this Oracle blog I set up the two machines for scp without passwords using shared keys.

The command to copy was:

scp -P yourportnumber  /var/tempbackup/joebloggs_eportfolio_05_04_15.zip yourbackupuser@backupserver:/home/yourbackupuser/

This command was very fast copying and there were never any processes left in an uninterruptible state.

2. Simple file copying to Samba share

The problem was this server reached EOL and the replacement was a Windows Server 2008 box. Setting up SCP on a Windows box seemed to require purchasing something so I decided to use a Windows share with a Samba mount on the Linux machine.

I could now do a simple copy to the share:

cp /var/tempbackup/joebloggs_eportfolio_05_04_15.zip /WindowsShares/Backups/

This appeared to be working well but after while, in the process list, I started to notice unfinished jobs and not all ePortfolios had been backed up. These uninterruptible jobs cannot be killed so continue to hog resources until the server requires a restart. Thankfully this happened over a period of weeks. It turns out that the data transfer rate in the Linux->Windows Samba copy was also painfully slow so I had to come up with an alternative when time permitted.

3. FTP

In the end I decided to try FTP as this is relatively simple to set up through Internet Information Services on Windows Server and easy to build into a shell script on Linux.

I install the package lftp on our Ubuntu 12.04 Server as it has very powerful command line functionality. This is the command I used to execute the transfer:

lftp -e "mput /var/tempbackup/joebloggs_eportfolio_05_04_15.zip;quit" -u "targetuser,targetpassword" "ftp://TARGETSERVERIP/Backups/"

With FTP we are getting the performance of SCP without having to buy more software and avoiding the issues the Samba copy was creating. You can see that the process count has returned to normal in the FTP part of the Munin chart above.