cc by-sa flurdy

ec2 - Amazon Elastic Compute Cloud tips and howtos

Bacup tips (s3sync)

This page is part of larger set of tips & howtos on ec2 by flurdy.
| More
Other ec2 docs by flurdy

Document state: draft

This is a short page detailing backup strategies and tools to use with ec2 instances.
In particular how to use s3sync to synchronise backups with Amazon S3.

Pre-requisites

Related


Why backup?

		maybe obvious
		ebs cover continously changing data
		ami cover quite static configuration and data
		reliable fault tolerant s3
		having data locally
		quick restore of ami after instance death or clustering
		ebs and use snapshots?
		not a replacement for proper enterprise backup
	

Register with S3

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

	

Planning

		plan s3 layout
		buckets, nameing strategy
		s3 restrictions
		plan what to backup, when to backup
			dont backup ami data at least not often
			ebs data is also not as volatile
			dont flood s3 which makes it untity and confusing
			but also dont miss out on vital data
			as s3 is relatively cheap so never not backup anything due to s3 cost
	

Get S3Sync

		if not already part of AMI
		wget http://s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz
		tar xzf s3sync.tar.gz
		sudo mv s3sync /opt/
		chown -R billy:nomate /opt/s3sync
		aptitude install ruby
		sudo vi /etc/profile.d/s3sync
		PATH=$PATH:/opt/s3sync
		export PATH
	

Learn s3sync

		create local accessKeys file as seen on the AMI page
		Read the s3sync articles
		play with s3sync
		s3sync.rb is an overkill, s3cmd.rb put and get should be enough
	

Create backups

		sudo mdir /mnt/backup
		sudo chown /mnt/backup
		cd /mnt/backup
		mkdir conf data logs home
		sudo tar czf conf/conf-etc-20090101-01.tar.gz /etc
		sudo tar czf conf/conf-etc-apache-20090101-01.tar.gz \
			/etc/apache2 /etc/default/apache2 /etc/ssl
		sudo tar czf conf/conf-etc-mail-20090101-01.tar.gz \
			/etc/postfix /etc/courier /etc/amavis /etc/clamav
			/etc/postgrey /etc/spamassassin \
			/etc/default/postgrey /etc/default/spamassassin
		sudo tar czf conf/conf-etc-shorewall-20090101-01.tar.gz \
			/etc/shorewall /etc/default/shorewall
		sudo tar czf conf/conf-etc-tomcat-20090101-01.tar.gz \
			/etc/tomcat /etc/apache2 /etc/default/tomcat
		sudo /etc/init.d/apache2 stop
		sudo /etc/init.d/courier-authdaemon stop
		sudo /etc/init.d/courier-imap-ssl stop
		sudo /etc/init.d/courier-imap stop
		sudo /etc/init.d/postfix stop
		sudo /etc/init.d/tomcat6 stop
		sudo tar czf data/var-www-20090101-01.tar.gz /var/www
		sudo tar czf data/var-mail-20090101-01.tar.gz /var/spool/mail
		sudo tar czf data/var-lib-tomcat-20090101-01.tar.gz /var/lib/tomcat6
		sudo tar czf home/home-username-20090101-01.tar.gz /home/username
		tar --newer-mtime
		mysqldump --skip-opt --skip-quote-names --skip-set-charset --no-data --compact -u USER -pPASSWORD maildb > /tmp/mysql-mail-schema.sql
		mysqldump --skip-opt --skip-quote-names --skip-set-charset --no-create-info --compact -u USER -pPASSWORD maildb > /tmp/mysql-mail-data.sql
		tar cfz data/data-mysql-maildb-20090101-01.tar.gz /tmp/mysql-mail-schema.sql /tmp/mysql-mail-data.sql
		be aware of symlinks
	

Upload backups

		/opt/s3sync/s3cmd.rb put \
			s3bucket:path/conf/conf-etc.2009-11-13.1.tar.gz \
			conf-etc.2009-11-13.1.tar.gz
	

Create script & cron it

		sudo vi /usr/local/bin/backupLocally.sh
		sudo vi /usr/local/bin/backupToS3.sh
		routines
		cron
		automation
		backup backup scripts as well
	

Restoring

		New instance
	

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

	



back to flurdy's ec2 docs

flurdy