cc by-sa flurdy

ec2 - Amazon Elastic Compute Cloud tips and howtos

Simple Ubuntu Server in the cloud

This page is part of larger set of tips & howtos on ec2 by flurdy.
| More
Other ec2 docs by flurdy

Setting up an simple server as a base for all future server demands.
Based on Canonical's own release images.
Enables easy launch of already setup and working server instances quickly.
Separated step by step so it should be easy to follow.

Pre requisits

Make sure you know enough about ec2, have elasticfox installed and know about the costs and risks involved.

Base image

In elasticfox or aws console locate your base image. I recommend Canonical's own images: lucid (10.04 LTS) maverick (10.10) natty (11.04) oneiric (11.10) precise (12.04). However images made by Eric Hammond at alestic are also commendable.

At the time of writing the images I would choose are these ones:

Region Arch Store AMI ID AMI Manifest
US-east 32bit ebs ami-ab36fbc2 ubuntu-images/ubuntu-lucid-10.04-i386-server-20110930.manifest.xml
US-east 64bit ebs ami-ad36fbc4 ubuntu-images/ubuntu-lucid-10.04-amd64-server-20110930.manifest.xml
EU-west 32bit instance ami-c00e3cb4 ubuntu-images-eu/ubuntu-lucid-10.04-i386-server.20110930.manifest.xml

Other alternatives exist such as 64bit and EBS as boot store.
(note: small instance is 32bit only, while all others including micro accept 64bit)

If another non official Canonical image is chosen, please make allowances for differences. E.g. Perhaps root is the default user and not ubuntu.

Assumtions:

Change commands below if any different.

Legends to replace

Step by step

Different columns for describing the steps, and whether the commands are to be run locally or on the actual ec2 instance.

Description Commands
Remote ec2 instance Local machine
Launch Canonical Ubuntu base image mentioned above.
Log in from your local machine into the ec2 instance ssh -v -i .ec2/yourAwsKey ubuntu@publicDNSname
Modify package source repositories sudo vi /etc/apt/sources.list
Add restricted and multiverse repositories, by entering this command in VI :%s/main un/main restricted multiverse un/g
:wq
Update apt then upgrade your packages, so that you are up to date. I do get prompted with grub configuration in the upgrade, which I respond to both questions with the OK button. sudo aptitude update;
sudo aptitude safe-upgrade
To be sure the update has not broken your system, reboot it. There is no risk as you have not really done anything with it yet. It will throw your user out when it shuts down. sudo reboot now
After a few minutes re-log in from your local machine into the ec2 instance ssh -v -i .ec2/yourAwsKey ubuntu@publicDNSname
Do a quick check to see if it did reboot last
Add your user, enter password and if any details adduser yourUser
Upload from your local machine your normal SSH public key (or your AWS SSH public key) to this user scp -i .ec2/yourAwsKey .ssh/id_rsa.pub ubuntu@publicDNSname:
Rename (if neccessary) your ssh key file to something recognisable. Append this to the ubuntu users accepted keys, and to your users keys. mv id_rsa.pub .ssh/yourLocalSshKey.pub;
cat .ssh/yourLocalSshKey.pub > .ssh/authorized_keys;
sudo cp .ssh/yourLocalSshKey.pub /home/yourUser/;
sudo chown yourUser:yourUser /home/yourUser/yourLocalSshKey.pub;
sudo su yourUser;
cd;
mkdir .ssh;
mv yourLocalSshKey.pub .ssh/;
cat .ssh/yourLocalSshKey.pub > .ssh/authorized_keys2;
chmod 700 .ssh;
chmod 600 .ssh/authorized_keys2;
exit
In another window try and login with your normal SSH key. First try your normal username, then log out. Then again as the ubuntu user. ssh -v -i .ssh/id_rsa yourUser@publicDNSname
exit ssh -v -i .ssh/id_rsa ubuntu@publicDNSname;
Now still as the ubuntu user in the old shell or a new one lets create some groups and add yourUser to these. sudo usermod -a -G adm,admin yourUser;
Optional:
I use some japanese inspired group selection which you can skip.
I also add a limited billy user for future purposes. 		sudo addgroup shogun;
sudo addgroup samurai;
sudo addgroup ninja;
sudo addgroup nikkei;
sudo addgroup issei;
sudo addgroup nissei;
sudo addgroup sansei;
sudo addgroup yonsei;
sudo addgroup sosen;
sudo usermod -a -G shogun,samurai,ninja,nikkei,issei yourUser;

sudo addgroup nomate;
sudo useradd -g nomate -M billy
Now again try to log in directly as yourUser. ssh -v -i .ssh/id_rsa yourUser@publicDNSname
Now as yourUser session see if you have full sudo pemissions. If okay exit the older ubuntu user session and from now use your yourUser session. sudo last

Stage 1 done



You now do have a workable Ubuntu server. You can skip down to making an AMI image if you want to, but I recommend following the next section first.


wont fix puta
kill bill devil smile
beast mode dumb people
prophet wtf
shirts.flurdy.com
us | uk | eu

New Shorewall guide

Below is an older description, for a more recent Ubuntu (14.04+) please refer to my Shorewall configuration

Description Commands
Remote ec2 instance
Next install some libraries. First the ec2 AMI tools, so that you can make ec2 AMI images. sudo aptitude install ec2-ami-tools
Then a firewall, Shorewall. You could stick with UFW if you like. Yes AWS/ec2 does already have Security groups, but with a local firewall you can be more flexible (and secure). sudo aptitude install shorewall-common shorewall-perl shorewall-doc
Now lets configure shorewall for default usage cd /usr/share/doc/shorewall/default-config/;
sudo cp interfaces zones hosts policy routestopped rules \
/etc/shorewall/;
cd /etc/shorewall
Open the interfaces file.
This defines the physical network interfaces on your instance. 		sudo vi interfaces
Add as second to last line: net   eth0   detect   dhcp,tcpflags,logmartians,nosmurfs
Open the zones file.
What we call the different network zones that may call this instance. 		sudo vi zones
Include and append after the firewall line: fw    firewall
ec2   ipv4
net   ipv4
Open the hosts file.
How we define zones that is not defined by network interface. 		sudo vi hosts
Add as second to last line: ec2   eth0:10.0.0.0/8
Open the policy file.
How the firewall will act on traffic. We later create exceptions in the rules file. 		sudo vi policy
Add as second to last line: $FW   $FW   ACCEPT
$FW   net   ACCEPT
$FW   ec2   ACCEPT
ec2   $FW   REJECT   info
net   $FW   DROP     info
ec2   all   DROP     info
net   all   DROP     info
all   all   REJECT   info
Open the routestopped file
This defines if and how the traffic should be allowed if for some reason the firewall is down. Since we have no physical means of getting to the machine I tend to think you have to let the world in... 		sudo vi routestopped
Add as second to last line: eth0   0.0.0.0   routeback
Open the rules file.
This defines the actual rules of the firewall. Since this is can in the future be used for many purposes, we add commented out lines to be toggled on later, but restricting ourselves as default. 		sudo vi rules
Add these as second to last line:
The template does seem to have a typo in it: so remember to comment out SECTION NEW 		#SECTION NEW

Ping(ACCEPT)     net   $FW
Ping(ACCEPT)     ec2   $FW
ACCEPT           $FW   ec2   icmp
Trcrt(ACCEPT)    net   $FW
Trcrt(ACCEPT)    ec2   $FW

SSH(ACCEPT)      net   $FW
SSH(ACCEPT)      ec2   $FW

#Web(ACCEPT)     net   $FW
#Web(ACCEPT)     ec2   $FW
#ACCEPT          ec2   $FW   tcp   8009

#SMTP(ACCEPT)    net   $FW
#SMTP(ACCEPT)    ec2   $FW
#SMTPS(ACCEPT)   net   $FW
#SMTPS(ACCEPT)   ec2   $FW
#Submission(ACCEPT)   net   $FW
#Submission(ACCEPT)   ec2   $FW
#IMAP(ACCEPT)    net   $FW
#IMAP(ACCEPT)    ec2   $FW
#IMAPS(ACCEPT)   net   $FW
#IMAPS(ACCEPT)   ec2   $FW
#POP3(ACCEPT)    net   $FW
#POP3(ACCEPT)    ec2   $FW
#POP3S(ACCEPT)   net   $FW

#VNC(ACCEPT)     net   $FW
#VNCL(ACCEPT)    net   $FW
#ACCEPT          net   $FW   tcp   5800:5809
Next lets test shorewall worked.
And if so lest start it.
And then enable for it automatically start 		sudo shorewall check;
sudo shorewall safe-start;
sudo vi /etc/default/shorewall
Toggle startup startup=1

Stage 2 done

You now have a very good Ubuntu server to create other servers from.
Follow the AMI making page to create an image of this.
It is important to launch and test the new images.

Swap memory

Micro instances only have 700MB of memory. If you have a micro instance you might want to add a Swap space to exend this amount.

You can check the current memory usage with: free -m
If on the Mem: line the free coloumn shows a number less than 100, you are running out of memory.
Please ensure there is no Swap: line, as that would indicate this instance already have swap space and you do not need another one.
Create & volume
Create a 2GB EBS disk via the AWS console.
  1. Ensure you are in the correct AWS "Region"
  2. go to "Elastic Block Store" "Volumes" page
  3. click "Create Volume"
  4. enter size as 2 GiB
  5. carefully choose correct "Availability zone"
  6. do not select a snapshot
  7. click "Yes, Create"
Attach the volume to your instance
  1. Once the status is "Available",
  2. attach it to your instance by selecting it
  3. and clicking "Attach Volume",
  4. select your instance,
  5. if desired modify the consoles suggested device name
    (eg.: /dev/sdf, /dev/sdi, /dev/sdg, /dev/sdk etc),
  6. click "Yes, Attach".
Format and mount swap space
Format the volume:
As of newer Ubuntu releases the devices are called /dev/xvdf.
Previously they were called /dev/sdf.
Replace sdf with relevant drive name. 		sudo mkswap /dev/sdf
Active as swap sudo swapon /dev/sdf
Check free memory and new swap space free -m
Persist swap space by editing fstab sudo vi /etc/fstab
Add this line /dev/sdf swap swap defaults 0 0

Please note that heavy use of the swap space might cost more money than simply using a Small instance. This is as IO usage across EBS are charged by AWS.

Extending

You can now extend this instance/image and make another image.

One very recommended step is to install a very usefull script for backing up and syncing with S3 called s3sync.
I have a detailed how to set up this on my backup page.

I would perhaps also prepare the use of EBS. But make sure your image works on its own without EBS. More on the EBS tips page.

Once you do have a ready Ubuntu server image (Congratulations) then make sure you boot it, and make sure you can log in etc. And at the make sure you shutdown and terminate all instances!

What next?

You can keep reading on my suggested use of these instances:


Update for Ubuntu 12.04+




back to flurdy's ec2 docs

flurdy