Rails, Cygwin, Git on Codebasehq, and EC2

Turns out it takes more than 5 minutes...

Last Updated: May 2009

by @esilverberg

Setup cygwin

  • Install the default. Also include make, gcc, ssh, ruby, git

    • Setup git

    With git installed, do

  • git init
  • git add .
  • git commit -m "Initial commit"

    • Make sure that your git is inside the top level rails directory. Codebasehq provides the string with instructions on how to push to their servers.

    Setup rails

  • Follow these instructions
  • You'll have to install gcc and make via cygwin
  • If you're on a dell, remember that Embassy Suite breaks cygwin, as do logitech webcams
  • You have to build mysql from source

    • Setup Codebasehq

    Pretty straightforward. The only trick is that you have to add public keys. You will have to add TWO public keys to Codebasehq.

  • Run ssh-keygen on your local cygwin box. This will generate id_rsa and id_rsa.pub.
  • Copy-and-paste the id_rsa.pub text into the public keys section of codebasehq. Click "My User profile", then "add public key"

    • EC2 setup

    Read the getting started guide here at amazon.

    Make sure you have downloaded the cert- and pk- .pem files. I have them in ~/.ec2/

    Setup environment variables

    JAVA_HOME='/cygdrive/C/Program Files/Java/jre6'
    EC2_CERT=/home/USER/.ec2/cert-EZC7LAIYSKPC546UKT7E3P6LQ4WUEOOI.pem
    EC2_HOME=/home/USER/ec2-api-tools-1.3-30349
    EC2_PRIVATE_KEY=/home/USER/.ec2/pk-EZC7LAIYSKPC546UKT7E3P6LQ4WUEOOI.pem
    PATH=$PATH:$EC2_HOME/bin

    Probably want to put these in your .bashrc with the 'export' command in front.

    Verify dev tools

    run this to make sure it works: ec2-describe-images ami-5394733a

    Setup an image

  • Setup a keypair: run ec2-add-keypair YOUR_KEY_PAIR
  • Save the private key in ~/.ec2/. I have it as id_rsa-testkeypair
  • Run this: ec2-run-instances ami-5394733a -k testkeypair Note that testkeypair is not a path, it's just the name
  • Per this page, you will have to use ami-5394733a.
  • After that, it will give you an instance number. If you type ec2-describe-instances i-INSTANCE It will tell you the URL to use to connect to your server

    • You can connect directly to your instance by typing:

    ssh -i ~/.ec2/id_rsa-testkeypair app@ec2-174-129-161-XXX.compute-1.amazonaws.com

    Elastic IP addresses

    If you have an amazon elastic IP address, then you can just associate it in the management console with your instance and ssh into that elastic ip

    ssh -i ~/.ec2/id_rsa-testkeypair app@YOUR_ELASTIC_IP

    Dealing with keys to your repo

    Your git repository requires a private key in order to connect and copy your source code. Capistrano therefore needs the source machine to have the keys already stored on the machine (at ~/.ssh/id_rsa)

    The automatic way

    I recommend generating a new key pair on your local machine, adding the public key to your git repo, and then copying the public and private key over to the aws server via a capistrano task

    ssh-keygen
    Enter file in which to save the key (/home/XXX/.ssh/id_rsa): /home/XXX/my_project/config/keys/id_rsa

    Put that key pair in #{RAILS_ROOT}/config/keys/

    Then, add this to your deploy.rb file

    namespace :repo do
       desc "Copy key so you can connect to codebasehq"
       task :keycopy do
          run "cd ~/"
          upload("#{RAILS_ROOT}/config/keys/id_rsa", "~/.ssh/", :via => :scp)
          upload("#{RAILS_ROOT}/config/keys/id_rsa.pub", "~/.ssh/", :via => :scp)
          run "chmod 600 ~/.ssh/id_rsa"
          run "chmod 600 ~/.ssh/id_rsa.pub"
       end
    end
    before "ec2onrails:setup", "repo:keycopy"

    This will make sure that your keys get copied over and capistrano can copy source from your repo.

    Or, deal with keys the old fashioned-way

  • Connect directly to your instance ssh -i ~/.ec2/id_rsa-testkeypair app@ec2-174-129-161-XXX.compute-1.amazonaws.com you have to give it the path to the keypair you just generated. You also have to login as app because that is the user that capistrano uses to run processes.
  • Generate an ssh key on the machine. Type ssh-keygen and then add the public key (id_rsa.pub) to codebasehq, just like you did before for your local machine.

    • Update tools on the image

    ssh into the machine and run the following

    sudo apt-get update
    sudo apt-get -y install build-essential
    sudo apt-get -y install imagemagick
    sudo apt-get -y install emacs22
    sudo gem update --system
    sudo gem install gemcutter
    sudo gem tumble

    You will need build-essentials to build the starling gem and emacs, and who knows what other ruby gems

    I tried to automate this with cap but it hangs at sudo apt-get -y install build-essential

    Setup your deploy.rb file

  • Install capistrano from the web per these instructions. This is what your deploy.rb file should look like. You can avoid dealing with s3.yml for now to get hello, world working. set :application, "your app"
    default_run_options[:pty] = true
    set :repository, "git@gitbase.com:you get this from codebasehq.git"
    set :scm, "git"
    ssh_options[:forward_agent] = true
    set :branch, "master"
    set :git_shallow_clone, 1

    ssh_options[:keys] = [File.join(ENV["HOME"], ".ec2", "id_rsa-testkeypair")]

    Make these be your roles:

    set :machine_name, "ec2-174-129-161-XXX.compute-1.amazonaws.com"
    role :app, "#{machine_name}"
    role :web, "#{machine_name}"
    role :db, "#{machine_name}", :primary => true

    Also set the mysql password.

    :mysql_root_password => "XXX"

    Change the packages line to this:

    :packages => ["logwatch", "librmagick-ruby"],

    Make sure you list your gems you need installed

    :rubygems => ["wvanbergen-http_status_exceptions", "twitter", "SystemTimer", "google-geocode", "geohash", "httparty", "sanitize", "base62", "htmlentities", "gchartrb"], Comment out # :server_config_files_root => "../server_config",

    Configure the database.yml

    You have to remember to make the host on production say db_primary production:
      adapter: mysql
      database: XXX_prod
      username: XXX
      password: XXX
      host: db_primary

    Configure with Cap

    Run

    cap ec2onrails:server:set_roles This ensures that mysql and the core services on the instance are configured.

    Continue with the standard EC2onrails install instructions

    Get public keys from server

    cap ec2onrails:get_public_key_from_server

    deploy with capistrano

    cap ec2onrails:setup
    cap deploy:cold

    Change the environment

    If you were deploying to a staging server but now would like to change that staging server into your production server, you will need to change the environment to production.

    First, associate the production ip address with your instance

    Next, delete your old server from known_hosts

    RAILS_ENV=production cap ec2onrails:server:set_rails_env
    RAILS_ENV=production cap deploy:restart

    You may have to change s3.yml to tell it which bucket to restore from

    Restoring blows away old tables in your db

    Also remember to re-generate workling:monit:make

    Cron services

    EC2onrails lets you do hourly and daily cron tasks, but not more frequent than that. I installed craken and have this in my deploy.rb to compensate: namespace :craken do
      desc "Install raketab"
      task :install, :roles => :cron do
        set :rails_env, "production" unless exists?(:rails_env)
        set :env_args, (exists?(:env_args) ? env_args : "app_name=#{application} deploy_path=#{current_path}")
        run "cd #{current_path} && rake #{env_args} RAILS_ENV=#{rails_env} craken:install"
      end
      task :uninstall, :roles => :cron do
        set :rails_env, "production" unless exists?(:rails_env)
        set :env_args, (exists?(:env_args) ? env_args : "app_name=#{application} deploy_path=#{current_path}")
        run "cd #{current_path} && rake #{env_args} RAILS_ENV=#{rails_env} craken:uninstall; true"
      end
    end

    before "deploy:symlink", "craken:uninstall"
    after "deploy:symlink", "craken:install"

    You can see more about the capistrano events here. The recipe was taken from here.

    Gotchas

  • Don't forget to open up port 443 on Amazon's management console if you need ssl support
  • Follow these instructions to configure your SSL certs
  • Setting up cron is described here
  • If you include submodules you can't do the git shallow copy

    • Useful links

  • EC2 on Rails
  • Getting it working with github (has some useful stuff about capistrano)
  • Random guy talking about what he did
  • The amazon ec2 console
  • Another guy deploys ec2 on rails
  • Starling and workling capistrano tasks

    • Advanced - Setting up a db server

  • Setup a cool mysql server backed by EBS here: http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=100&externalID=1663
  • This server only has a root user, but capistrano needs an admin user that is capable of sudoing...d'oh! You have to do the following:
    • adduser admin

    Now, add your public keypair via these instructions here: http://blogs.techrepublic.com.com/opensource/?p=86

    I think you have to manually copy the keypair; who knows where it is on the system??

    Now, add the admin user to the /etc/sudoers file so it is what capistrano is expecting.

    # The 'admin' user can run sudo without a password
    admin ALL=(ALL) NOPASSWD: ALL