LaunchSchool - An Online School for Developers /

Blog

Chef Basics for Rails Developers

Chef is a cloud infrastructure framework. It is a tool that allows us to manage configurations, similar to Puppet and a few other tools, but Chef is my favorite because is written in Ruby. Chef can help you manage your infrastructure dependencies, create folder structure (with ‘knife’) and bootstrap our entire system or update configurations with just a few commands. We’ll walk through the process of setting up a Ruby on Rails server deployment with Chef with this tutorial.

We are going to build this tutorial on a brand new Ubuntu/trusty64 server. I have created the following repositories to go along with this tutorial.

Our target is: over a Ubuntu/trusty64 brand new server, we want to configure it and install our Rails Application.

For you to have example projects to go with all the instructions written here. I have created the following repositories:

Step 1. Vagrant: Creating your Development environment (Server Virtualization)

Vagrant allows us to run a Virtual Machine (VM) on localhost that we can use to test the configuration before working on the production server.

Prerequisites:

  • Install VirtualBox. Download the proper .deb package
  • To unzip the images you need to install ‘bsd-tar’: sudo apt-get -y install bsdtar

We will need a infrastructure project folder:

1
mkdir chef-basics && cd chef-basics

You should be using rbenv as your Ruby installation, so make 2.1.2 your local ruby version:

1
rbenv local 2.1.2

After that, you should create a Gemfile:

1
2
3
source 'https://rubygems.org'

gem 'vagrant', github: 'mitchellh/vagrant', tag: 'v1.6.3'

And just install the gems:

1
bundle install

Setting up Vagrant:

1
vagrant box add lebrijo/trusty64-chef

This downloads the box image from VagrantCloud to your machine.

1
vagrant init lebrijo/trusty64-chef

This creates the Vagrantfile of the project.

At this point we need to configure our the network interface in our Vagrantfile:

  • We should fix the public interface if we have more than one (i.e.: ethc, wlan0,…)
  • We need to have an appropriate fixed IP that will define our NODE with Chef
1
config.vm.network :public_network, bridge: "wlan0", ip: "192.168.10.25"

Other useful Vagrant commands are:

  • vagrant up creates and boots the VM.
  • vagrant provision runs Chef recipes on the VM.
  • vagrant ssh connects to the VM.
  • vagrant destroy -f destroys the VM.

Step 2. Knife-solo: Kitchen structure

For bigger infrastructures (if the number of your servers and site-cookbooks starts to grow) you can use chef-server.

But our use case is pretty simple – we just need a server with a Rails App. Here we should use chef-solo which allocates the chef-repo in the local sever (instead of the chef-server). For that purpose we use the knife-solo gem, which wraps chef-solo and offers some powerful commands to manage the configuration on target your Node:

  • knife solo init is used to create a new directory structure (the “kitchen”) that fits with Chef’s standard structure and can be used to build and store recipes.
  • knife solo prepare user@target-node installs Chef on a given node.
  • knife solo cook user@target-node uploads the current kitchen (Chef repo) to the target node and runs chef-solo on that node. You can use this command if you are updating a configuration in a node.
  • knife solo bootstrap user@target-node combines the two previous ones (prepare and cook). Useful for first server configurations.
  • knife solo clean removes the uploaded kitchen from the node.

So now, add the gem to your Gemfile: gem 'knife-solo' and install it: bundle install

And run knife solo init . to prepare the Kitchen folder structure.

Step 3. Librarian: Managing dependencies

The Chef community has prepared tons of cookbooks that we can use to build our system. Lots of them can be found at Supermarket.

Librarian-chef is a gem which allows you to control your cookbook dependencies and lock them (like Bundler with our ruby dependencies).

Add the gem to your Gemfile: gem 'librarian-chef' and install it: bundle install

librarian-chef init will create your Cheffile, where you should add your cookbooks:

1
2
3
4
5
6
7
8
9
10
#!/usr/bin/env ruby
#^syntax detection

site 'https://supermarket.getchef.com/api/v1'

cookbook 'application_ruby'
cookbook 'apt'
cookbook 'user'
cookbook 'ssh_known_hosts'
cookbook 'ruby_build'

librarian-chef install (like bundle install) will download all cookbooks to the local machine (to the ‘cookbooks’ folder), and creates a ‘Cheffile.lock’ in order to lock the books versions.

Step 4. Creating your own cookbook (rails-stack)

I will paste directly the recipe because it is pretty self-explanatory, and I will comment it below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
execute "apt-get update" do
  command "apt-get update"
end

# OS Dendencies
%w(git ruby-dev build-essential libsqlite3-dev libssl-dev).each do |pkg|
  package pkg
end

# Deployer user, sudoer and with known RSA keys
user_account 'deployer' do
  create_group true
end
group "sudo" do
  action :modify
  members "deployer"
  append true
end
cookbook_file "id_rsa" do
  source "id_rsa"
  path "/home/deployer/.ssh/id_rsa"
  group "deployer"
  owner "deployer"
  mode 0600
  action :create_if_missing
end
cookbook_file "id_rsa.pub" do
  source "id_rsa.pub"
  path "/home/deployer/.ssh/id_rsa.pub"
  group "deployer"
  owner "deployer"
  mode 0644
  action :create_if_missing
end

# Allow sudo command without password for sudoers
cookbook_file "sudo_without_password" do
  source "sudo_without_password"
  path "/etc/sudoers.d/sudo_without_password"
  group "root"
  owner "root"
  mode 0440
  action :create_if_missing
end

# Authorize yourself to connect to server
cookbook_file "authorized_keys" do
  source "authorized_keys"
  path "/home/deployer/.ssh/authorized_keys"
  group "deployer"
  owner "deployer"
  mode 0600
  action :create
end

# Add Github as known host
ssh_known_hosts_entry 'github.com'

# Install Ruby Version
include_recipe 'ruby_build'

ruby_build_ruby '2.1.2'

link "/usr/bin/ruby" do
  to "/usr/local/ruby/2.1.2/bin/ruby"
end

gem_package 'bundler' do
  options '--no-ri --no-rdoc'
end

# Install Rails Application
include_recipe "runit"
application 'capistrano-first-steps' do
  owner 'deployer'
  group 'deployer'
  path '/var/www/capistrano-first-steps'
  repository 'git@github.com:gotealeaf/capistrano-first-steps.git'
  rails do
    bundler true
    database do
      adapter "sqlite3"
      database "db/production.sqlite3"
    end
  end
  unicorn do
    worker_processes 2
  end
end

You have to copy these lines in ‘site-cookbooks/rails-stack/recipes/default.rb’:

  1. First it runs ‘apt-get update’ in order to update all Ubuntu packages and security patches every time we cook this recipe.
  2. Installs the software we will need for the next steps like git, sqlite,…
  3. Creates a ‘deployer’ user/group, in order to isolate the execution of our app. Making it sudoer.
  4. You will also need to provide known keys (‘site-cookbooks/rails-stack/files/default’ you will need to have id_rsa and id_rsa.pub), so that this will identify ‘deployer’ in your github.com account (remember how to create ssh keys)
  5. Allows all sudoers to run sudo commands without password prompt. This will make easier to work with Capistrano in deployment tasks.
  6. Authorizes you to make SSH connections from your workstation, by adding your public key to ‘files/default/authorized_keys’ file.
  7. Adds ‘github.com’ website key to your known_hosts file, in order to not be asked for authorization when ‘deployer’ clones your repos.
  8. Installs your Ruby version. RBENV is really useful in Developer environments; but having more than one ruby version installed in our server is a pain when you have: to automate tasks, to write Capistrano tasks or simply to write a daemon script. So here we will just compile 2.1.2.
  9. Installs bundler to manage our App gems.
  10. Downloads our example Rails App (/capistrano-first-steps) from our git server.
  11. “bundler true” makes a bundle install.
  12. Creates our database.yml file.
  13. And configures and starts up the Application Server (Unicorn).

As you can guess at this point, we have created your own cookbook called ‘rails-stack’, which has some dependencies that you should define at ‘metadata.rb’ file:

1
2
3
4
5
6
## Recipe dependencies
depends 'application'
depends 'application_ruby'
depends 'ruby_build'
depends 'user'
depends 'ssh_known_hosts'

This cookbook (rails-stack folder) could be published in a Chef-Server as is.

Step 5. Cook your Vagrant Box

Here we have to tell to Vagrant where are our cookbooks and recipe. So add to Vagrantfile:

1
2
3
4
  config.vm.provision "chef_solo" do |chef|
    chef.cookbooks_path = ["cookbooks", "site-cookbooks"]
    chef.add_recipe "rails-stack"
  end

Then, just start up your VM: vagrant up

You should wait a few minutes (12’ on my laptop), to execute the recipe. And finally, you could see the index page at your browser:

If you want to change the recipe or check other configurations, you do not need restarting the server (you will lose 12 minutes again), just re-run the recipe on it: vagrant provision

Step 6. Cook a real server

We will use DigitalOcean, but other providers have similar knife gems (EC2, Linode, Slicehost,…).

For this purpose we have a gem which acts as a plugin of knife called knife-digital_ocean. So that you need to include it in our Gemfile as gem 'knife-digital_ocean' and run bundle install.

You also need to identify yourself to manage your account:

  • Access to your DigitalOcean console and search for the API Keys:

  • Annotate client_id and create api_key:

These keys should be added at your ‘.chef/knife.rb’ file:

1
2
knife[:digital_ocean_client_id] = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
knife[:digital_ocean_api_key] = 'yyyyyyyyyyyyyyyyyyyyyyyy'

Now you can manage your Servers (aka Droplets) from the command line:

  • knife digital_ocean droplet list list all our droplets.
  • knife digital_ocean size list RAM sizes.
  • knife digital_ocean region list location list.
  • knife digital_ocean image list --global list built-in DO images.
  • knife digital_ocean sshkey list your ssh keys to manage your servers.

So that let’s create your Droplet:

1
2
3
4
5
6
7
8
jlebrijo:chef-basics$ knife digital_ocean droplet create --server-name chef-basics \
                                                         --size 66 \
                                                         --location 3 \
                                                         --image 5141286 \
                                                         --ssh-keys 183962
Droplet creation for chef-basics.gotealeaf.com started. Droplet-ID is 2535558
Waiting for IPv4-Address..done
IPv4 address is: 104.131.132.10

Once you have your IP address you can create the node at ‘nodes/104.131.132.10.json’, with the content:

1
2
3
4
5
{
  "run_list": [
    "recipe[rails-stack]"
  ]
}

And bootstrap your server: knife solo bootstrap root@104.131.132.10

This will take a few minutes (12 minutes in my case), but finally, in your browser, you have to see:

If you need to apply recipe changes at your server in the future, just run: knife solo cook root@104.131.132.10

Step 7. Deploy with Capistrano

After your Server provision and configuration, we have the next Chapter in Software life-cycle: DEPLOYMENT. When developers have a new feature implemented, they will need to deploy this. We discuss this in one of our earlier posts: Use Capistrano 3 to Deploy Rails Applications – a Step-by-Step Tutorial.

We can clone the example repo git clone git@github.com:gotealeaf/capistrano-first-steps.git, and create our capistrano environments (vagrant, staging, production).

You should change ‘config/deploy/production.rb’ file, in order to set your new server IP:

1
server '104.131.132.10', user: 'deployer', roles: %w{web app}

At this point you can develop/push your new features, and deploy them with: cap production deploy

Conclusions

Let’s see what we can do with Chef to manage deployment of our Rails project:

  • We can manage our server configuration like a development project: centralizing files, user creation, keys, using a git repo, etc.
  • We can test our server configuration locally, in a Vagrant VM, before configuring it on staging/production.
  • After your recipe is tested with ‘Vagrant’, you can install (bootstrap) or update (cook) your server configuration with ‘knife-solo’.
  • We can install our Rails App with Chef, and this is compatible with our Capistrano deployment process.