Chef is a cloud infrastructure framework. It is a tool that allows us to manage configurations, similar to Puppet and a few other tools, but Chef is my favorite because is written in Ruby. Chef can help you manage your infrastructure dependencies, create folder structure (with ‘knife’) and bootstrap our entire system or update configurations with just a few commands. We’ll walk through the process of setting up a Ruby on Rails server deployment with Chef with this tutorial.
We are going to build this tutorial on a brand new Ubuntu/trusty64 server. I have created the following repositories to go along with this tutorial.
Our target is: over a Ubuntu/trusty64 brand new server, we want to configure it and install our Rails Application.
For you to have example projects to go with all the instructions written here. I have created the following repositories:
- chef-basics as our project of infrastructure with Chef/Vagrant. This is the product of all instructions written in this post.
- capistrano-first-steps as an example project, in order to demonstrate how to combine Chef with Capistrano. This repo is the product of following the instructions of Deploy Rails Applications with Capistrano 3 and this post.
Step 1. Vagrant: Creating your Development environment (Server Virtualization)
Vagrant allows us to run a Virtual Machine (VM) on localhost that we can use to test the configuration before working on the production server.
- Install VirtualBox. Download the proper .deb package
- To unzip the images you need to install ‘bsd-tar’:
sudo apt-get -y install bsdtar
We will need a infrastructure project folder:
You should be using rbenv as your Ruby installation, so make 2.1.2 your local ruby version:
After that, you should create a Gemfile:
1 2 3
And just install the gems:
Setting up Vagrant:
This downloads the box image from VagrantCloud to your machine.
This creates the Vagrantfile of the project.
At this point we need to configure our the network interface in our Vagrantfile:
- We should fix the public interface if we have more than one (i.e.: ethc, wlan0,…)
- We need to have an appropriate fixed IP that will define our NODE with Chef
Other useful Vagrant commands are:
vagrant upcreates and boots the VM.
vagrant provisionruns Chef recipes on the VM.
vagrant sshconnects to the VM.
vagrant destroy -fdestroys the VM.
Step 2. Knife-solo: Kitchen structure
For bigger infrastructures (if the number of your servers and site-cookbooks starts to grow) you can use chef-server.
But our use case is pretty simple – we just need a server with a Rails App. Here we should use chef-solo which allocates the chef-repo in the local sever (instead of the chef-server). For that purpose we use the knife-solo gem, which wraps chef-solo and offers some powerful commands to manage the configuration on target your Node:
knife solo initis used to create a new directory structure (the “kitchen”) that fits with Chef’s standard structure and can be used to build and store recipes.
knife solo prepare user@target-nodeinstalls Chef on a given node.
knife solo cook user@target-nodeuploads the current kitchen (Chef repo) to the target node and runs chef-solo on that node. You can use this command if you are updating a configuration in a node.
knife solo bootstrap user@target-nodecombines the two previous ones (prepare and cook). Useful for first server configurations.
knife solo cleanremoves the uploaded kitchen from the node.
So now, add the gem to your Gemfile:
gem 'knife-solo' and install it:
knife solo init . to prepare the Kitchen folder structure.
Step 3. Librarian: Managing dependencies
The Chef community has prepared tons of cookbooks that we can use to build our system. Lots of them can be found at Supermarket.
Librarian-chef is a gem which allows you to control your cookbook dependencies and lock them (like Bundler with our ruby dependencies).
Add the gem to your Gemfile:
gem 'librarian-chef' and install it:
librarian-chef init will create your Cheffile, where you should add your cookbooks:
1 2 3 4 5 6 7 8 9 10
librarian-chef install (like
bundle install) will download all cookbooks to the local machine (to the ‘cookbooks’ folder), and creates a ‘Cheffile.lock’ in order to lock the books versions.
Step 4. Creating your own cookbook (rails-stack)
I will paste directly the recipe because it is pretty self-explanatory, and I will comment it below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89
You have to copy these lines in ‘site-cookbooks/rails-stack/recipes/default.rb’:
- First it runs ‘apt-get update’ in order to update all Ubuntu packages and security patches every time we cook this recipe.
- Installs the software we will need for the next steps like git, sqlite,…
- Creates a ‘deployer’ user/group, in order to isolate the execution of our app. Making it sudoer.
- You will also need to provide known keys (‘site-cookbooks/rails-stack/files/default’ you will need to have id_rsa and id_rsa.pub), so that this will identify ‘deployer’ in your github.com account (remember how to create ssh keys)
- Allows all sudoers to run sudo commands without password prompt. This will make easier to work with Capistrano in deployment tasks.
- Authorizes you to make SSH connections from your workstation, by adding your public key to ‘files/default/authorized_keys’ file.
- Adds ‘github.com’ website key to your known_hosts file, in order to not be asked for authorization when ‘deployer’ clones your repos.
- Installs your Ruby version. RBENV is really useful in Developer environments; but having more than one ruby version installed in our server is a pain when you have: to automate tasks, to write Capistrano tasks or simply to write a daemon script. So here we will just compile 2.1.2.
- Installs bundler to manage our App gems.
- Downloads our example Rails App (/capistrano-first-steps) from our git server.
- “bundler true” makes a
- Creates our database.yml file.
- And configures and starts up the Application Server (Unicorn).
As you can guess at this point, we have created your own cookbook called ‘rails-stack’, which has some dependencies that you should define at ‘metadata.rb’ file:
1 2 3 4 5 6
This cookbook (rails-stack folder) could be published in a Chef-Server as is.
Step 5. Cook your Vagrant Box
Here we have to tell to Vagrant where are our cookbooks and recipe. So add to Vagrantfile:
1 2 3 4
Then, just start up your VM:
You should wait a few minutes (12’ on my laptop), to execute the recipe. And finally, you could see the index page at your browser:
If you want to change the recipe or check other configurations, you do not need restarting the server (you will lose 12 minutes again), just re-run the recipe on it:
Step 6. Cook a real server
We will use DigitalOcean, but other providers have similar knife gems (EC2, Linode, Slicehost,…).
For this purpose we have a gem which acts as a plugin of knife called knife-digital_ocean. So that you need to include it in our Gemfile as
gem 'knife-digital_ocean' and run
You also need to identify yourself to manage your account:
- Access to your DigitalOcean console and search for the API Keys:
- Annotate client_id and create api_key:
These keys should be added at your ‘.chef/knife.rb’ file:
Now you can manage your Servers (aka Droplets) from the command line:
knife digital_ocean droplet listlist all our droplets.
knife digital_ocean size listRAM sizes.
knife digital_ocean region listlocation list.
knife digital_ocean image list --globallist built-in DO images.
knife digital_ocean sshkey listyour ssh keys to manage your servers.
So that let’s create your Droplet:
1 2 3 4 5 6 7 8
Once you have your IP address you can create the node at ‘nodes/184.108.40.206.json’, with the content:
1 2 3 4 5
And bootstrap your server:
knife solo bootstrap email@example.com
This will take a few minutes (12 minutes in my case), but finally, in your browser, you have to see:
If you need to apply recipe changes at your server in the future, just run:
knife solo cook firstname.lastname@example.org
Step 7. Deploy with Capistrano
After your Server provision and configuration, we have the next Chapter in Software life-cycle: DEPLOYMENT. When developers have a new feature implemented, they will need to deploy this. We discuss this in one of our earlier posts: Use Capistrano 3 to Deploy Rails Applications – a Step-by-Step Tutorial.
We can clone the example repo
git clone email@example.com:gotealeaf/capistrano-first-steps.git, and create our capistrano environments (vagrant, staging, production).
You should change ‘config/deploy/production.rb’ file, in order to set your new server IP:
At this point you can develop/push your new features, and deploy them with:
cap production deploy
Let’s see what we can do with Chef to manage deployment of our Rails project:
- We can manage our server configuration like a development project: centralizing files, user creation, keys, using a git repo, etc.
- We can test our server configuration locally, in a Vagrant VM, before configuring it on staging/production.
- After your recipe is tested with ‘Vagrant’, you can install (bootstrap) or update (cook) your server configuration with ‘knife-solo’.
- We can install our Rails App with Chef, and this is compatible with our Capistrano deployment process.