If there’s one tool which streamlined our production workflow it must be git.

We started using it about a year ago – like everyone else. We joined Github for our own projects, and the open source ones, too.

But one post from Bryan Helmkamp helped us getting the server setup streamlined, particularly, the multistage Setup. So this is mixing gits’ and capistranos’ best aspects.

Why Multiple Stages? For many, bliss is when you can deploy with one command. This is great at the start of the project and the default for capistrano. The problem is that as soon as the project has a certain size, the speed at which one can deploy actually slows stuff down. This is because everyone has to wait with adding stuff until the deployment is done and agreed upon by the customer.

How it works Now with multiple stages, at one point we decide that a number of features are complete and can be tested by users. We update the staging branch and deploy that with capistrano to the stage server. There, the customer can check out the new features while we continue developing. When we reach an agreement we can deploy we update the production branch from the staging branch and deploy that.

So let’s jump in.

Installation

We are using the capistrano-ext gem and the gitworkflow.rb from Brian’s post in lib/tasks to standardize branching for deployment, and the actual deployment process. If you want to set up a new project to use those methods, you need to create

• config/deploy.rb
• config/deploy/*.rb
• lib/tasks/gitworkflow.rb
• Capfile

into your application. You can look here for some explanation – basically, you have to ‘capify’ the application and create deploy/envs.rb for each environment. If it’s aready capified, check the differences.The project should be in sync with a master branch (on github in my example).

Setup

This deployment setup requires two extra branches on github. This can be done with:

git checkout -b staging             # creates a branch locally
git push origin staging             # pushes the branch to github
git checkout -b production          # same for the production branche
git push origin production
git checkout master
git branch -d staging
git branch -d production

The local branches are deleted after each deploy.

For the following to work, the git config needs to be right. Much of this (maybe all of it) will be set correctly when cloning the project from github. Check:

git config --list

If the variables are not set right you can use

git config remote.origin.url git@github.com:simplificator/yourapp.git
git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
git config branch.master.remote origin
git config branch.master.merge master

Or alternatively, perform a new git clone git@github.com:simplificator/yourapp.gitin another directory and work from there.

Then, the deployment targets need to be set up.

cap staging deploy:setup
cap staging deploy
cap production deploy:setup
cap production deploy

It’s likely you need to go to the server to set up the databases. One way of doing this is going to the deployment location into the current checkout and then running

rake db:create:all

We need

• yourapp_production
• yourapp_staging

Here you need to do some badly documented magic:

• get gems
• setup apache for passenger
• run the migrations

Workflow

• Branch for working.
• Add and commit in your branch. Stash everything else.
• Unit test.
• Checkout master, pull, and merge your branch.
• Unit test.
• Push the master up.

Staging

When you’re happy with what you’ve done you can stage your work:

rake tag:staging
cap staging deploy:migrations

or the shortcut rake deploy:staging. When you do that, the staging branch is replaced with your local master branch.

Production deploy

When you want to push the staging branch into production, do

rake tag:production
cap production deploy:migrations

When you do that, the production branch is replaced with the staging branch from github.

Fixing

When you have to do a fix for the production server:

export BRANCH=hotfix
rake branch:production

hackhackhack…

git commit
export BRANCH=hotfix #(unless it's the same shell still)
rake tag:staging BRANCH=hotfix
cap staging deploy

After this, your staging server is running your corrected production code. If it works, you can redeploy this using

rake tag:production
cap production deploy

from the same shell (because of the BRANCH variable). Then the stage server is free again to take the more recent code.

New development doesn’t affect that process.

Conclusion

I’m sure a lot more of this could be automated (like the initial db setup) – I’m not claiming we got all edges out of the process. But it can give you some ideas on how to do it yourself.

For us, it has rendered our deployments much more professional, without adding too much cruft (after the initial setup of the first project). The stage server helps us to train ourselves with deployments of complicated software involving many moving parts until they run smoothly and will execute on production without trouble.

Until today we were happy users of Hpricot (0.8.1) . Easy to use, nice API, fast (enough)… but today we had to move a project over to Nokogiri.Why? Hpricot does not support XPath with namespaces. Worse: it does not even complain when you use a namespaced XPath query. It just keeps telling you that your XPath ended up in Nirvana and that there is no result.“Hey pal, you are using namespaces in XPath and i do not understand this” would have been a nice message.Since Hpricot does properly parse the namespaces, you can work around the problem using the at (aliased as %) method. But we need full XPath support hence the switch to Nokogiri.Some links i came across while researching the issue:

• http://enfranchisedmind.com/blog/posts/hpricot-does-namespaces
• http://www.rubyinside.com/nokogiri-ruby-html-parser-and-xml-parser-1288.html

Wan’t to know what language a text is written in? Maybe a comment on your Blog, an e-Mail you received, let your CMS guess the language of an article? Enter the babel gem!

Babel uses a n-gram approach to build reference profiles for languages. It also builds a profile for the input text and then calculates the distance between each reference profile and the input profile. The closest profile is probably the one with the language of the input text. Proabably. Chances are that Babel gives you wrong results on short sentences or single words.  You want to know more? Here is a paper that describes and compares different approaches.

We’ve setup a demo app so you can try out babel. And of course you can get the gem from github.

Thanks to the neat Inflector if ActiveSupport we know that the plural of tax is taxes. But if you ask for the singular form of taxes the inflector will try to convince you that it’s taxis!

>> "tax".pluralize.singularize
=>"taxis"

The reason we even found out about this was, that our tests (you do write tests, do you?) blew up when we used the taxes() fixture with a FixtureClassNotFound error. Of course: we do not have a Taxis class.

So how do you fix this? Simple: Edit config/inflections and add your own rule:

ActiveSupport::Inflector.inflections do |inflect|
  inflect.irregular 'tax', 'taxes'
end

>> "tax".pluralize.singularize
=> "tax"

Be sure to prefix Inflector with the ActiveSupport module. Otherwise you’ll face `load_missing_constant’:NameError: uninitialized constant Inflector

Currently we are working in a refactoring project. It’s a big huge project communicating with legacy systems with lots of code.On really good days (that is: few concurrent users with a tendency to 0 …) the application was working as planned (well, almost). On any other day it was throwing exceptions and consuming memory like crazy, producing random results and was slower than  its predecessor.The last days we’ve been investigating the master data management part of the application and decided that is time to start an overall renovation. After all the rest of the business depends on that data and it controls a lot of business processes. Here roughly what we did to put things straight:

1. Read the code to gather the requirements (no documentation on-hand)
2. Wrote automated tests to assure functionality
3. Refactored, fixed bugs
4. Done

So what’s the point you ask? You always write tests? That’s the way it should be? Well in this environment automated tests are rather scarce and we are had a hard time to convince people that they are a great thing. We told stories about how tests improves quality of code, how they encourage coders to change and improve code, how they will bring peace to the world. But with this part of the application now working flawlessly and faster with less LOC than before, we see some more acceptance for “losing time writing tests”. A good day.

We’re using ssh with public keys quite often. One thing I always missed on OS X was the ssh-copy-id script. Its job is to take the public key of the user running the script and helping to copy that over to a server, so that the user doesn’t have to enter the password all the time. It’s a great time saver and allows you to secure the accounts with harder passwords (instead of using the same password for all accounts, like you do).

Github has a nice guide how you can create a public key, in case you don’t have one. Now, grab a copy of the ssh-copy-id script from the openssh package (for example from here). Copy it into /usr/local/bin and chmod it:

sudo cp ssh-copy-id.sh /usr/local/bin/ssh-copy-id
sudo chmod a+x /usr/local/bin/ssh-copy-id

Then you can use the following command (you’ll have to enter the password one last time)

ssh-copy-id user@machine.com

The thing tells you to check which keys you copied, so have a look at .ssh/authorized_keys after the act. Do that, it’s wise. And change your silly password into something more complicated.

By the way – we have a piece of software blocking all IPs which try to log in using passwords more than five times in a row. And it happens dozens of time a day.

We’ve written up a tiny ruby gem for the compete api. It can be used to query the compete service for ranking and trust information about a domain. We’ll eventually integrate it into railsworld.com so we have another source for ranking information (right now it’s alexa).

 Find more at the github repo for the compete gem

This is something I just have to blog about. I get to speak at the Namics Camp 09 this year in Davos!

Waay back in October 2007 I had a presentation on unit testing at the Rails Höck in Zurich. The core value I wanted to bring over was the fact that testing is not a chore, doesn’t make you slow (quite the opposite) but renders coding a fun chase for the green bar (those of you who do regular testing know what I’m talking about here).

It looks like I had some success with igniting that spark. One of the attendees has applied test first programming in a small circle of developers at his company, Namics AG in Zürich. They were really successful with it and he wanted to accelerate the uptake of regular testing. So he asked me to sort of repeat that speech at his company, Namics in Zurich at the internal tech talk, which I did.

Now I got invited to their annual meetup to talk about testing again, in front of a (hopefully!) bigger crowd. I look forward to a few days of geeky talk and deep discussions. And hopefully infect a few more coders with the testing habit. And I look forward to learn a lot.

Remove all files from the repo which are deleted: 
git ls-files -z --deleted | git update-index -z --remove --stdin

I just pushed a new version of the scriptaculous multi drag and drop addon. It’s much cleaner, supports mere Draggables and can react on shift clicks too. Check it out at http://simplificator.com/scriptaculous-multidrag/ (source is available on Github).