Friday, May 30, 2008

Another point of defensive programming

Another point of defensive programming that I need to work on is not drilling too deep into an object. "object.relation.group_of_attributes.value" is bad code. Any time something changes inside, you have to change code in multiple other locations.

I've often used the excuse, "But requiring accessors reduces my flexibility." But now I realize that that kind of flexibility leads to very fragile code, makes testing impossible because you have hundreds of possible failure modes and prevents you from installing any central code to deal with those failure modes.

Shame on me.

Don't assume your data is valid

I've come to think that you should never code with the assumption that your data is valid. Now that doesn't mean that you're going to get the desired result if your data is invalid, maybe you just don't have what you need. But your code should try and do as much as it can with as little as possible.

For example, if you have a rails controller that returns a collection, don't let an exception accessing one of those objects prevent the entire list from returning. If you have a shopping cart and heaven forbid, you code finds an object that doesn't pass validations but is nonetheless persisted in the database, don't return an error page, log the issue and return a partial list and telling the user that an error occurred and some items had to be removed from their basket.

This has applications in large scale applications too, where maybe you don't have a strict database schema or you can't run migrations on your entire dataset everytime you make a code change. Your code is going to receive data that is out of date, incomplete, incorrectly formatted, or just plain broken. Take any data your code receives, don't make any assumptions and do the best you can. Your program will be more flexible, more resilient, and it will be more likely when an major error occurs that the user will be able to continue in some way.

Wednesday, May 28, 2008

Updates to my ext customizations

I've moved the repository from SVN to git and I'm hosting a public repo of it at github: http://github.com/curzonj/ext-extensions/tree/master. There are a lot of new things in there that I find really useful. The biggest components we use are the CRUD components. We use them to build data-entry style applications for companies. crud_components.js and crud_editor.js are the old ones (I still have lots of code that uses them) and the mvc_* files are the new CRUD stuff. I'll try and post some examples of them both and some information about how I build tests for my UI code. If you find any bugs, keep in mind, this is written against Ext 2.0.2.

Wednesday, May 21, 2008

Troubles deploying with capistrano and git

We have a project that we need to deploy from via authenticated https and it gave me some real trouble getting it to deploy via capistrano.

First, after a lot of digging, git (at least the version installed) doesn't always respect "http.sslVerify=false" in your ~/.gitconfig and we don't have a valid certificate on the repository. I also found that capistrano doesn't use an interactive ssh shell to do its work so putting "export GIT_SSL_NO_VERIFY=true" in your .bashrc file only makes it work for you, not capistrano. For capistrano I had to add "default_environment['GIT_SSL_NO_VERIFY'] = 'true'" to my deploy.rb file.

From there I just had to create the ~/.netrc file with my credentials for curl, and for good measure I setup ~/.gitconfig with "http.sslVerify=false".

Thursday, May 08, 2008

Model factories for Rails

In looking for alternatives to fixtures in Rails I found the Object Mother/Factory pattern. The only thing I don't like about it is that all your examples are in a seperate file somewhere. I wrote a monkey patch for ActiveRecord that lets you put basic validating examples in your model code that you can call from your tests. I like to use random data in my factories, but it's not required. It's really nice to not have to build the object trees by hand for each test. You can pass in any attributes you want to override the defaults.

(Readable code)

## category.rb, using random data to find unknown edge cases
minimum_example do
set_name = Faker::Lorem.word
sometimes { set_name = 'Materials' }

{
:name => Faker::Lorem.unique,
:set_name => set_name
}
end

## product.rb model
minimum_example do |attrs|
# override attributes from the caller
cat = attrs[:category] || example(:category)

{
# The category set_name is random, we might get a material
:name => cat.is_material? ? nil : Faker::Lorem.unique,
:category => cat,
:material_type => cat.is_material? ? example(:material_type) : nil
}
end
optional_example do |obj|
# you can modify the object directly
obj.color = 'blue'

# or just return more attributes
{
:size => 23
}
end
# anything_example will be alternate types of examples
material_example do
{
:inv_category => example(:inv_category, :set_name => 'Materials'),
:material_type => example(:material_type)
}
end
bob_example { { :color => 'blue' }} #any name works

Product.material_example #use autogenerated examples
Product.bob_example
Product.example
Product.example(:category => nil)
Product.example(:category => nil, :save => false) # intantiate the object w/o saving

# use optional_example, for non-required attributes
Product.example(:category => nil, :full => true)

# a more complete example excerpt
should 'list unique purchase_quotes' do
quote = Quote.example(:full => true)
pq1 = PurchaseQuote.example
pq2 = PurchaseQuote.example

pq2.purchase_quote_items << quotable =""> quote.quotables[0], :vendor => pq2.vendor)
quote.quotables.each do |q|
pq1.purchase_quote_items << quotable =""> q, :vendor => pq1.vendor)
end

list = quote.purchase_quotes
list2 = list.uniq
assert_equal list2.size, list.size
end


In writing some of these examples, I have no recollection whan attributes are required or how to build the complete object tree to create a valid product, but the code does it for me. That reduces the deterrent to write tests and therefor we have more tests.

There are many uses once you have easy access to a generator for random data, you can fill up your database with varied test data and run things against it.

Monday, May 05, 2008

Tracking rails code deployed with git

With git and multiple remote repositories I decided that it would be really nice if I had a branch ref in my local repository that points at the currently deployed version of my code. You could do capistrano tasks and update a branch in the central repository when you deploy code, but I found an easier way that works for my small project. I only have one app server, and I use the remote_cache strategy so this option is easy.
  1. I put a reference to .../myproj/current in my local .git/config file:
    1. [remote "servername"]
      url = ssh://user@appserver/project/path/current
      fetch = +refs/heads/deploy:refs/remotes/deployed/servername
  2. Run "git fetch", and I have a remote ref that points to the code currently on the app server regardless of the success of any other code that may or may not have run.
I really love using git-rev-list and gitk with my deployment remote refs to double check before I deploy.

Deploying Rails from git

Git being the new kind on the block, information about capistrano with git isn't as plentifully as cap with subversion. Here is how I setup my project to deploy from git.

Garry Dolley wrote a git module for Capistrano, but it is included in version 2.1 so you don't need to do anything special to use it. In your gem directory it's at capistrano-2.2.0/lib/capistrano/recipes/deploy/scm/git.rb, and I'd suggest reading the doc header because it has some nice ways to play with variables that take advantage of the distributed nature of git.
  1. Set the scm type and replace your repository variable with the URI of your git repository:
    1. set :scm, :git
    2. set :repository, "username@server.domain.name:/path/to/repo/myproj.git"
    3. set :deploy_via, :remote_cache
  2. If you are using git 1.5.3 or higher on all your machines, you can use submodules by
    1. set :git_enable_submodules, true
  3. Set the branch to deploy from:
    1. set :branch, "master"
The "set :deploy, :remote_cache" also works with subversion (would have been nice to know), but is really important with git, otherwise you clone the whole repository on each deploy which takes a long time. "remote_cache" makes incremental deploys much faster. It clones the repository in shared/cached_copy and then just updates it and copies it into place on each future deploy. It also has a very nice advantage over "set :git_shallow_clone, 1" which I'll talk about later.

If you are using multi stage deployment you can put the "set :branch, 'master'" in the stage files and could even point each stage at a different branch that you merge to when the code is ready.

You can use the normal scm_user/scm_password options and capistrano will handle the authentication via ssh. Another option is setting up password-less public keys on the target servers with a restricted shell on the git server. Git will give you trouble if your HTTP repository is authenticated (needs ~/.netrc) or has an SSL certificate that doesn't validate (doesn't work if the CN differs from the dns name) so I didn't go that route.

Remove any :checkout, or other :deploy_via variables you have in your deploy.rb file and you're set.

Migrating from subversion to git

Last week I moved my project at work from subversion to git. I know you can use git-svn, but I needed some branching and history rewriting that I couldn't get to work with the way git-svn rewrites your commits. It may have been a solution looking for a problem, but oh well, because since I have switched, I've been able to do some amazing things in cleaning up my history and code branches.

If you haven't used git-svn here are the two posts that helped me most:
If you want a more full-fledged git hosting setup, you can use gitosis: http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way

I already had my subversion tree checked out via git, but it was up-to-date and because each git-svn checkout creates it's own set of revisions to represent the repository, I just started from scratch. Also, HTTP doesn't seem useful as a central repository for an in-house group like mine because you're going to be pushing back to the repository, so didn't even bother setting it up. I know you can setup push over HTTP, but I heard working over ssh is faster. By the way, make sure you're using 1.5 or higher, anything less just doesn't cut it.
  1. I created a directory called /var/git and did a git-svn checkout of the subversion repository from the same server in there:
    1. mkdir myproj
    2. cd myproj
    3. git svn init -s file:///path/to/repo
  2. Then I created the public git repository: (from the Git user manual)
    1. cd ../
    2. git clone --bare myproj myproj.git
    3. cd myproj.git
    4. git --bare update-server-info
    5. chmod a+x hooks/post-update
Now you can clone your repository with ssh://username@server.name.com/path/to/repo/myproj.git, the ssh:// prefix isn't strictly neccesary, but it I like it.

You'll need to make sure you have public key ssh authentication setup for everyone using git. We already did so I didn't have to go through that. Also, if you don't trust your developers on the server, you can use set their shell to 'git-shell' which is a restricted shell that only allows git commands.

Git with Putty on Windows

It took me awhile to find this, so here it is for others. If you want git to use your private ssh keys through pageant that comes with PuTTY here's how to do it from http://www.mail-archive.com/msysgit@googlegroups.com/msg00269.html

  1. Set an environment variable named GIT_SSH to the full path of plink.exe (ie. C:\Program Files\PuTTY\plink.exe)
  2. Include your login name in git urls. Otherwise plink.exe will report strange errors.
After that your git will use pageant just like PuTTY does. The mailing list link has more details about setting up PuTTY if you need them.