While it is easy to include plugins in your Rails projects, it isn't easy to extend and customize the plugin for your own application. Desert solves that limitation/complication by making it just as easy to extend or modify a plugin class as it would be with any other class. In this post we will go over how Desert provides an easy way to manage and extend your plugins.

At Pivotal, we offer an integrated platform of Rails plugins named Socialitis.

Socialitis is an internal project that grew out of the observation that many of our start-up clients needed to build the same non-differentiating features; user management, friends/contacts, activity feeds, on-site messaging, etc.

The Socialitis platform is broken up into number of plugins that extend the Rails app in specific ways. These plugins may have dependencies on other plugins.

One of the major design goals of Socialitis is easy, drop-in, integration into existing Rails apps. This means using convention over configuration and removing as much integration responsibility from the user of the plugin as possible.

Another design goal was to provide sensible defaults and make each plugin easy to customize for your app.

We used Desert to achieve these goals, and so can you for your own platform.

The major features that Desert provides are:

• Defining a Rail's like directory structure into your plugin (models, views, controllers, helpers)
• Plugin dependencies
• Seamless overriding of classes and modules defined by parent plugins
• Plugin migrations
• Plugin routing

Desert provides a similar feature set to the Radient plugin system and the now defunct Appable Plugins framework.

For a simple example, lets say you have two plugins, name User and Messaging. The User plugin provides basic authentication and login features, and the Messaging plugin allows Users to send Messages to each other. The Message plugin depends on the User plugin.

The directory structure of the full Rails app looks like:

  |-- app
  |   |-- controllers
  |   |   |-- application.rb
  |   |   `-- blogs_controller.rb
  |   |-- helpers
  |   |   |-- application_helper.rb
  |   |   `-- blogs_helper.rb
  |   |-- models
  |   |   `-- user.rb
  |   `-- views
  |       |-- blogs
  |       |-- layouts
  |       |   `-- users.html.erb
  |       `-- users
  |           |-- index.html.erb
  |           `-- show.html.erb
  |-- db
  |   `-- migrate
  |       `-- 001_migrate_users_to_001.rb
  |-- lib
  |   `-- current_user.rb
  |-- spec
  |   |-- controllers
  |   |   `-- blogs_controller_spec.rb
  |   |-- fixtures
  |   |-- models
  |   |-- spec_helper.rb
  |   `-- views
  |       `-- blogs
  `-- vendor
      `-- plugins
          `-- user
              |-- app
              |   |-- controllers
              |   |   |-- logins_controller.rb
              |   |   `-- users_controller.rb
              |   |-- helpers
              |   |   |-- logins_helper.rb
              |   |   `-- users_helper.rb
              |   |-- models
              |   |   |-- login.rb
              |   |   `-- user.rb
              |   `-- views
              |       |-- logins
              |       |   |-- edit.html.erb
              |       |   |-- index.html.erb
              |       |   |-- new.html.erb
              |       |   `-- show.html.erb
              |       `-- users
              |           |-- edit.html.erb
              |           |-- index.html.erb
              |           |-- new.html.erb
              |           `-- show.html.erb
              |-- config
              |   `-- routes.rb
              |-- db
              |   `-- migrate
              |       `-- 001_create_users.rb
              |-- init.rb
              |-- lib
              |   `-- current_user.rb
              |-- spec
              |   |-- controllers
              |   |   `-- user_controller_spec.rb
              |   |-- fixtures
              |   |   `-- users.yml
              |   |-- models
              |   |   `-- user.rb
              |   |-- spec_helper.rb
              |   `-- views
              |       `-- users
              `-- tasks
          `-- message
              |-- app
              |   |-- controllers
              |   |   `-- message_controller.rb
              |   |-- helpers
              |   |   |-- message_helper.rb
              |   |   `-- user_helper.rb
              |   |-- models
              |   |   |-- message.rb
              |   |   `-- user.rb
              |   `-- views
              |       `-- messages
              |           |-- edit.html.erb
              |           |-- index.html.erb
              |           |-- new.html.erb
              |           `-- show.html.erb
              |-- config
              |   `-- routes.rb
              |-- db
              |   `-- migrate
              |       `-- 001_create_messages.rb
              |-- init.rb
              |-- spec
              |   |-- controllers
              |   |   |-- message_controller_spec.rb
              |   |   `-- user_controller_spec.rb
              |   |-- fixtures
              |   |   |-- messages.yml
              |   |   `-- users.yml
              |   |-- models
              |   |   |-- message_spec.rb
              |   |   `-- user_spec.rb
              |   |-- spec_helper.rb
              |   `-- views
              |       `-- messages
              `-- tasks

The User plugin introduces the various User and Login Rails objects. The Message plugin introduces its respective Message objects. Notice that the Message plugin also reopens some of the User objects to insert functionality.

For example, vendor/plugins/users/app/models/user.rb looks something like:

class User < ActiveRecord::Base
  has_many :logins
end

The Message plugin would then reopen User in vendor/plugins/message/app/models/user.rb:

 class User < ActiveRecord::Base
   has_many :messages_received
   has_many :messages_sent
 end

Meanwhile, the main application can also reopen User in app/models/user.rb

 class User < ActiveRecord::Base
   def custom_app_method
     # custom app logic #
   end
 end

Desert allows you to utilize Ruby's ability to repoen classes to layer on functionality in your plugins and application. At Pivotal, we have had success in sharing code across multiple client applications using this technique.

Another thing to note is normally the Message plugin would be loaded before the User plugin. Desert allows you to create plugin dependencies. So in vendor/plugins/message/init.rb:

require_plugin 'user'
require_plugin 'will_paginate'

This means you no longer need to define plugin load order inside of environment.rb. Your plugins can take care of that. Desert works with practically all plugins. That means you can have a plugin dependency on any existing Rails plugin.

To see more examples & documentation, take a look at the Desert project at http://github.com/pivotal/desert.

Over the next week or so I'll be sharing Ruby idioms and flourishes that I quite like. Today I'd I'll show a few tiny uses of splat! that make me tremble with delight.

Splat! - For Beginners

Splat! is the star (*) operator, typically used in Ruby for defining methods that take an unlimited number of arguments:

def sprintf(string, *args)
end

It can also be used to convert an array to the multiple-argument form when invoking a function:

some_ints = [1,2,3]
sprintf("%i %i %i", *some_ints)

Splat! - For Wizards

Array to Hash Conversion

The best use of splat! for invoking a infinite-arity functions I've ever seen is the recipe for converting an array to a hash. Suppose you have an array of pairs:

array = [[key_1, value_1], [key_2, value_2], ... [key_n, value_n]]

You would like to produce from it the hash: {key1 => value1 ... } You could inject down the array, everybody loves inject, but there is a better way:

Hash[*array.flatten]

Amazing right? This relies on the the fact that the Hash class implements the [] (brackets) operator and behaves thusly:

Hash[key1, value1, ...] = { key1 => value1, ... }

Heads or tails?

Splat! can be used for more than just method definition and invocation. My personal favorite use is destructuring assignment. I read this in Active Record's source code recently:

  def sanitize_sql_array(ary)
    statement, *values = ary
    ...
  end

This is invoked when you do something like User.find(:all, :conditions => ['first_name = ? and last_name = ?', 'nick', 'kallen']). Splat! is used here is to get the head and tail of the conditions array. Of course, you could use always use shift, but the functional style used here is quite beautiful. Consider another example:

first, second, *rest = ary

One final trivium (#to_splat aka #to_ary)

You can actually customize the behavior of the splat operator. In Ruby 1.8, implement #to_ary and in 1.9 it's #to_splat. For example

class Foo
  def to_ary
    [1,2,3]
  end
end

a, *b = Foo.new
a # => 1
b # => [2,3]

This also works for method invocation:

some_method(*Foo.new) == some_method(1,2,3)

When I first learned this at RubyConf I thought this was mind-blowing. I have since never used it.

I just upgraded to Leopard on my Mac. Previously, on Tiger, I had installed Ruby from source, in the default /usr/local/lib prefix. After reading the discussion on the Apple-provided Ruby installation, I decided to try it - mainly to ensure that my apps, such as GemInstaller, play well with it (on Pivotal's Mac pair workstations, we still install Ruby from source, so everything matches our demo/production environments as closely as possible, and things are in consistent locations).

So, I wanted to uninstall the old Ruby source installation, and only have the Apple-provided Ruby on disk. Googling for a few minutes did not provide exact instructions for this, so I'm writing up what I did, in hopes that it will help you!

I didn't use the "--prefix" option when I originally installed Ruby from source, so it was in the default location of /usr/local/lib/ruby, with binaries in /usr/local/bin.

WARNING: Use 'rm -rf' at your own risk - a sleep-deprived encounter with 'rm -rf' and a stray file named '~' is what "motivated" my Leopard upgrade in the first place...

First, I deleted the old ruby libraries/gems, which was easy enough, because they all lived under /usr/local/bin/ruby:

sudo rm -rf /usr/local/lib/ruby

However, this left all the old ruby/gems executables in /usr/local/bin. This resulted in errors when trying to run executable gems that I had not yet installed under the Apple Ruby installation:

$ cheat
/usr/local/bin/cheat:9:in `require': no such file to load -- rubygems (LoadError)
from /usr/local/bin/cheat:9

Instead of a cryptic rubygems error, I should get a 'file not found error':

$ sudo rm /usr/local/bin/cheat
$ cheat
-bash: /usr/local/bin/cheat: No such file or directory

So, I want to purge everything ruby-releated from my /usr/local/bin folder. I whipped up a quick ruby one-liner which just prints out (almost) all ruby-related files in /usr/local/bin:

ruby -e "old_ruby_execs = \`egrep 'rubygems|bin/ruby|env ruby' /usr/local/bin/*\`; require 'pp'; pp old_ruby_execs.split(\"\n\").collect{|line| line.split(':').first}.uniq"

Yeah, I know, ugly and obtuse, but one-liners are kind of fun, and help me remember that Ruby is great tool for sysadmin scripts. Feel free to put it in a class and test it if you are so inclined.

Even though I tried to make a fairly specific regexp for egrep, when inspecting that list, I did find a 'jgem' file, which was part of JRuby. I'm planning on reinstalling JRuby anyway, so I didn't care if that got deleted along with the other ruby stuff.

Anyway, if the output of that looks like everything you want to delete, then run this one-liner to do the actual deed (the 'sudo echo' is to 'prime' the sudo auth, so you don't get a noninteractive password prompt):

sudo echo; ruby -e "old_ruby_execs = \`egrep 'rubygems|bin/ruby|env ruby' /usr/local/bin/*\`; old_ruby_execs.split(\"\n\").collect{|line| line.split(':').first}.uniq.each { |exec| p 'removing ' + exec; \`sudo rm #{exec}\`}"

After that, the only thing that I saw left was the 'ruby' executable itself, which I whacked as well:

$ sudo rm /usr/local/bin/ruby

That seems to be about it, as least good enough to get all the old invalid executables off my path. I'm sure this could have been done cleaner if I had taken more care with the original source install. However, a good brute-force approach never hurt anyone. Much. Feel free to post links to relevant and helpful stuff.

Many people (including me) have complained about the lack of a good GUI debugger for Ruby. Now that some are finally getting usable, I've found I actually prefer IRB-style ruby-debug to a GUI.

There's good tutorial links on the ruby-debug homepage, and a very good Cheat sheet, but I wanted to give a bare-bones HOWTO to help you get immediately productive with ruby-debug.

Install the latest gem

$ gem install ruby-debug

Install the cheatsheet

$ gem install cheat
$ cheat rdebug

Set autolist, autoeval, and autoreload as defaults

$ vi ~/.rdebugrc
set autolist
set autoeval
set autoreload

Run Rails (or other app) via rdebug

$ rdebug script/server

Breakpoint from rdebug

(rdb:1) b app/controllers/my_controller.rb:10

Breakpoint in source

require 'ruby-debug'
debugger
my_buggy_method('foo')

Catchpoint

(rdb:1) cat RuntimeError

Continue to breakpoint

(rdb:1) c

Next Line (Step Over)

(rdb:1) n

Step Into

(rdb:1) s

Continue

(rdb:1) c

Where (Display Frame / Call Stack)

(rdb:1) where

List current line

(rdb:1) l=

Evaluate any var or expression

(rdb:1) myvar.class

Modify a var

(rdb:1) @myvar = 'foo'

Help

(rdb:1) h

There are many other commands, but these are the basics you need to poke around. Check the Cheat sheet for details.

This can also be used directly from any IDE that supports input into a running console (such as Intellij Idea).

That should get you started. So, before you stick in another 'p' to debug, try out ruby-debug instead!

We all love to Monkey Patch Rails and other Ruby apps. However, we sometimes want to target these patches to the specific versions where they are needed.

Here's the easiest way to do this, via RubyGem's built-in version requirement support. The version 0.11.0 should indeed be greater than version 0.9.0:

irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> Gem::Version::Requirement.new(['> 0.9.0']).satisfied_by?(Gem::Version.new('0.11.0'))
=> true

Notice that you can't do this with string comparison, because with a per-character comparison,1 is not greater than 9:

irb(main):001:0> '0.11.0' > '0.9.0'
=> false

Here's a little class which puts some helper and example methods around this approach (these methods are all in real use for some of our multipart mailer hacks):

module Pivotal
  class VersionChecker
    def self.current_rails_version_matches?(version_requirement)
      version_matches?(Rails::VERSION::STRING, version_requirement)
    end

    def self.version_matches?(version, version_requirement)
      Gem::Version::Requirement.new([version_requirement]).satisfied_by?(Gem::Version.new(version))
    end

    def self.rails_version_is_below_2?
      result = Pivotal::VersionChecker.current_rails_version_matches?('<1.99.0')
      result
    end

    def self.rails_version_is_below_rc2?
      Pivotal::VersionChecker.current_rails_version_matches?('<1.99.1')
    end

    def self.rails_version_is_1991?
      Pivotal::VersionChecker.current_rails_version_matches?('=1.99.1')
    end
  end
end

(note: some angle brackets changed due to code formatting bug)

Here's an example of how you'd use this:

if Pivotal::VersionChecker.rails_version_is_below_2?
  # do some backward compatibility stuff
  # or handle bugs that have been fixed in Rails > 2
end

Note that this is only possible now that Rails has started using a more sensible strategy for versioning edge gems and improved support for using advanced versioning with RAILS_GEM_VERSION.

For many projects, this may be overkill. It is useful at Pivotal, though, where many various projects may be on different rails versions, but still want to use the latest common core libraries (and monkey patches) without having to upgrade Rails for their app.

This isn't only useful for monkey patching. It can be handy for any library that wants to be backward- or forward-compatible with its dependencies. I've used this approach at Pivotal and on my personal projects to have Continuous Integration automatically run my tests against multiple dependency versions, without having to change anything other than the CI project name:

There are numerous other related topics for discussion in this area, such as the power of versions or the wisdom of freezing, but I'll save those for future posts. Even if you do freeze the trunk of Rails/plugins/gems, since the version is included in the source, this approach should work barring any conflicts with trunk changes since the last release.

Happy Versioning!

Sometimes I call a named route incorrectly: edit_user_project_path(project). And I get an illegible error message:

user_project_url failed to generate from {:action=>"show", :user_id=>#<Project id: 1, name: "Andy falls off a cliff", created_at: "2007-12-03 15:15:08", creator_id: 2, completed_at: nil, description: "", deleted_at: nil>, :controller=>"projects"}, expected: {:action=>"show", :controller=>"projects"}, diff: {:user_id=>#<Project id: 1, name: "Andy falls off a cliff", created_at: "2007-12-03 15:15:08", creator_id: 2, completed_at: nil, description: "", deleted_at: nil>}

I can't read that. This, however, is much clearer:

user_project_url failed to generate from {:action=>"show", :user_id=>"1", :controller=>"projects"}, expected: {:action=>"show", :controller=>"projects"}, diff: {:user_id=>"1"}

The error message really ought to call #to_param on the path parts, don't you think?

class ActionController::Routing::RouteSet
  # try to give a helpful error message when named route generation fails
  def raise_named_route_error(options, named_route, named_route_name)
    helpful_options = options.inject({}) {|hash, (key, value)| hash.merge(key => value.to_param) }
    diff = named_route.requirements.diff(options)
    unless diff.empty?
      raise RoutingError, "#{named_route_name}_url failed to generate  from #{helpful_options.inspect}, expected:  #{named_route.requirements.inspect}, diff:  #{named_route.requirements.diff(helpful_options).inspect}"
    else
      required_segments = named_route.segments.select {|seg| (!seg.optional?) && (!seg.is_a?(DividerSegment)) }
      required_keys_or_values = required_segments.map { |seg| seg.key rescue seg.value } # we want either the key or the value from the segment
      raise RoutingError, "#{named_route_name}_url failed to generate from #{helpful_options.inspect} - you may have ambiguous routes, or you may need to supply additional parameters for this route.  content_url has the following required parameters: #{required_keys_or_values.inspect} - are they all satisfied?"
    end
  end
end

Make it so!

I dislike the vague error message produced by validates_associated.

class User
  validates_associated :profile
  delegate ..., :to => :profile
end

I see the following error message: profile is invalid. But WHY was the profile invalid? The validation errors from the profile should bubble up to the user. So,

module ActiveRecord::Validations::ClassMethods
  def validates_associated(association, options = {})
    class_eval do
      validates_each(association) do |record, associate_name, value|
        associate = record.send(associate_name)
        if associate && !associate.valid?
          associate.errors.each do |key, value|
            record.errors.add(key, value)
          end
        end
      end
    end
  end
end

Now we see:

Music tastes can't be blank

Eh, voila!

We just implemented bookmark_fu on Pivots and the experience was very smooth, taking only a few minutes. We how have an "Iconistan" of social bookmarking chiclets for either remembering or promoting content on Digg, reddit, del.icio.us -- almost 20 sites in all.

Install via the normal plugin install process (the -x installs it as an SVN:EXTERNAL):

#> ruby script/plugin install -x svn://rubyforge.org/var/svn/pivotalrb/bookmark_fu/trunk/bookmark_fu

I did have one issue -- the script/plugin install script pulled all the code down but ultimately failed because we have multiple versions of Rails on our development machine (about 5); this seemed to confuse the install script. No problem, though: I ran the install.rb script manually:

#> script/runner vendor/plugins/bookmark_fu/install.rb

(6:30 pm: updated to use mysqldump) (12/14/07: updated to remove db:reset since the Rails 2.0 version now does something different.) (12/15/07: updated to not set ENV['RAILS_ENV'] since that gets passed down to child processes)

There was an old hacker who lived in a shoe; she had so many migrations she didn't know what to do. Every time her build ran clean, she spent a whole minute staring at the screen.

Fortunately, she read this blog post and now her db:setup task is so fast she's started building multiple test environments so she can run tests in parallel!

QueryTrace is a great Rails plugin (which I learned about from ErrTheBlog) for pinpointing where in your Rails application that slow query is running. Once you have it installed, your logs won't just tell you that you have a problem, they will pinpoint the exact location of that problem for you. This is invaluable when doing load & performance testing or just trying to understand what the hell ActiveRecord is doing.

Unfortunately, even though it only logs query traces in DEBUG log mode, it still clutters up your test and development logs, and actually slows things down a bit too. So you don't want to leave it in your project's vendor/plugins directory after you're done using it. So I wrote a pair of rake tasks to enable and disable it.

rake query_trace:on                     # Enables the query_trace plugin. Must restart server to take effect.
rake query_trace:off                    # Disables the query_trace plugin. Must restart server to take effect.

The "on" task actually checks out the plugin from query_trace's subversion repository, makes a tarball in vendor/query_cache.tar.gz, then leaves the tarball around so it doesn't have to keep going to the network all the time. You can even check the tarball in to your project and it'll never go to the query_trace repository again. The "off" task just removes the whole vendor/plugins/query_cache directory.

namespace :query_trace do
  desc "Enables the query_trace plugin. Must restart server to take effect."
  task :on => :environment do
    unless File.exist?("#{RAILS_ROOT}/vendor/query_trace.tar.gz")
      Dir.chdir("#{RAILS_ROOT}/vendor") do
        url = "https://terralien.devguard.com/svn/projects/plugins/query_trace"
        puts "Loading query_trace from #{url}..."
        system "svn co #{url} query_trace"
        system "tar zcf query_trace.tar.gz --exclude=.svn query_trace"
        FileUtils.rm_rf("query_trace")
      end
    end
    Dir.chdir("#{RAILS_ROOT}/vendor/plugins") do
      system "tar zxf ../query_trace.tar.gz query_trace"
    end
    puts "QueryTrace plugin enabled. Must restart server to take effect."
  end

  desc "Disables the query_trace plugin. Must restart server to take effect."
  task :off => :environment do
    FileUtils.rm_rf("#{RAILS_ROOT}/vendor/plugins/query_trace")
    puts "QueryTrace plugin disabled. Must restart server to take effect."
  end
end