riot.projects

record_filter adds column comparisons across tables

I just released version 0.9.14 of record_filter with a spiffy new feature that allows you to compare columns across tables instead of simply comparing columns against your data. Let’s say you have a Blog class that has_many Posts and you want to only get the articles that were created since the last time their publication was updated. Now you can do this:

Blog.filter do
  having(:posts)
  with(:updated_at).lt(:posts => :created_at)
end

This came in handy for the world’s nastiest query that I’m working on, and I hope it will help others as well.

Setting up Kerberos on Snow Leopard

For some inexplicable reason, the Kerberos tools have been pretty much completely removed in Snow Leopard, in favor of a simple “Ticket Viewer” app that manages tickets but doesn’t help at all with the Kerberos setup. If you don’t have an existing kerberos setup, your only option in the ticket viewer will be “Add Identity” and it will give you this excellent error message when you try to use it:

Configuration file does not specify default realm

Because they have removed the setup tools, you now need to configure Kerberos yourself by editing the /etc/krb5.conf to add the default domain and configure the various servers. Here’s an example of the config file that we use on our servers:

[libdefaults]
default_realm = AOL.COM


[realms]
AOL.COM = {
kdc = server1
admin_server = admin_server
default_domain = AOL.COM
}

Replace AOL.COM with your realm, replace the various servers with the IP addresses that apply to your setup and add any more kdc servers you need and you should be good to go.

RecordFilter turns 0.9.8

Record Filter, everybody’s favorite ruby library for specifying queries in ActiveRecord, has just hit version 0.9.8. This version has some handy improvements for doing distinct queries and aliasing tables in joins, but it in some cases it is not backwards compatible with previous versions. If you are using an earlier version, take note of the breaking changes described here.

Distinct

Prior to 0.9.8, there was no support for forcing queries to select distinct results. The new version adds a ‘distinct’ method to the DSL to take care of that situation.

Blog.filter do
  with(:created_at).greater_than(1.day.ago)
  having(:posts).with(:permalink, nil)
  distinct
end


# SELECT DISTINCT "blogs".* FROM "blogs" 
#  INNER JOIN "posts" AS blogs__posts ON "blogs".id = blogs__posts.blog_id 
#  WHERE (("blogs".created_at > '2009-06-03 14:00:56') 
#  AND (blogs__posts.permalink IS NULL))

Table aliasing for joins

In earlier versions, it was only possible to set a specific alias for a table that was joined with an explicit join. That caused problems in cases where you want to either specify a specific alias with an implicit join or to join against the same association twice. So, this version adds support for an options hash for both types that supports both :alias and :join_type. This is a breaking change from 0.9.7, where join types were specified differently for each method and were required for only one.

Blog.filter do
  having(:posts, :join_type => :left, :alias => 'posts_1').with(:title, nil)
  having(:posts, :alias => 'posts_2').with(:title, 'abc')
end


# SELECT DISTINCT "blogs".* FROM "blogs" 
#  LEFT OUTER JOIN "posts" AS posts_1 ON "blogs".id = posts_1.blog_id 
#  INNER JOIN "posts" AS posts_2 ON "blogs".id = posts_2.blog_id 
#  WHERE ((posts_1.title IS NULL) AND (posts_2.title = 'abc'))

In earlier versions, the alias argument was not available and the join_type was passed as the first parameter. For explicit joins, the same options hash is used, which is a change from 0.9.7 where the join type and alias were passed as parameters.

Blog.filter do
  join(Post, :join_type => :left, :alias => 'da_posts') do
    on(:id => :blog_id)
    with(:title, 'def')
  end
end


# SELECT DISTINCT "blogs".* FROM "blogs" 
#  LEFT OUTER JOIN "posts" AS da_posts ON "blogs".id = da_posts.blog_id 
#  WHERE (da_posts.title = 'def')

For both types, :join_type defaults to :inner and :alias defaults to a clever, unique name for the table.

Changes to limit

This version brings a breaking change to the ‘limit’ method, which was guilty of some ugly argument swapping in the previous version. The method still allows you to specify both a limit and an offset but now makes the offset an optional second argument rather than the first.

Comment.filter do
  with(:offensive, true)
  limit(10, 100)
end

# SELECT * FROM "comments" WHERE ("comments".offensive = 't') 
#   LIMIT 10 OFFSET 100

If anybody has problems with these changes, feel free to get in touch… aubreyholland at gmail dot com

Ruby 1.8 funkyness with Class.new and inherited

I just found a bug in Sinatra where this code will fail with the error “can’t dup NilClass”:

Class.new(Sinatra::Base) do
  get('/') { 'abc' }
end

Sinatra uses the ‘inherited’ method to initialize a number of class variables in new subclasses of Sinatra::Base when they are defined. For normal subclassing, this works great because ‘inherited’ is called before the contents of the class are evaluated. When using Class.new, however, it is called after evaluating the class contents, which means that Sinatra will not have had a chance to set up the variables when ‘get’ is called in this example. Apparently Ruby 1.9 has fixed this issue, but we’re still stuck with it in 1.8.

Run this code and you’ll get:

Inherited!
Some method!
Some method!
Inherited!

class Base
  def self.some_method
    puts 'Some method!'
  end

  def self.inherited(subclass)
    puts 'Inherited!'
  end
end


class Sub < Base
  some_method
end


Class.new(Base) do
  some_method
end

It would be lovely if quick fix for doing this in Sinatra was to call ‘reset!’ as the first line of the class definition, but unfortunately because inherited is called after the class contents are evaluated all of your routes will be wiped out.

Class.new(Sinatra::Base) do
  reset!
  get('/') { 'abc' }
end

Simple Rails performance analysis

Working on fixing performance problems in our rails app, I often find myself interested in simple performance metrics, like the number of queries that are performed, the slowest partials, and the slowest queries. These things are all there in both the output from script/server and the rails log, so it seems that a lightweight tool ought to be all that’s required to see what’s going on. I recently wrote a simple script to do just that. All you need to do is download the script and then pipe the output of the rails log into it:

script/server | log_analyzer.rb

tail -f log/development.log | log_analyzer.rb

This will give you the standard output plus some interesting facts like these:

Top five partials:
global/_header - 1.01129 - 35%
global/_navigation - 0.73292 - 25%
global/_navigation_sub_menu - 0.60205 - 21%
global/_most_popular - 0.3204 - 11%
posts/_post - 0.28982 - 10%


Top five selects by type:
Type: Post Count: 59 Time: 0.020275 Percent: 0%
Type: Article Count: 51 Time: 0.017761 Percent: 0%
Type: Section Count: 1 Time: 0.01623 Percent: 0%
Type: Tag Count: 23 Time: 0.009717 Percent: 0%
Type: Place Count: 17 Time: 0.00725 Percent: 0%


Top five selects by count:
Type: Post Count: 59 Time: 0.020275 Percent: 0%
Type: Article Count: 51 Time: 0.017761 Percent: 0%
Type: Section Count: 23 Time: 0.009717 Percent: 0%
Type: Tag Count: 17 Time: 0.00725 Percent: 0%
Type: Place Count: 16 Time: 0.005853 Percent: 0%

By the way, if your output looks like this, your app is slow.

ActiveRecord observers in gems/plugins

I’ve recently been working on a caching tool for Rails that has a declarative API for invalidating caches when objects of specific types are changed. The idea is that we should be able to tell the caching system that specific types of caches are automatically invalidated for a given type:

cache_config.article_cache :expiration_types => [ :article ]

This requires that the gem be able to set up an ActiveRecord observer in order to do the requested invalidations. There are a couple of non-obvious challenges here that seem worth working through. The first issue is that the observer needs to be added to the list of ActiveRecord::Base.observers in order to be fired correctly. One option is to require the user to set this up in their environment file or in an initializer, the standard places for this sort of thing, but that pretty much ruins any sense of transparency you might have had. A better option is to set it up yourself in the rails initializer of your gem. The three big lessons here are:

1. If you create a rails/init.rb file in your gem, it will be executed as the rails app is initialized.
2. When this file is executed, the Rails::Configuration object used in environment.rb is in scope, and
3. The Rails::Configuration object has some lovely hook methods you can use to initialize your observer

config.after_initialize do
  ActiveRecord::Base.observers << CacheAdvance::ActiveRecordSweeper
end

OK, so we’re all set, right? Unfortunately, your awesome observer is just about to fail miserably in development mode. The problem there is that while the observer in your gem isn’t going to get reloaded with every request, the models that it’s observing will, and that will break the observer/observed relationship. The solution is to force the observer to restate what it observes, and fortunately Rails has a callback to help us do just that:

config.after_initialize do
  if config.action_controller.perform_caching
    ActiveRecord::Base.observers << CacheAdvance::ActiveRecordSweeper

    # In development mode, the models we observe get reloaded with each request. Using
    # this hook allows us to reload the observer relationships each time as well.
    ActionController::Dispatcher.to_prepare(:cache_advance_reload) do
      CacheAdvance::ActiveRecordSweeper.instance.reload_sweeper
    end
  end
end

The to_prepare method in ActionController::Dispatcher allows you to specify a block that will get called before the first request in production mode and before each request in development mode, and we can use that block to restate the observer relationships:

class ActiveRecordSweeper < ::ActiveRecord::Observer
  def reload_sweeper
    observed_classes.each do |klass| 
      klass.name.constantize.add_observer(self)
    end
  end
end

And with that you should be observing like a champ.