I stumbled across a little bit of hidden Rails fun last night when I was trying to get the form_for method to stop wrapping error fields with extra div tags. Did you know that? Maybe you never noticed, but when you use the field helper methods, like text_field, password_field, etc, Rails will wrap fields with errors in a <div> with the class ‘fieldWithErrors’.

This was causing all sorts of grief for me in some JavaScript I was trying to write. At first I tried to go with the flow and fix my JS, but it got really hacky really fast. So I went on a little source-code spelunking to figure how to make the problem go away.

In actionpack-2.1.0 there is a Proc attached to the ActionView::Base.field_error_proc class attribute. It’s not documented in the RDoc, but this is also a writable attribute which means I can shut the damn thing up. Here’s what it looks like in the original file, GEM_HOME/actionpack-2.1.0/lib/action_view/helpers/active_record_helper.rb:

require 'cgi'
require 'action_view/helpers/form_helper'

module ActionView
  class Base
    @@field_error_proc = Proc.new{ |html_tag, instance| "<div class=\"fieldWithErrors\">#{html_tag}</div>" }
    cattr_accessor :field_error_proc
  end
  ...
end

My solution was to create a little file in config/initializers named field_errors.rb with this text:

ActionView::Base.field_error_proc = Proc.new do |html_tag, instance|
  html_tag
end

Et voila! Just a simple pass-through with no more fancy-pants markup. Putting stuff like this in separate files in the initializers directory keeps your config and environment files from getting out of hand.

I’ve just spent the last two hours pulling my hair out trying to get Single-Table Inheritance (STI) working with associations in ActiveRecord. After essentially walking through all of the possible ActiveRecord options in this setup, I finally stumbled upon a configuration that seems to work. So this post is an attempt to help the next poor bastard who is Googling in earnest for a solution to a similar problem.

So let’s start with the domain model. I’m too spent at this point in the evening to port this to one of the standard examples. Instead I’ll expose you to the domain of my particular problem. The app I’m working on is one that tracks (non-financial) lending transactions between two individuals. The parties involved, the item in question and when it’s due are all tracked in the Transaction model (and transactions table). A Transaction has a number of states it walks through, using the acts_as_state_machine plugin. These state transitions are triggered by opaque-looking URLs that are sent via email to either party. These are one-time use actions that once consumed are no longer available. When an Action instance is created it also has a before_save callback that generates a unique ID (used in the URL) using Digest::SHA1.

So my plan was to have my Transaction class write one or more Action records for each possible action based on my state transitions. Take a look at the state diagram below:

I want to encapsulate the actual work to be performed within the Action instance the user invokes by following the link in their email. So my plan is to use STI to have different sub-classes of Action that operate on a transaction and march it forward to its next state polymorphically.

Now STI may appear to be total overkill for this problem, but here are my reasons for going this route:

• I want to have these opaque IDs written down somewhere to associate a specific action with a URL
• When the action is complete, I want to remove the record so it can’t be performed again
• The state for a given Transaction can have more than one possible action. I want a separate for each action.

Whew. Okay, clear so far?So my initial code looked something like this:

require "digest/sha1"

class Action < ActiveRecord::Base

  belongs_to :transaction
  before_save :create_guid

  def create_guid
    sha1 = Digest::SHA1.new
    sha1.update transaction_id.to_s
    sha1.update type.downcase
    sha1.update DateTime.to_s
    self.guid = sha1.hexdigest
  end
end

class ReturnAction < Action
  def execute
    transaction.return!
  end
end

class AbortAction < Action
  def execute
    transaction.abort!
  end
end

class DisputeAction < Action
  def execute
    transaction.abort!
  end
end

It seemed like a good idea at the time, but the strange thing was that no matter which incantation I tried, I simply couldn’t create a new Action instance and have it write a record to the database. This simply didn’t work:

ReturnAction.create! :transaction_id => 1

There were no errors on the returned object. No exceptions were thrown. No queries to the database and certainly no insert statements executed. Just complete and utter silence. Out of desperation, as much as anything else, I removed the belongs_to declaration from the Action class and instead declared a has_many on the Transaction class. Voila! It worked like a champ.

After a bit of thought, the has_many association makes complete sense to me in the case where we want to create new Action instances for a particular Transaction. However, if you look in the code above, the execute methods of each sub-class are referring to a transaction object/method—which I no longer have. However I don’t necessarily need the full-blown belongs_to association here. I can just fake the bits I want in the parent Action class like so:

class Action < ActiveRecord::Base
  def transaction
    @transaction ||= Transaction.find(self.transaction_id)
  end
end

So none if this is particularly earth-shattering. Sorry folks, no great gems of philosophical wisdom today. Just one man’s small accomplishment blown completely out of proportion.

RailsConf closed up last Sunday afternoon and after three-hour drive back and day of work to contemplate, here’s what I’ve boiled it down to:

The Good:

I met a ton of people this year. Last year I went with a co-worker and we pretty much stuck together. This year I was on my own and made a concerted effort to just meet folks. By the end of the conference I couldn’t go more than about fifteen minutes without running into someone I had previously met. I ordered a fresh batch of Moo cards before I left and I was hell-bent to hand as many out as I could. Just meeting people turned out to be my favorite part of the conference.

The Kent Beck address was fantastic. I’ve had the fortune earlier in my career to work at a company that had Kent come out and run XP workshops with us. Those experiences left a last impression on me (much like Chad Fowler expressed in the introduction) and so I was eager to hear his talk after seeing him on the roster. While his main content probably dragged for a bit, the Q & A ended with a bang. The answer he gave to the final question expressing a mixture of hope and concern brought the crowd to its feet. Go on, Kent.

There were a couple of presentations I went to that I though really knocked it out of the park. In some cases the material alone saved the day (in spite of the presenters) and in a few other cases the two came together nicely. I thought some of these presentations were particularly good:

• “Facebook Development and Performance with Rails” – Mike Mangino
• “The Launch: Do’s and Don’ts of Real-life Deploys” – Chris Wanstrath
• “Assembling Pages Last: Edge Caching, ESI & Rails” – Aaron Batalion
• “Skynet: A Ruby Map/Reduce Framework” – Adam Pisoni
• “Vertebra” – Ezra Zygmuntowicz
• “Advanced ActiveRecord Techniques: Best Practice Refactoring” – Chad Pytel

The Bad:

Sadly, a number of the presentations were pretty lacking. Now I think presenting is just plain hard and very few people are good at it. Keynote helps a bit, but really it’s a crutch for people who don’t have good public speaking skills (which I’m not necessarily claiming I have). Really exceptional content can help overcome the stylistic short-comings of a particular speaker, but I think that’s rare. I think it’s pretty easy to lose an audience quickly if your material can’t shine in the way you present it.

Since I have another year of Ruby and Rails experience under my belt, many more of the talks just really didn’t do anything for me. That’s why I’m psyched that the RailsConf team has decided to incorporate Caboose Conf as the “hallway track” in next year’s meeting. I think that’s the track I’ll be taking next year. I did a lot of ingesting this year, next year I’d really like to produce more.

The Style Report:

Apparently the black t-shirt is king in the Rails community. Something like 99% of all attendees had black t-shirts on. Of the free t-shifts to be had, the GitHub tees were the only ones that weren’t black. All of the others were red on black. So apparently the new black is, well, black. I’m hoping that nuclear orange makes a comeback next year. All-black is just a little too Depeche Mode for me.

Portland:

Portland deserves special mention because I just flat-out love that city. Besides Seattle, it’s probably the only other city I would choose to live in. One of the highlights of experiencing the local flair was getting involved in the 1000-person “doughnut march” held by Portland’s beloved Voodoo Doughnuts. Somehow they convinced officials to get a parade permit and police escort as they crossed from their Pearl District digs to their new shop in East PDX. It was a very “Portland” experience with a whole crowd of folks letting their freak-flags fly high.

The big treat at the end was the bacon maple-bar. I’m not kidding folks, this is real and it’s freakin’ brilliant. So hear this Chad Fowler and the rest of the RailsConf committee, please don’t move this to Vegas! Besides, can you imagine putting a bunch of hygienically-challenged nerds in 100-degree heat in a desert? That is simply not a good idea.

The day opened with a disappointing keynote speech from Tim Bray. He was upfront about the fact that Sun wrote a big check to big a sponsor and proceeded to launch, unapologetically, into a forty-five minute advertisement of Sun and just how great their servers and JRuby are.

In contrast the opening act, Cyndi Mitchell of ThoughtWorks, had a much better presentation that called out to the Ruby and Rails community to “take back the enterprise”. Her slide deck and presentation were fantastic. Anyone who crosses John Travolta’s awful “Battlefied Earth” with Dick Cheney gets big points in my book. I hope they post them somewhere.

Memcache

This feels like the year the conference has gotten more “serious”. To that end, a consistent underlying theme of the whole conference has been scalability and performance. I’ll walk away from this conference with a new and, hopefully, enlightened view of the whole software/hardware ecosystem. It ain’t just about Rails folks. A real production app has a lot more moving parts than what’s on your dev box.

My first morning session was Chris Wanstrath’s presentation about Rails and memcache. I wasn’t real close so I didn’t get a good look at the dude, but from where I was sitting I’d swear he looked an awful lot like Shaun White (aka “the Flying Tomato”).

I loved the fact that his first slide in his presentation was titled “YAGNI”. Memcache is such a neat tool with such a clear, appealing design that it’s easy to be wooed by it. Chris made an excellent point that you shouldn’t even think about using memcache until you have some real numbers backing up your need.

Chris showed off the cache_fu plugin which looked like it did everything short of make you breakfast. I’ll definitely have to spend some quality time with the docs on that. He also hipped us to libketama which is a cool replacement for memcache’s default host-hashing algorithm. It allows you to dynamically add and remove memcache nodes without invalidating your entire memcache cluster.

What’s brilliant about memcache is just how dumb it is. That dumbness keeps it simple to manage and deploy. That means though, that there is a back-pressure in your system to use the tool wisely. However it seems like a fair trade-off, especially when you consider that any decent performance-enhancing tool should force you to think about what the hell you’re doing.

Getting Real Numbers

The next session was put on by Julian Boot of ThoughtWorks, a hyper-active guy who was clearly more enthusiastic about his talk than about 90% of the presenters at the conference. He started his presentation of with a bang by making the claim that no off-the-shelf testing framework or product would be able to do proper performance testing for your application. Because setting up a test harness is (relatively) cheap and each team/environment/application is different, this is the one case where rolling-your-own makes more sense.

He had a couple of interesting concepts that I hadn’t thought of. He suggested including a 90th-percentile measurement in your summary stats (y’know, min, max, avg) since max response times can jack your average. Seems like a helpful data-point to me.

He also didn’t want to fool around with configuring several test nodes so that reporting stats would land in a single place. His answer was to simply have each test node send its stats out via UDP broadcast. A single stat-gathering “master” would listen on the broadcast address and simply write the packet contents (serialized samples) to a flat file for processing. In a switched network the likelihood of losing UDP packets is pretty low. I loved it the solution—it was brilliantly dumb and took about five lines of code.

His last simple suggestion was to have each test client node output a single character indicating its state. I’ve seen a lot of processes that do this and the output is just noise. What makes this different though, and why I like it, is that you are actually interested in what’s going on at that moment. Often this kind of output is misapplied because it isn’t helpful until the process finishes and the output is too terse to be of much help.

One of his key points is that writing a test harness is relatively simple. That hard part is the analysis and decision making made after the fact. While writing the harness might be fun, it’s a fraction of the time you will spend writing the ugly little test actions that will simulate what your users are doing. Julian stated that you know your test harness is correct when your test logs look like your production logs.

Nice answers to dumb questions

In between the morning keynote and the first session, I had a thirty minute window to get caffeinated and catch up on email. I only had to do the first, didn’t want to bother with second, and needed to find something to entertain myself until the next session. In the main foyer I noticed that DHH was chatting with a few folks. After a drive-by skulk realized that they were talking about one of my favorite topics, REST.

During the conference I’ve been noodling around with the resources and URLs in a little side project I’m working on and I was getting really stuck on a satisfying way to apply REST-ful modeling to sessions and credentials. Since sessions are really managed by cookies, it’s kind of a round-peg-in-a-square-hole problem when applied to REST. So I walked up and posed this question to DHH directly. He hipped me to the map.resource call in the routing configuration and the concept of ‘singleton resources’. We chatted a little bit more, I thanked him and headed off to my next session.

Later, I started going over the Rails docs to get hip to this singleton resource concept. I realized that my question was on the edge of warranting a RTFM answer, but David was gracious enough to take the time to describe his solution and what the mechanism was. I have to say that I was really impressed with how helpful and gracious he was. I know that people sometimes want to treat DHH as a rockstar, but I think part of what has made Rails successful is the lack of a deeply-nested hierarchy that separates the ivory-tower types from the great unwashed masses. It’s comforting to know that you can pose a n00b question to someone like David and get a polite, helpful answer. I have great hope for this platform and this community.

RejectConf 2007

Living in Seattle I’m really fortunate to have access to such an active group of Rubyists. One fellow Seattle-ite, Ryan Davis, put on RejectConf which was described as a “gong show” style presentation forum where people get up and give very short (< 5min) presentations on things they’ve been working on.

The near-anarchic atmosphere (fueled by beer and snacks graciously donated by Addison-Wesley) gave a great energy to the gathering. Most of the ideas were pretty interesting, some were fringey, but it was a great forum to see what the rest of the community is up to. Oh yeah, a big shout-out to Free Geek for providing the space.

Day Three Takeaway

By the end of the day I came away feeling like I have some serious wood-shedding to do. I’ve only been doing Ruby for about nine months so I don’t have a problem with this. For years I’ve relied on being an experienced Java programmer to give me that nice warm feeling of comfort and confidence. However that has also led to a sneaking feeling of gathering dust. I jumped onto Ruby because it reached a good head of steam right as I was looking for something to push myself in new ways. Right now, I’m okay with being a total n00b dork to learn something new.

After the keynote speech by David Heinemeier Hansson, I attended several session that felt disparate at the time, but magically coalesced into a meaningful takeaway by day’s end. In short the day was about testing, REST and Amazon’s suite of Web Services.

Getting Religion With Uncle Bob

Update 2007-05-19: Robert posted a PDF of his slides here.

The first session I attended was one titled “Clean Code” presented by Robert C. Martin. I will admit that I’m only familiar with his name as being the “signature” endorsement on some of my favorite texts by the likes of Kent Beck and Martin Fowler. After hearing him speak I can see why he has endorsed their writing and would include him into the small team of pragmatic agile bandits consisting of the aforementioned Kent and Martin as well as Ward Cunningham.

The point of the talk was essentially about constant vigilance. Putting off fixing smelly code today only leads to much more expensive changes later—either in the form of grand redesigns (a bad idea) or in incrementally paying down the debt you’ve incurred. The answer is to use the tried-and-true weapons at your disposal are a refined nose, testing and refactoring.

He walked through a pretty good test-driven development (TDD) example that I suspect surprised a good bit of the crowd. I say ‘surprising’ because when someone who doesn’t use TDD sees it for the first time, it looks very dogmatic and slow. People who don’t do TDD get really hung up on this, but fail to understand what underpins the approach: humility. TDD is really based on the notion that we all write legacy code—no one is brilliant enough to get it right the first time.

Another hang-up point for first-timers is the use of the term “test-first”. I’m surprised at how often people think that means your write a comprehensive set of tests before you write any code. This, of course, is nonsense. Just like writing a detailed specification with pseudo-code up front is not a practical pre-cursor to a healthy system realization. I’ve always preferred the term “test-driven” as I think it sheds a little more light on another of unit-testing’s benefits which is another form of system feedback.

Sometimes you have a test that is so ugly that it alone will make you refactor code even though your production consumers may not really be suffering in the same way. Why would you do this? Test-code isn’t production code, why let it force change? My answer is that you change the code because of the test for the same reason you change code used by the production system: if it hurts to understand or modify you have found a smell. If you let that smell linger, you will start incurring “debt” on your project.

While I didn’t get any real juicy new bits from Uncle Bob’s talk, he was an extremely entertaining speaker who made and excellent presentation. If you ever get a chance to hear him speak, it’s worth it.

Pushing the Boundaries of Rails Testing

The next session was put on by Jay Fields with ThoughtWorks. I think he probably had some very good points to make but without a few visual examples a lot of his points were lost on the audience. However he did make some points that brought questions for me about testing.

When writing Java I’ve found that my tests and classes are quite fine-grained compared to the Rails code I test. In part a lot of that is to enforce a stronger separation between domain-level code and external resources like RDBMS persistence. When you use ActiveRecord in Rails, your model is welded directly to the database. When I first saw this as a Java programmer my nose wrinkled up and I was extremely suspicious about this approach.

To my surprise testing ActiveRecord classes directly against a database has been much less painful than I suspected. I think this probably has to do with supporting structures such as rake tasks and fixtures that don’t require you to drop down to SQL just to test your model. On the other hand, if your code is going to evolve to anything moderately sophisticated I think it’s possible to exceed the capacities of these facilities. It’s something to pay attention to for sure.

Jay also threw out a couple of other thoughts that caused unease in the audience:

• While your code under test is object-oriented, your tests don’t have to be
• Repeating yourself in your production code it bad, but it’s okay in your tests
• The value of private and protected methods in Ruby is debatable

Like books that get banned or musicians that get called up in front of congressional committees, the very things that give you unease are probably worth addressing head-on rather than sweeping under the rug. At the end of the session, my final thoughts were that we have a long way to go in evolving the craft of testing.

REST

I had about ten minutes to kill between sessions and I was desperate for a free cup of coffee. In a stroke of marketing genius the conference organizers moved the coffee bar from breakfast area to the vendor’s floor show. Alright, I’m game. I’ll look at your ads to get a free cuppa joe.

Most of the booths held no interest until I stumbled across the Powell’s Books stand. There before my eyes were stack of geek books. The night before I made the trek out to Powell’s via Portland’s delightful MAX transit system. If you’ve never been to Powell’s do yourself a favor and go. For geeks, the Technical Bookstore simply can’t be beat.

Anyway, I figured there wouldn’t be anything new there that I hadn’t seen the night before at the Mothership. Of course, I was wrong. There before my eyes was shiny new O’Reilly book titled “REST Web Services”. I had been aware that Leonard Richardson and Sam Ruby were working on this book, but had lost track of when they were actually going to publish. I spent a little quality time with the chapter on S3 and my hopes were confirmed—this is going to be a good book.

REST has gotten a lot of press lately, but I think a lot of people really don’t have a good grasp of the concepts. To this point the only literature has either been Roy Fielding’s original dissertation and a few derivatives of that paper. I think Fielding’s paper is definitely with reading. It’s very dense and repays the reader with each read. However, for REST to truly take hold the community really needs a thoughtful source that boils down the essence of the architectural style in a consumable format. Given the eagerness with which REST is being embraced and the dearth of practical resources on this topic, I would expect this book to be a well-loved, dog-eared volume on a lot of desks.

Amazon Web Services

We took a long break from the geeks and had a fine dinner at McMenamins Kennedy School. We returned to the conference and headed off to a late-night BOF session about Amazon’s S3 Service. The turnout seemed pretty good. After a brief dog-and-pony show from an Amazon rep the floor was opened to the attendees. Despite the size of the session, only a handful or participants were actually using S3 in production—everyone else was curious to see how it was being used.

While Amazon won’t expressly claim that S3 can replace content distribution networks (e.g. Akamai), they aren’t scaring people away from it. I would venture to guess that %50% of the attendees wanted to know if they could store assets in S3 that they could serve back up directly to their client. My impression is that Amazon is trying to evolve S3 to this kind of platform. This could be really big and is definitely going to be something I’m going to keep my eyes on.

Post-Game Show

I needed a little ramp-down time before I turned off the light for night-night time. I cracked open the “REST Web Services” book and started reading the chapter about S3 until my eyes drooped and I slipped into deep relaxation. As I slipped into unconsciousness, a few final thoughts on the day coalesced:

• If Rails is a big simplifying assumption around web apps, REST is a big simplifying assumption around web services
• While the Rails community is pretty good at embracing testing, the craft has a long way to go (this is true of all software platforms)
• AWS has the potential to make a fundamental economic and technical shift in the way large web applications are deployed

This is my first RailsConf so I can only say this given second-hand information. But it seems that I’ve arrive just as the interest and popularity in the framework has hit the bend in the hockey-stick. The enthusiasm at the morning keynote address by David Heinemeir Hanssson was equals parts technology preview, religious revival and idol worship.

I’ve always found the use of ‘DHH’ to be a little off-putting as it always smacked of a strange geek cult-of-personality. However given the length of the guy’s name, I can see why he went with the initials. At least it’s better than some obtuse l33t h4×0r handle.

Anyway…

Let me give you a quick 30,000 foot overview of things coming in Rails 2.0.

REST-ful Resources are the New World Order

“The world of resources is a better, greener place” — DHH

REST will be the default way things are done in 2.0, meaning that there will be more default notions built-in to support the style.

Controllers

def create
  @person = Person.create(params[:person])
  respond_to do |format|
    format.html { redirect_to(@person) }  # person_url(person)

    format.xml do
      ...
    end
  end
end

Look at how simple it is to render a the @person object. Isn’t that lovely?

Views

Views get cleaner by using a single form_for that understands updated vs. new creations (as well as the default URLs for that resource and actions). You no longer have to “drop-down” on the bare metal and expressly configure your form_for declarations with the correct URL and HTTP verbs.

Routes

REST-ful routing is getting a bit of a makeover, adding some more flexibility within the otherwise liberating constraints of REST.

# /admin/products/inventory
# /admin/products/5/tags
# /admin/products/5/seller
map.namespace(:admin) do |admin|
  admin.resources :products,
    :collection => { :inventory => :get },
    :member => { :duplicate => :post },
    :has_many => { :tags, :images, :variants },
    :has_one => :seller
end

9 other things to like about Rails 2.0

The best part of the keynote was the preview of new features in Rails 2.0. There is enough good-looking stuff in here that I’m probably going to move my main Rails side-project to Edge Rails and feel the good lovin’ on the bleeding edge.

Breakpoints are Back!

This was broken by a fix made in Ruby 1.8.5. The takeaway of course is not to implement features based on hacks.

ruby-debugger

One of the few embarrassing things about developing in Rails is the lack of a first-class debugger. So far I’ve been able to work with Rails quite well without having a debugger. In part this is because using a Test-driven approach means the debugger is usually the last tool you reach for rather than the first. But I will have to admit that I have had to “devolve” into using logging or puts statements to dump what was going on in the running app.

Thanks to the inclusion of the ruby-debug into the core you can actually do real debugging in Rails. With this new technology you can set a breakpoint in your code with a single debugger line in your code. When the code is hit your console will drop into a command-line interface where you can:
* View your place in the stack
* Modify objects in place
* This looks like it’s gonna rock!

HTTP Performance

JS/CSS Bundling

There is a tension between segmenting JS and CSS into separate files and HTTP performance. As a developer you want to separate our Javascript and CSS to keep your head straight. But the performance of downloading those assets suffers given the de-facto two-connections per site limitation in browsers. Rails 2.0 will have a way to send a single request to Rails for a collection of JS and CSS at once and it will returned as a single gzipped bundle.

<%= javascript_include_tag :all, :cache => true %>
<%= stylesheet_link_tag :all, :cache => true %>

Connection limits

JavaScript and CSS aren’t the only assets where you run into the two-connection limit. Any other static assets (particularly images) can kill you here. Rails 2.0 will have support for a type of name-based virtual hosting for your assets so that you can ”trick” the browser into using more simultaneous connections to download content. You can set this in your config:

config.action_controller.asset_host = 'asset%d.example.com'

Query cache for ActiveRecord

This new feature sniffs the SQL being executed and looks between calls for the exact same SQL. If any UPDATEs or DELETEs were not executed between queries that affected those records AR will return a cached version of the objects.

MIME type handling

In the past the means to generate content and its format were mixed into the name (ex. index_rss.rhtml). Rails 2.0 has made some naming changes to better call out the separation between the type of the final content and the means by which it was generated.

Now view files look like [template].[format].[rendering]. For example:
* people/index.html.erb # Use ERB to generate HTML
* people/index.xml.builder # Use builder to generate XML
* people/index.rss.erb # Use ERB to generate RSS
* people/index.atom.builder # Use builder to generate Atom

Configuration initializers

Application-specific configuration that used to go into config/environment.rb now goes into separate files in the config/initializers directory. I haven’t felt the pain of this, but I can certainly imagine how this might be helpful.

Sexier Migrations

A typical migration looks something like this:

create_table :users do |u|
  u.user_name :type => :text, :null => false
  u.first_name :type => :text
  u.last_name :type => :text
  u.age :type = >:integer
end

See all the redundancies? Why keep specifying the :type parameter. Why not reverse it like this?

create_table :people do |t|
  t.integer :account_id
  t.string :first_name, :last_name, :null => false
end

HTTP Authentication

Because REST-ful web services have become much more first-class in Rails, the need to support Basic HTTP Authentication has become important. HTTP Basic Auth for browsers and user-facing applications just plain sucks: there is no notion of “logging out” and there’s no control of the user interface. However for web services HTTP Basic Auth is fine. Rails 2.0 provides a new authenticate_or_request_with_http_basic method that can be used in controllers. This is very nice for computers interfacing to your APIs

Spring cleaning

Remember those deprecation warnings you see in the console when you run your app? Yeah, it’s time to address those and clean them up. Those deprecated features are going to be going away in Rails 2.0. Fix your code. Don’t say you weren’t warned.

Update: 2007-05-19 — Bruce and Marcel posted a PDF of their talk here.

After knocking down lunch I was faced with a conundrum. I had signed up for a tutorial on view-layer patterns, but saw that Jamis Buck was doing a tutorial on Capistrano. When I registered originally I had no idea that Cap 2.0 was coming out and I didn’t really care to get any more familiar with Cap 1.0. But between then and now Cap 2.0 came out and it looks like there is some good stuff to learn. But the view tutorial looked good too. What to do? What to do?

The tutorial that Bruce Williams and Marcel Molina gave for refining the view layer was excellent. I think a lot of the content seemed to go over the heads of the audience–they seemed to get a little too wrapped up in the ugly details rather than the important over-arching points.

The first point was that the view-layer deserved as much expressiveness and thoughtful design as we put in our models and controllers. In essence, we prefer expressing the what rather than the how in our code. Why not apply that to the view layer? This is general concept that I hope to explore in a later post.

They started with a brief survey of other view layer technologies such as ERB, Markaby, HAML and others. However the takeaway from this review was that most of these technologies were focused on cutting down on the amount of typing necessary to do the templating, but didn’t really provide a means to increase domain-level expressiveness. The approach proposed by Bruce and Marcel was to embrace ERB/RHTML and use the power of Ruby to factor out complications within the view.

One way to achieve this factoring is paying attention to the natural tensions that emerge as you evolve your application. When you find that certain names come up over and over again in describing the system with your customer, you probably need something with that name in your system.

Another way to do this is to naturally evolve your view. Using regular ol’ ERB in RHTML is probably a great way to start out. However once you start getting things in your templates like calls to finders, conditional logic, calls to Enumerable methods or temporary variables, it’s time to push that work into a terse, but descriptive, method in your view helper.

When you accumulate enough of these helper methods that share state or particular strategies, it’s probably time to make it into an Object. Normally people think of Objects as belonging strictly to the domain layer which (wrongly) implies the use of ActiveRecord. Don’t get hung up on this. Objects are powerful ways to encapsulate concepts. Don’t hesitate to use them.

You can use helpers as the “glue” between your view templates and these newly-emergent, sophisticated objects that are cleaning up your view. Again, we want to focus on the what, not the how.

So, perhaps a way to consider the evolution of view layer encapsulation is like so:

ERB in templates > Helper methods > first-class Objects

You need to use your past experience and your sense of smell to figure out when to evolve. Picking some complicated Object strategy up front is probably doomed to not meet future needs and will require even more re-work to replace it with something else.

Update: 2007-05-19 — Jason posted a PDF of his presentation here.

Jason Hoffman, CTO of Joyent, gave a (mostly) fascinating session on scalability of Ruby on Rails. Here’s the news-flash: Rails is only a tiny part of a large scale deployment. He started off the talk with a couple of points I really take to heart: large-scale apps are not achieved in their first iteration. They are realized in the fifteenth to twentieth iterations. Put another way, he said that “…the road to a top site on the internet is not from one iteration”. A premature decision to mold the architecture without a real pain point and test data to back it up is as likely (if not more likely) to send you down the path to hell that solve any real problem.

One of the first eye-openers in Jason’s talk was the consideration of power consumption. I’ll admit that this is a topic that I never considered, but according to the presenter it can fundamentally shape your physical deployment. In essence the intersection of cost and legal limits to power consumption in a physical space can limit what you can put in a rack more than anything else. In some cases co-location facilities may actually waste space because they simply can’t power any more machines than are occupying a fraction of their total space! In short, the goal is to cram as much computational power into as small a space using as little power as possible.

The middle section of the talk left my mind wandering as Jason rattled off performance metrics around the Solaris servers that they used. I got the sense that my mind wasn’t the only one wandering at this point.

After the break, the talk hit high-gear and finished with a bang. Jason began to get into the nitty-gritty deployment strategies that they used at Joyent. Now we were all sitting up straight in our chairs and paying attention.

First, they do a lot of Big IP work. Lots of layer 7 (application) routing rules to partition the work in all sorts of interesting ways–even load-balancing to mongrel that handle a single controller. Instead of putting mongrel instances behind Apache (which Jason claims maxes out to about 150 reqs/second with Apache and mod_proxy), they load-balance thousands of mongrel instances directly behind Big IP.

They don’t use plain old vanilla mongrel either. Instead they are using the new event-driven, non-threaded mongrel. In addition, they are running this inside of four virtual machines on a single box. A typical hardware configuration is a 16 GB RAM machine with 4 AMD CPUs. They run four virtual machines with ten mongrel instances each for a total of forty mongrel instances on a single box.

Why virtualization? According to Jason, as you consolidate computing power into a single box, you may miss the “sweet spot” of smaller hardware that most software has been written for. You can recapture that with virtualization. It’s a fascinating idea that I hadn’t really considered. Of course you wouldn’t dare do any of this without some real testing and numbers to back it up.

Speaking of testing, Jason had an interesting approach. Start with a single mongrel instance and test it with some benchmarking tool like httperf. Find out what a single mongrel instance can do in terms of requests per second. Get an idea of how much CPU and memory that single instance uses. Now multiply it by some number such that the total resources consumed by your mongrel instances won’t exceed your hardware. Now fire up that number of mongrel instances and benchmark each instance separately but simultaneously and monitor what your hardware is doing.

You want to see that adding another mongrel instance provides a proportional level of capacity. Once you’ve hit the limits of that piece of hardware you know what a single box should do. As Jason stated several times during the session, you want things to “add up”. That is, adding another 50 mongrel instances to the overall deployment should provide a commensurate level of capacity.

Now that you know what a single node can do, you should be able to put any number of them behind a front-end load balancer and get a total throughput equal to the capacity of a single node times the number of nodes. If you are getting significantly less total throughput than you would expect, you probably need to revisit or refine your front-end load-balancing solution.

This ties into another interesting point made during the presentation. Rails is just a small part of the overall application. Questions about whether the Rails stack will scale are somewhat silly since the Rails stack comprises only a small part of an overall application deployment.

Of course, some of those other parts are things like persistent storage. He brought up several examples of how Joyent has moved some responsibilities off of the shoulders of the RDBMS and pushed them to other technologies including LDAP, Memcache, message buses and even the file system. I’m firmly convinced that it’s pretty easy to max out the capabilities of RDBMS systems. They have a very hard job to do and adding lots of complicated clustering and scaling only make that job harder. Using a heterogeneous approach where certain kinds of data are stored via different kinds of mechanisms struck me as a very pragmatic approach.

One final note was about DNS partitioning. If caching provides a way to avoid expensive operations over and over again, partitioning is way to spread operations across your hardware. Jason was a big fan of using DNS and Big IP routing to spread load across hardware.

After a morning spent hopped up on cold-medicine and two cups of coffee (how else do you think I keep my girlish figure) I hope I can keep my stamina up for the afternoon session and BOFs tonight. Somehow I think the combination of readily-available caffeinated beverages and major geek convergence going on at the Portland Convention Center will carry me through.

Next post: a review of the afternoon tutorial: “When V is for Vexing: Patterns to DRY Up Your Views”