Chris' recent article has stirred up quite a debate (see comments) about Rails and whether you'd have to be drunk to suggest using it. I'd like to expand a bit on Chris' point and talk a little more about how we approach technology choices and where/why we use Rails. This article's working title was "Why Rails (Mostly) Reigns Supreme (at Pivotal)"

Personally, I'm very suspicious of any argument that says one technology will dominate all others. It's like saying that eventually the only tool anyone in their right mind will use is a 3/4" socket wrench. I don't believe this is Chris' argument, but it was clearly taken that way.

At Pivotal we use a number of technologies, and in fact most web projects involve at least four languages. We generally use Ruby (though not always Rails, for general server-side web work, Java (Lucene, etc), extensive JS in the browser (often more lines of code than the Ruby), and C bits (Ruby C libraries, mongrel, etc) where performance matters. As well, we aggressively take advantage of Amazon's web services (ec2, s3, and sqs). I have some knowledge of their implementations, but realistically it doesn't matter.

The unifying theme here is Business Value. We choose technologies from our toolbox because they maximize the value we provide our clients. If Ruby/Rails doesn't maximize business value, for example when a client already has 50 expert Java developers AND a million lines of Java invested in a "Platform", we don't use it.

Today a large part of Pivotal's business (the part I'm involved in) is building web-based applications for early-stage start-ups. These companies in turn are trying to prove out novel business concepts. They don't care about which language we use. They care about putting good software in front of users quickly so that they can make any necessary course corrections and get their business off the ground before the money runs out. This is where Ruby/Rails comes in [1].

As Agilistas we've always delivered software early and often. But, Ruby/Rails gives us the ability to deliver much more, much more quickly. As Chris points out, this makes it cheaper to experiment, and therefor more likely that these businesses will succeed.

To borrow a Perlism, there is generally more than one way to do it. Parker's corollary is that there's often a best way. Having built quite a few applications in Ruby/Rails, our experience is that it is the best tool for the job it does.

[1] There's are other arguments to be made about process, business value, Rails, and the relationship between, but we'll save that for another post.

This week I had the pleasure of attending Dapper Camp put on by the folks at Dapper for their user community. Mitch Kapor kicked it off with a talk on disruptive technology, openness, and innovation. We then got to hear both from the Dapper team about new and upcoming features, and folks like Aaron Fulkerson of MindTouch about using Dapper to repurpose data. All around it was pretty interesting.

What I love about Dapper is that it helps solve one of the big issues I see our clients have: data. We can build just about anything, but if an application needs some specific data (and many do), products must be launched with sub-par (but available) data, or worse launches can end up being delayed. In many cases, we can end up spending a large amount of time (aka money) getting/munging data rather than developing features. Note: I also think the ease if pushing data out of apps via instant gadgets makes Dapper very interesting but that's a whole separate post.

The "Dapp Factory," a Rhino-based server application and a web front-end that deals with just about any site by proxying your requests and modeling the DOM on the proxy, then recording your actions for later replay. But, their secret sauce is a super-cool algorithm that figures out the structure of pages in such a way that your API can withstand changes to the target site, making your feed resilient to all but massive site overhauls. You then simply consume an XML or JSON feed, or use a simple API to dynamically construct paramaterized feeds.

There are other companies trying to make data less painful. Metaweb, for example, provides an incredibly fast graph engine and relational schemas (think RDF) that makes real-time use interesting. But, if the data you need isn't in Freebase (a likelihood until they get larger), or your data is continually being updated, you will still be stuck scraping and relating the data, and that's generally where most of the work is.

Take as a small example Dav's awesome Vacation Planner. The concept is simple, the feature set is small, but getting the data is a pain (see article). Some sites don't have APIs, and those that do provide unstansardized, sometimes buggy, and are often often missing the data you need.

I could imagine writing a dapp parser akin to ActiveResource pretty easily (I hear a ruby SDK came out of DapperCamp, but I can't find it). With a little more work, it would probably be easy to add cache_fu support, and ruby modules that could be mixed into models (for asynchronous data gathering) and controllers (for serve now vs. polling) to easily support Dav's polling mechanism.

This would leave Dav with pretty simple model (data) code, and the luxury of focussing on whether to add wikipedia integration for population figures or the Big Mac Index, rather than tweaking his Mechanize xpaths all weekend. I vote for the Big Mac Index.

So, the next time someone suggests you screen scrape to get that data you need, tell them to give Dapper a shot. And if anyone wants to write 'ActiveDapp' let me know. It could be really fun.

Note: In the spirit of full disclosure, Jon Aizen (Dapper CTO) is a friend of mine and they gave me a free t-shirt...and a sandwich (thanks).

As geeks we like science experiments. As San Franciscans (mostly) we like anything novel, particularly if it's organic, healthy, and foreign. So, a few months ago, after realizing our employer couldn't afford our kombucha (fermented green or black tea) habit, we started brewing our own:

Now we've got more "mushrooms" (actually a bacteria and yeast symbiote called a scoby). These are the "strata" in the jars above. So, if ever you've ever wanted to make your own kombucha (it's easy and tasty) , or if you've never heard of it and the idea of a beverage somewhere between ice tea, lemonade, and beer sounds tasty, come by or paypal me postage and I'll give you a scoby.