Stop The Bug-Based Development

I enjoyed reading Phil Haack's little "you're either with us or against us" diatribe from earlier today, where he rails against non-test-driven development. I only differ in that I prefer to call it Bug-Based Development (BBD).

So are you one of those fifth-column types who is providing aid and comfort to the enemy? Draw a line in the sand... never make your troops take casualties retaking the same ground twice... no bastard ever won a war by dying for his country. You won it by making the other poor dumb bastard die for his country.

Give 'em hell, Phil!

Weighing In On The Upcoming Ruby VM Smackdown

Earlier today, I read with great interest the potentially inflammatory post where Ola Bini proclaims the Ruby VM wars already over, and basically divides the Ruby VM empire into two halves, with the center controlled by Rome, err, I mean Rubinius primitives.

I had already come to this conclusion a while ago. Not because I am smarter than anyone, FAR from it. Probably because I am more superficial, and easily impressed by meaningless characteristics, like how cool is the name Rubinius?

Seriously, back to something more erudite, like Ola's post. Notably missing from this future of Ruby utopia is any IronRuby/Ruby.NET vision. Is this a simple oversight, due to the pro-Sun anti-MS bias that any Java programmer has to have, or more likely knowing the source, is it because of the sadly glacial progress of Lam and his new friends, as compared to both Rubinius and especially JRuby?

Man, it must SUCK to work at Microsoft sometimes!

RubyConf 2007 => Ruby VM Smackdown

I just registered for RubyConf 2007, which will be held Nov. 2-4 in Charlotte, NC. If you want a more intimate, more hard-core Rubyist experience, than what RailsConf is turning into, RubyConf looks to be it.

There not being that many tickets available, you better register right now if you want to go. I registered without even looking at the agenda first... just filled out the registration form, and hoped it's was not too late.

Once I has nervously typed my credit card info, and gotten my email confirmation, I took a few breaths. and got around to checking out the schedule for the conference I had just committed to attending. I was not disappointed.

Lo and behold, the ultimate battle of the Ruby VM's will be taking place on Saturday, Nov. 3. Take a peek at what's in store:

9:00A - John Lam - State of IronRuby
10:00A - Charlie Nutter - JRuby: Ruby for the JVM
11:00A - Evan Phoenix - Rubinius 1.0

Wow! If you care at all about the future of Ruby as a language, not just as a vehicle for fevered dreams of Ruby on Rails-based world-domination, you can't help but get excited. The three ascendants to the throne of dynamic language glory, all duking it out in a back-to-back 3-hour frenzy. Yum!

Improving the language performance of Ruby is the big thing needed to push out all those other wanna-be next big languages into the dustbin of interesting languages that no one uses. I think Erlang is really neat, don't get me wrong, but one look at that syntax, and sugar is not what comes to mind!

Visualize This

Did you ever want to present information in a visual format, but didn't know which one was most appropriate? Don't know a Petri net from a concept fan? Or perhaps you just need even more visual stimulus from your information?

Well, welcome to a feast of visualization splendor. Just have a glance at the Periodic Table of Visualization Methods... your eyes will never go hungry again.

Thanks to Visual Literacy for creating this great info, and to Jeff Atwood of Coding Horror for blogging about it.

Lord Raise Me Up

As usual, human ingenuity has raised science to a new height. In this case, a very small height, but let us not quibble about a few details. UK researchers have discovered a way to circumvent the Casimir force, which is what causes very small things to clump together. By using a clever type of lens, the researchers were able to reverse the Casimir force in their experiment.

Despite the fun headlines, this does not mean repulsors for the masses. Any practical application would likely be amazing new manufacturing techniques for nano-tech materials, and new types of nano-devices, not flying cars at this point. More like nano-engines that are more efficient due to reducing friction by "floating" the components of the device together.

At the very least, this should spawn a new outbreak of copycat perpetual motion machines and the like...

Fun At Yahoo! In Los Angeles

Last Tuesday night was July's Los Angeles Web Application Developer Meetup. This event is starting to get really good. Not only were there lots of people who came out, but our hosts Yahoo! actually bought us beer too. Hint to future hosts of the meetup... be cool like Yahoo! and provide BOTH a projection screen and beer.

There were two especially fun presentations, at least to me. Jim Bumgardner of Yahoo showed off some of his really interesting web gadgetry. Now I didn't know Jim when I walked into the meetup, but I have seen his work. The man has done some innovative stuff over the years, that is for sure. Also, he has to have one of the best gigs I have seen for a while, getting paid to build really fun things.

He showed several different ways to choose one item from a group of items, without resorting to any of the usual tired UI metaphors. My personal favorite was the Wheel of Lunch.

The other extra fun presentation was Richard Herrera showing Pickleview, an iPhone application created to track real-time baseball scores. Now if they were soccer scores, I would have had to run out and get an iPhone... actually, I may just end up doing that anyhow. But I digress.

Developed in PHP (but we won't hold that against them, will you, dear reader?), Pickleview combines an XML feed of Major League Baseball scores, and mashes it up with Twitter. A pretty cool little application, I would have to say.

Now I am getting interested in whipping up something for the iPhone myself... stay tuned.

Drunk In Space

Apparently, the altitude in orbit is not high enough for more than one of NASA's finest. According to an independent panel, on at least two occasions astronauts were allowed to fly after flight surgeons or other astronauts warned that they were drunk, so drunk that they posed a flight safety risk.

You cannot have a machinist fabricating spacecraft parts who is unable to urinate in a little cup, but a DUI is space is not a problem that NASA is concerned with? I guess the sky IS no longer the limit...

An alternate view is as follows: the astronauts know how poorly designed the shuttle is, and need to throw back a few stiff ones to get back in the saddle. These are likely the more experienced ones who have actually been up a few times, and know the risks. Perhaps they were colleagues with the crews of the Challenger or Columbia. We shouldn't judge too quickly what it takes to do that job, as much as many of us many have fantasized about a journey into space. But they should still need to pass a breathalyser before they fly a 1.7 billion dollar spacecraft.

IronRuby Lives!

John Lam and company have finally gotten something together to release to the world at large for their IronRuby project. I have not had a chance to play with it yet, but as a hard-core Rubyist I will, of course, have to do so.

A few interesting observations from how they have chosen to go about developing this project. One is that they are distributing the source code, albeit as part of the Microsoft Permissive License. I don't really know much about this MS-license, I will have to go find out how permissive it really is.

Another interesting point, is that the IronRuby project will be managed through RubyForge, instead of CodePlex or another of the .NET open source communities. They really want to make an effort to reach out to the Ruby community, it would seem, by being in the old familiar places.

I look forward to playing with it...

Eventually The Machines Will Win

I have never been a big checkers fan. Good thing, cause I would have had to give up the game after the recent "solution" of the game by a team of Canadian computer scientists. Every possible move of every possible game has been mapped, and a checkers playing program can take each of these into account before deciding on its next move. No human player has such ability.

Actually, as it turns out, the best possible outcome in a game of checkers, given two players who each make perfect moves, is a draw.

The next game to be solved will likely be othello, than other even more complex games like chess, which has 10²⁰ more possible moves than checkers.

It is only a matter of time before sufficient computing power allows the machines to contemplate eventualities of which we have not even postulated the existence.

The Singularity IS coming...

Ruby Is More Of A Free Love Kind Of Language

I have been really, really busy lately, which is why I have not had a chance to post anything. I finally caught up on some reading and listening, and John Lam, father of the IronRuby project at Microsoft, has a fun interview on the DotNetRocks podcast. Irrespective of how I may currently feel about the direction over at Microsoft or rock, it is an interesting interview.

My favorite line has to be where John is describing the cultural differences between the users of Python vs. Ruby programming languages. "Ruby is more of a free love kind of language," he says.

I guess when they say that they have drunk the Ruby kool-aid over at Microsoft, they really mean it. But will they wake up hungover and pregnant with a baby they will name "Moonbeam"?

CruiseControl.rb and RCov Are SO Good Together

I spent a bit of time this morning getting CruiseControl.rb and RCov working together for a new client's fairly LARGE Ruby on Rails (70+ models, 40+ controllers) project. If you are not familiar with CruiseControl.rb, it is the Ruby based continuous integration tool from ThoughtWorks. RCov, on the other hand, is a very well known code coverage tool for Ruby. The implementation was incredibly simple, but finding the information on how to get the two working together required a perusal of the CruiseControl.rb code itself.

First of all, you will need to install the rails_rcov plugin. This useful plugin allows you to really easily run RCov using rake, and is essential to simple RCov integration with your CruiseControl.rb builds. You can install rails_rcov like this:

./script/plugin install http://svn.codahale.com/rails_rcov

Use the "-x" option if you want to use an svn:external link to the latest version of the plugin. I personally don't like to do that...I want control over when the software I am using gets updated, thanks.

Once the plugin is installed into your project you have some useful rake tasks like this one:

# sort by coverage
rake test:units:rcov RCOV_PARAMS="--sort=coverage"

Now you are ready for the final bit, which is adding a "cruise.rake" task to the "lib/tasks" directory of your Ruby on Rails project. The task should have something like this, which was lifted verbatim from the CruiseControl.rb build of CruiseControl.rb itself (that all sounds kinda twisted, but it is cool):

desc 'Continuous build target'
task :cruise do
  out = ENV['CC_BUILD_ARTIFACTS']
  mkdir_p out unless File.directory? out if out

  ENV['SHOW_ONLY'] = 'models,lib,helpers'
  Rake::Task["test:units:rcov"].invoke
  mv 'coverage/units', "#{out}/unit test coverage" if out
  
  ENV['SHOW_ONLY'] = 'controllers'
  Rake::Task["test:functionals:rcov"].invoke
  mv 'coverage/functionals', "#{out}/functional test coverage" if out
  
  Rake::Task["test:integration"].invoke
end

That's it. Now you have incredible code coverage goodness right within your build. You can even browse the source within the browser window of the CruiseControl.rb dashboard and see which lines of code were covered by your tests, and which were not...

So if you have a Ruby on Rails project, with a project team of greater than 1 developer, you had better start using CruiseControl.rb. It is too easy, and the payoff of doing continuous integration is huge!

A Little Bit Of Merb In Los Angeles

Last night, I gave a short presentation on Ezra Z's cool Ruby web development framework Merb at the Los Angeles Web Developer Meetup. Microsoft was kind enough to provide space at their new downtown offices, and even buy pizza. Can you say "(munch munch) Silver (crunch) light (munch) launch (gulp) budget"?

When I started my presentation, I asked the room how many people were using Ruby on Rails. Almost everyone in the place raised their hand. Nice! My talk was a brief review of what Merb is, and some of the similarities and differences with Rails. The crowd was very attentive, and there were some good followup questions asked afterward.

Anyone who is interested in the slides from my presentation, they are available here.

There were also presentations by Ari Lerner about Ruby metaprogramming, and Ben Sandofsky on the Seaside web framework. I especially liked the example that Ben used to illustrate continuations.

Lastly was Woody Pewitt, the Microsoftie tasked with demoing Silverlight. It was funny how long it took their poor projector to recover when he plugged in his Vista notebook to show his demo, given that every other presenter was using a Mac. Woody was a good sport about it, and there really is a lot of interest in Silverlight right now, so I saw a number of people talking to him after the demo.

Anyhow, a good time was had by all, and I am looking forward to the next one. Here in L.A. we have a thriving little tech scene of our own, albeit not as huge as the bay. But don't count us out!

Without Air, No One Can Hear You Scream

I was shocked to read about the ongoing problems with the life support computers onboard the International Space Station. Every time I hear of problems up there, I feel a serious rant coming on.

I grew up on science fiction books. In fact, I would have to say that the vast majority of my childhood was spent far away from this planet. Suffice to say, I am a serious supporter of space exploration.

But this madness with the ISS is about more than I can take. Between the noise, the need to keep reboosting its orbit, and the lack of any real scientific value, we really SHOULD abandon ship at this point!

Microsoft Vs. The Galaxy

"The more you tighten your grip...the more star systems will slip through your fingers..." - Princess Leia

Power Rangers Should Test First

Last night, I got home a bit late. My 8 year old son was still up, watching the Power Rangers. "Sit down and watch with me, Dad," he said. "I'm surprised you still like the Power Rangers," I said, as I sat beside him on the sofa. "I mostly just like the vehicles," he told me.

The Power Rangers were in trouble in this episode. The bad guys had taken remote control of the Rangers' giant robots, and the Power Rangers were helpless without their usual firepower. The Smart Rangers, meaning the guy with the glasses and the Asian girl (WTF!!!), were constructing a jamming device to block the bad guy's signal, while the other Rangers got their behinds handed to them by their own robots.

"OK, that finishes it. Let's go!" exclaimed Glasses Ranger as he attached the last part to the jamming box. They rushed off to the scene of the battle. "I hope this works," said Glasses Ranger and pushed the button. Nothing happened. "Oh no, what could be wrong?!" cried out the Smart Rangers.

My son, who had been quietly watching all of this, suddenly spoke out. "They're stupid. They should have tested it first."

Ruby/Microsoft?

Martin Fowler just posted on his bliki a little piece entitled RubyMicrosoft. As I read it, I felt a strange chill run down my spine. Perhaps a few words of explanation are needed...

At RailsConf recently I stood at the Thoughtworks booth for a moment chatting up Fowler, and embarrassing myself by asking for a photo with him (he doesn't pose for photos with people). Luckily, I was rescued by the arrival of Oli Bini, and an ensuing short discussion about JRuby. I expressed my admiration about how they had been "cleaning Microsoft's clock" as far as their much faster progress with a runtime for the Ruby language then Microsoft's efforts with the DLR.

Much to my surprise, Martin was actually paying attention to what we were saying (I mean to what I was saying, after all he knows and speaks with Ola), and chimed in about how Thoughtworks was working with Microsoft on language compatibility for IronRuby. I expressed a poorly articulated skepticism at the prospects.

Lo and behold, the RubyMicrosoft blog posting sums up rather well the half-formed mumblings I was trying to stammer out. Well, this is why he is a keynote speaker, and I am a dead programmer. However, impressed as I am at his intellect, I am not sure I share his optimism about Microsoft.

I certainly don't know if I am an "alpha geek" or anything like that. I'd rather be a jazz programmer than a rock star. But given my long history working with Microsoft technology, and then the seemingly contradictory fact that I was at both RailsConfs, I must really identify with the movement. Chalk it up as just one more developer who Microsoft is losing to the rebellion.

I Would Rather Be A Jazz Programmer

I have reading a bit lately about "rockstar" programmers. Recruiter ads are proclaiming "Rockstar programmer needed". Various websites named rockstar<name of technology> or alternately <name of technology>rockstars are springing up everywhere.

I would rather be a "jazz musician" programmer, myself. Nothing against rock, don't get me wrong. Glam, punk, metal, I can go there. I frequently do. It's the "star" part, that has been starting to bother me.

Here are some differences, as I see them:

Rockstar
- One big hit song, then disappears
- Embarrass themselves as they age
- Claims they wrote the song
- Keeps trying to get back that sound they used to have
- Gets back together with the old band after unsuccessful solo careers
- Wants to marry a model and have a movie cameo
- Won't play without a contract and advance payment

Jazzer
- One big hit, and they become an influence
- Get cooler with age
- Claims the song is just a cool arrangement of a standard
- Keeps trying to produce a new sound
- Records with a variety of musicians over time
- Wants to become a professor at Berkeley School of Music
- Jams on the street corner just because they feel like it

RailsConf 2007 - Day 3

By the time we got to RailsConf Day 3, only the most hard core remained. In other words, almost everyone. And we were glad to have stuck around, because some of the best content of the entire conference was yet to come. David Black, introduced two integral members of the Ruby on Rails team, Jamis Buck and Michael "Koz" Koziarski, the co-authors of the seminal "Rails Way" blog.

The Rails Way
Jamis and Koz and both written and read a lot of Ruby on Rails code. The Rails Way is not just what could be done, but what should be; the best way to build a Rails application. Their knowledge of best practices is second to no one, and they shared a whole bunch of great information. I can't really do their tag team code review justice, but here are the highlights:


Fat controller is anti-pattern
*hard to test
*break apart into a model object

A model is anything that encapsulates data and logic, not just an object that talks to a database.
*one benefit of this encapsulation is testing
*another advantage is the form_for handle can easily use a model

When to use with_scope
*move the logic to the parent class, so as to freely get the scoping with no extra work
*do not go crazy with DRY

ActiveSupport has helpers to make some code self-documenting for dates/times

Asociations
*People make associations, but then fail to use them throughout the rest of the application
*this helps keep code from breaking in the case where database keys change, for example

Cargo culting
*He hates !! (not-not) idiom.
*Ternary operator...likes it sometimes
*Explain exactly why you need boolean returned, again?

Model helpers
*use associations in views instead of instance variables

Routing
*simplifying non restful routes - map.with_options

Override to_param to make nice looking URL

Returning method

 returning new do |booking|
  booking.some_thing
  booking.some_other_thing
 end

The session was really great. Pretty much everyone in the room, no matter how advanced their level, got something good out of it. We could have stayed there all day, with different people bringing up pieces of code for these guys to refactor. If you can, hire them.

Javascript - Fu
Dan Webb is the author of the LowPro extensions to the Prototype JavaScript library. He had a really nice tight Takahashi style presentation. He was also extremely funny, and the information incredibly useful. Once again, the last day of RailsConf really had a lot of good stuff!

JavaScript is the language we all love to hate. A peasant language, meaning that "lower-level" programmer have to work on the JavaScript code while "higher-level" ones get to work on the "architecture".

JavaScript - fu is not easy to master. Developers are forced into refuge by using frameworks and libraries.

The Ancient Manuals of JavaScript-Fu
* The Tao Of The Event Handler (Events)
* Five Method of DOM Fist (DOM)
* Lightning Script Style (Optimization)
* Iron AJAX Technique (Progressive enhancement)

There are big differences in browser implementation of JavaScript.

Two main techniques: inline vs. scripted event handlers

Inline - applied as soon as browser loads

What happen when more than a little bit of inline code

Scripted event handlers
*problem in they attach after the element is loaded

LowPro library is an extension library for Prototype, but Prototype library trunk now has a similar thing.

// Prototype 1.5+ and LowPro
Event.onReady(function() {
  $('item').observe('click', function() {...});
});

// Prototype trunk
Event.observe(window, 'DOMContentLoaded', function() {
  $('item').observe('click', function() {...});
});

// Basic onload
Event.observe(window, 'load', function() {
  $('item').observe('click', function() {...});
});

Attach handler from script to each object you want, in order to stay DRY

Separate your JavaScript out of your pages, just like you separate CSS

Large number of event handlers can choke browser performance

How:
* use script based by default
* in bug page try event delegation
* if all else fails use inline

Event delegation - yahoo tehnique
* event bubbling

Better inline handlers
*a tag needs an actual HREF
*do as little as possible, for example:

 return view(this);
 function delete(elemtn)
 {
  new Ajax.Request(element.href), {
   method: 'delete'
  });

  return false ;
 }

5 methods of DOM fist
*3 oficial W3C methods to modify the DOM
- appendChild
- insertBefore
- replaceChild
*1 non-standard (from IE)
- innerHTML

DOM methods insert with precision - have to create nodes first
InnerHTML shifts large bulk amounts of HTML, but with little control

Really easy to use innerhtml from Rails with Ajax

The bastard son - unholy marriage of DOM methods and HTML

Element.fromHTML = function(html) {
  var root = document.createElement('div');
  root.innerHTML = html;
  return root.firstChild;
}

var node = Element.fromHTML('<div>');
element.insertBefore(thing, node);

Lightning Script Style
* making things as fast as possible
* do not use javascript_include_tag :defaults cause it downloads 5 big files
* browsers take time both to download AND evaluate a script
* the less JS the better
* mooeffects smaller than scriptaculous, but less powerful
* browsers can only load 2 resource at once
* combine js files
* Use gzip instead of JS based minification.

Make sure everything is cacheable

Faster loops

 for(var i - 0, enemy;
  enemy = enemies[i]; i++) {
 defend(enemy);
  }

Be careful with selectors
$$('.anything'); slower
$$('span.anything'); slow
$$('#item .anything'); // css xpath selector

Iron Ajax Technique
* rule #1: browsers suck
* main browsers are getting better quickly
* but what about the others?
* Corporate security blocks Javascript at firewall
* the traditional rails opin: f* you!
* but why turn away users if you don't have to

Progressive Enhancement
1. start with plain HTML
2. test if necessary brwt features are there (XMLHTTPreq, canvas, etc)
3. If present, then apply xtra JS features

Tt's easy to apply to ajax
* HTML
* JS
* RJS

Try progressinve enhancement first

Lastly, Dan recommended a few books on Javascript he said are better than all the others:
* Professional JavaScript Techniques
* Bulletproof Ajax
* DTHML Utopia: Modern Web Design Using Javascript and DOM

Dan's session was full of good stuff. Even the people who thought they knew it all were scribbling down a couple things before the end. And thank you, Dan, for making it really fun as well as informative.

Dave Thomas
Dave's speech was unconventional, in that he doesn't say what people expect him to. I consider that a good thing. Many people in the crowd were perplexed for long moments as he spoke. For those people who need a simple explanation of Dave's speech, the two recurring themes for RailsConf were "give", and "don't get stuck".

"Just to set the record straight, the first RubyConf was 3 people and a dog, and the speaking was in between sips of beer"

Orthodoxy - do what is expected
Orthopraxy - do the "right thing"

It's called Object oriented programming, not class oriented programming. Challenge: write next next program with objects instead of classes

What Are Your Cargo Cults?

Dave is a real thinker, and I hope that our community can rise to the challenge and keep progressing. Embrace Orthopraxy!

RailsConf 2007 - Day 2

After the incredible energy of RailsConf Day 1, Day 2 seemed to slow down the pace a little bit. That was good for me, as I could spend some time catching up with friends both old and new, and not just attending sessions like an information junky. First would come the morning keynote speeches from Cyndi Mitchell of ThoughtWorks, and headliner Tim Bray of Sun Microsystems, with a short interlude from Charles Nutter.

Cyndi Mitchell - Thoughtworks

Ms. Mitchell is UK Managing Director of ThoughtWorks. As such, she is a polished corporate representative, which is exactly the kind of emissary that Rails needs in big organizations. She had a few interesting points:

* 40% of new work at ThoughtWorks is coming from Ruby on Rails in the enterprise
* We need to reclaim the word "enterprise" back from meaning "bloated, inefficient, and expensive" to its original meaning of "entrepreneurial"
* The "real" enterprises are using Ruby on Rails to get stuff to market quickly
* ThoughtWorks is offering a complete Rails production stack called RubyWorks, and is will be providing paid 24/7 support for Rails.

Tim Bray - Sun Microsystems
After Cyndi, the irrepressible Tim Bray took the stage. Despite not really being that much into Java, I have been following Tim's blog for quite some time, as he is a smart thinker with plenty to say to the industry at large.

Tim thinks that the Ruby language is as important as Rails (I agree!!). With JRuby, Sun wants to sell lots of hardware to large enterprises that will be choosing Sun Hardware and Solaris as the preferred platform for their Rails applications. Sun is also doing lots of work on NetBeans to do better Rails development.

Tim then briefly brought up Charles Nutter to give him some Rails street cred...er, I mean to explain why Ruby developers should care about JRuby.

Charles made two main points:
* A whole different way to do Ruby dev...using tried and true Java tools that already exist
* Move into enterprise where Java is accepted but Rails is still not

Then back to Tim:
It is not CIO's who lead industry...it is people like us.

How to make money on free products
* first you get adoption
* then you get deployments
* then you can monetize at point of value - no large business will adopt any tech that is not supported

Big companies will either get a handle on web 2.0 or get disintermediated

He assumes Rails will succeed beyond our wildest dreams.

Java, php, and .NET will never go away

The network is the computer...and its a heterogeneous network. And JRuby is how we get everything to talk to each other.

Java really consist of three main parts: language/JVM/Java APIs.
JRuby lets Rails programmers access the latter two.

How do we get everything to talk to each other? REST

Etags are a way to solve the REST problem with different versions of the same data.

ATOM - Everything should have a publish button: cameras, phones, browsers, etc

Is REST all there is? SOAP... WS-* 36 specs, ~1000 pages of documentation.

WSIT - interop with MS WCF

Hot developer issues
*static vs dynamic languages
*maintenance
*tooling
*integration
*concurrency
*time to market
*scaling
*load balancing
*observability
*file IO
*DBMS
*Shared nothing architectures
*CPU
*concurrency
*debugging
*unit test
*functional programming?

Which matter the most?
*Conventional wisdom for each language
- Java: performance, tooling
- php: web scaling, ease of use
- Rails - time to market, maintainability
*Tim says: The most important are Time to market, and maintainability

Extra Action Marching Band
After the speeches, I went to a session I didn't find that interesting at all, and so in the interest of not offending anyone else too badly, I shall let it remain anonymous...you know who you were. But near the end, a massive blast of sound came from the entrance to the conference center, and then proceeded past the sessions rooms toward the main hall.

It was the Extra Action Marching Band, an amazing avant-garde performance troupe. If San Francisco (which is where they are from) had a love-child with New Orleans, it would be something like this band.

Despite an almost-incident with the conference center security staff, they were able to finish their set. The music and performance were quite incredible, showing a high level of musicality as well as serious creativity. The bullhorn with a delay pedal attached was really amazing sounding...I have to make one like that for my harmonica.

After this unique musical interlude, it was back to the conference sessions...thamks to Tim Bray for the photograph.


Xen And The Art of Deployment
Ezra Zygmuntowicz of Engine Yard, Merb, and BackgrounDRb fame gave my favorite presentation of the day. The session was packed, with both information and people. If you wanted information about scalability, at least from the perspective of the stack, then this was the high point of the entire conference. Here are copious notes of what he said:

Serving up Rails right now is all about Mongrel

Mongral uses a Ragel state machine for validation

Why is mongrel better:
*HTTP is well known protocol
*Mongrel easy to setup and use
*Transparent wire protocol

Thread safety
*Rails not thread safe
*1 request at a time per mongrel

New dog, old tricks
*Wide # of options to load balance front end
*pen, pound, HA-proxy
*Lightspeed can serve static files
*Apache 2.2/mod_proxy_balancer can do same

Each current solution has problems

Nginx
*serious performance
*small resource footprint
*no leaks under heavy load
*killer rewrite and proxy mods
*author is cool and responsive
*Nginx + Mongrel
*This is THE stack
*apache for mod_dav_svn
*flexible config allows both serving static and dynamic requests
*fast!!!
*Nginx cannot do upload progress due to buffering
*no connection rate limiting for proxy modules yet

The future for nginx
*mod_rewite going away
*replace with http_script_module
*embed NekoVM directly into nginx

Perfect simple stack
*Linux
*Nginx
*Mongrel (mongrel_cluster)
*Monit

Swiftiply - the cool new solution

Evented Mongrel
*hot patch
*remove ruby treeads
*replace with eventmachine event loop
*mongrel single threaded
*big IO speed increase
*stand up to higher load

How is single threaded faster?
*Ruby green threads are slow
*mutex locks are expensive
*one process can only do so much IO
*event driven means running in a tight loop and firing callbacks
*without context switching between threads, single process has less overhead

Mongrel vs evented mongrel - performance stats
*1 user - 1K more req/sec
*100 users - 758 req/sec vs. 3700+

Swiftiply proxy
*event drive proxy, small memory footprint
*faster than ha-proxy

How is swiftiply different
*In a normal proxy, you need to config each back end server in advance before starting proxy
*swiftiply is a client/server architecture, where new back end servers register with proxy
*keeps persistent connection between proxy and back end servers
*auto-configuration of proxy

This means you can start/stop as many mongrels as you want, and they get configured into proxy, opening the door to auto-scaling

The Zen of Xen
*monolithic linux VS modular lnux
*old model: one box running all services
*new: specific virtual machines setup for specific purpose

We all want to modularize application why not modularize servers

What about more than 1 box?

Old school - more boxes
*mysql box
*another box for other services

New school
*add another xen box
*pick resource intensive parts, and more to another VM
*each VM only runs one thing, but well
*target performance problems
*scale better

Advanced clustering
*boot off dom0 off of USB drives
*SAN for all Xen domU
*red hat clustering suite for fencing and cluster options
*GFS for 100% posix file sys
*hardware load balancer, or dedicated box running Ultra Money or LVS

Create a fabric of compute and storage
*If a node fails, just swap in another
*Move hot VM's to less loaded machines
*Deploy app on 1 node, then bounce mongrels with GFS
*Fragment and page cache across nodes
*Scale from 1 or 2 VMs to as many as traffic requires and then back down again as traffic subsides

RAM RAM RAM
*most rails apps are RAM bound before they become CPU bound
*Avg mongrel serves 70-120 mMP PER mongrel
*RMagick is a typical RAM hog
*:include on active record also very bad performance
*95% of rails apps will leak memory at point or another

Use Rails plugin leakhouse to find leaks

Rails eats DB resources for breakfast

Most people's production apps hav no database indexes! Learn where and when to apply indexes.

ActiveRecord insulates developers so much to the point of massive inefficiencies.

You will need to denormalize data and use custom SQL statements to get performance

Other Tips
*do not use file system sessions
*script/runner is inefficient. do not load rails into bgprocesses.
*Use raw MySQL and Ruby without Rails if you can
*DO NOT use script/runner to pocess incoming email. Run deamon in loop and poll svr with net/pop3 or net/imap

Rails is great for 80/20 rule, but you are on your own for last 20%

Learn how to write custom mongrel handlers

When is optimization NOT premature?

Ruby is plenty fast, its Rails that is on slow side

Cache, cache, cache espcially static HTML files

Do not store full ActiveRecord object in session, just store hash to recreate it

Parting thoughts
*do not take anyone's word as gospel
*test and benchmark for yourself
*trust but verify

The Rest Of The Day
After Ezra'a session he was swarmed like a rock star. I know, cause I was standing nearby having a really nice chat with Zed Shaw the creator of Mongrel and Evan Phoenix, the creator of the Rubini.us Ruby runtime I was mentioning yesterday. You go, Ezra!

Later on, I got to hang out at the Engine Yard booth with Ezra for a few minutes, where Mike Brown and some other brilliant Kiwi whose name I didn't catch were chatting him up. I learned even more about hard-core clustering from listening to those guys go off...all I can say is I'm glad I have hosting at Engine Yard!

I wanted to go to Dr. Nic's RejectConf, but so many people told me they planned on going that I figured I wouldn't get in. I couldn't take the prospect of being rejected twice at the same conference, so I made other plans which turned out to be really fun.

After dinner with the same group of RailsConf friends from last year, with more delicious Portland beer, and excellent conversation, I headed to the BoF sessions, to hear about Amazon's EC2 service. I really wanted to learn more hard-core details about it, but so many people were just hearing about it for the first time, that the session was dominated by introductory questions. Oh, well!

I was about ready to turn in. Lucky for me, Rails Machine was having their party at my hotel. Bradley and company are really smart, cool people, and also offer a fantastic service, so I was glad to hang out with them for a few minutes. The most fun was jamming on harmonica with Joey deVilla the AccordianGuy. After hearing him last year, I had brought a couple of my harps with me just in case, and we finally had a chance to bust out a couple of songs together. That was really fun! Thanks, Joey. If anyone has any photos or video, please post it somewhere...if not, well, you had to be there.

RailsConf 2007 - Day 1

I was primed and ready for RailsConf 2007 Day 1. The excitement was in the air, and the crowd was primed. We awaited a day absolutely packed with mind candy. Starting with Chad Fowler, David Heinemeier Hansson, Robert Martin, Adam Keys, Avi Bryant, and Ze Frank. Wow, what a lineup!

As Chad Fowler took the stage, his ukulele in hand, we eagerly awaited his opening remarks. Chad really did a great thing, by not just giving us the "feel-good rah-rah Rails is taking over the world" intro speech that most people probably expected. Instead, he took a higher path. The virtues of sharing are not limited to open source projects, but need to get into the real world. For example, using Rails to create applications that solve real problems that have not been tackled yet, due to the high cost of developing a solution. We as a community can also do a lot for the world outside of just developing software, no matter how cool it is. The Pragmatic Programmer fund-raising drive is a fantastic example of mobilizing the community, and Chad exhorted the crowd to keep those donations coming.

Is the Rails community more concerned with altruism and spirituality than other tech people? Or are we following the once again popular meme of greater social awareness and activism? That may be fodder for another posting some other time...back to the show.

His opening remarks finished, Chad played a little uke ditty, under a delightful beat poetry introduction given by Rich Kilmer for David Heinemeier Hansson. As David took the stage, we all wondered, "what would he spring on us this year? We just spent the last year RESTifying our applications, what MORE do you want from us?!"

And here is what he said:


Keynote - David Heinemeier Hansson

REST is where it is at, and Rails 2.0 doesn't alter that path. But there are 9 other things he really likes about Rails 2.0.

* Debugging with breakpoints
David showed the ability to put a breakpoint into your Rails controller that opened irb with all of the context and objects ready to be introspected. Although dropping to command line is not exactly what most of us want out of a debugging session, this could easily have a superior UI put on it.

* HTTP Performance
HTTP performance in an area where a couple of small changes can dramatically increase performance. His examples showed very easy combining of all CSS files, and separately all JS files together into a single file, and then compressing of each group of files into a gzip archive for quick download.

Another HTTP performance technique that 2.0 introduces is a way to get around the two connection limit that browsers enforce, by serving static assets from multiple host servers using the new :asset_hosts configuration option.

* Query Cache
Rails 2.0 can look at all of the SQL that ActiveRecord is executing, and if there have not been any inserts or updates, automatically caches that database query.

* Rendering
Rendering of a resource request has now been separated from the MIME type of the request. Now, it is much easier to handle multiple format requests like RSS or ATOM that both return XML.

* Configuration Initializers
Separates the config.rb into separate files, which reduces the effort and extracts the application unique config info into easier to read chunks.

* Sexy migrations
This plug-in, which simplifies the syntax for database migrations, started out as a Rails plug-in, but is so useful that it is now part of Rails core.

* HTTP Authentication
DHH didn't feel the pain of not having easy HTTP authentication till he started using his own ActiveResource. Then he realized that although human users with their web browsers might prefer a forms-based authentication, machine users that wabt to call the REST interface for an application would prefer HTTP authentication.

* MIT License
Generating a new Rails assumes that you want to use the MIT license. The MIT license being simple and appealing.

* Spring Cleaning
Removing things form Rails core, like ActiveWebService, that are not as commonly used by many Rails applications, and turning them into plug-is that can be added separately by applications that require them. In the case of ActiveWebService, DHH also wants to guide users away from SOAP and toward REST, and removing ActiveWebService from Rails core does so in a "subtle" way.

DHH is funny, smart, and yes, opinionated. The Rails community wouldn't have it any other way.

Bob Martin - Writing Clean Code

Next, I had the extreme pleasure of seeing the great Bob Martin. Getting to be up close and personal to a legend of software, who normally is a keynote speaker at massive conferences like SDWEST, was a special treat. And Bob certainly lived up to expectations. He had incredible energy, was funny, and very insightful. Here are some excerpts from his remarks.

*Ward Cunningham noticed that "it was too easy to make a mess in Smalltalk".
*Have you ever been significantly impeded by bad code?
*Why did you write it? - "don't have time to write it correctly"
*Due to the quality of the code, we end up with diminishing returns from productivity over time as the code bloats
*Only solution to the problem is to clean up your code

*So what do we do about bad code?

• grand redesign in the sky, or


• incremental improvement

* Grand redesign in the sky

• all requirements are buried in the old system


• most companies will create a tiger team


• tiger team is hated by old team


• both systems are still under development however, so


• it takes years for the old system to be replaced by the new system


• as a result, the new system needs to be redone by the time it replaces the old system

*Incremental improvement

• always check in code better than when you checked it out


• never let sun set on bad code


• test first


• provides necessary flexibility

*Elizabeth Hendrickson had given Uncle Bob a green wristband labeled "test-obsessed". He feels like he cannot take it off. Ever. Because if he does, he is not really committed to testing.

*How much code coverage should we have? We should attempt to reach 100%, asymptotically.

• rspec is really good. Use it.
  
• Open closed principle is violated by repeating code
  
• Modules should open for extension, but closed for modification
  

*What do you do when it gets yucky?

• Refactor quick when you smell the mess
  
• Do not ignore these rules, or you will end up with a festering pile of code!
  
• Remember, we cannot write the correct code the 1st time. Trying to is egomaniacal.
  

*Iterate, like your teacher showed you how to write your grade school papers. Start with a rough draft, then refine.

*One of the best ways to ruin a program is try to make massive changes all at once
*Keep it running at all times
*You are not allowed to make a change that breaks the system

*How do you refactor? You need tests!!!
*Stick to the principle of least surprise
*Make tiny incremental changes, always running tests each time, minute by minute
*Notice what happens when you shrink the granularity of your development
*Uncle Bob is not sure that designating class methods as private is worth it?
*Software design is about separation of concerns - volatile and non-volatile code should be separated

*What if you make a mess? Was it worth it?
*Bad code gets harder to clean - Clean as soon as it gets messy
*So what about time to market? Isn't that the most important thing? What is the fastest way to finish dinner? Once you are done eating, just leave everything on the table, get up, and walk away. However, the fastest way to finish dinner is also the slowest way to start dinner, as you will have to clean everything before you can start cooking, and the plates will be hard to clean.

*Bad code

• Nothing has worse effect on a project than bad code
  
• Schedule slips
  
• Requirements missed
  
• Team dynamics problems
  
• Bad code rots and ferments
  
• Becomes a weight that drags team down
  

*Professional behavior

• Commitment like the "green band"
  
• Pros write tests first
  
• Pros clean their code
  
• Pros know going fast means going well
  
• Making the mess is NOT faster
  

*Like DHH said earlier today: "it's about making things a little bit nicer"


Standing On The Shoulders Of Giants

I was not really familiar with Adam Keys, but on the advice of a friend, I went to his session. Being a bit of a code archeologist myself, I looked forward to hearing Adam's thoughts about reading Ruby code. It was indeed worthwhile attending the session. Here are some notes of his remarks.

*Rails sucking all of the air in the web space has been a good thing...
*You need to listen or perform a composer's music to really know it...likewise we need to read the code to fully understand any piece of software.
*Reading code requires 5 times more effort than writing code
*We all write legacy code, if you look at a long enough timeframe
*Protect yourself from yourself, 5 days ago - going too fast with a pet technique

*Why reading code could be hard

• Not a lot of tool support for reading ruby
  
• Unfamiliar idioms
  
• Code written by non-heros
  
• You need a helmet with light on it - concentrate on only one part of code
  
• Find the entry points and hooks
  
• Don't chase rabbits into holes
  

Adam's guide to spelunking the code

• Look at URL, look at routes
  
• Controller
  
• Controller filters
  
• Controller actions
  
• Method called by actions
  
• templates and temp. helpers
  

Preface to spelunking

• filters in application controllers or helpers in application helpers
  
• model observers, validations
  
• code in libs or plugins
  

Keynote Introduction
After a rather dry and not that well executed speech by Five Runs CEO Steve Smith, Joey deVilla aka AccordianGuy gave a brilliant rendition of Radiohead's "Creep" as "I'm A Noob" an absolutely hillarious ode to DHH.

Avi Bryant

Avi Bryant is the creator of DabbleDB which was built using Smalltalk and the Seaside framework. As a Smalltalk enthusiast, many people were confused as to why on earth he would be a keynote speaker at a Rails conference. Ah, how we limit ourselves by putting everything into a little conceptual box and leaving it there for the rest of our lives. Getting outside perspective is the only way we ever learn anything and grow.

Avi started out by telling us that "He comes from the future". What he really meant was that Smalltalk has already gone through a similar evolution as Ruby, but that he considers Smalltalk 10-20 years ahead as far as solving certain problems that the Ruby community is still working on.

Avi considers Ruby and Smalltalk really the same language, at the core level. He showed a couple simple examples of their similarities. Then, he explained why he was giving his talk: to help out Smalltalk.

He had a number of other points:

* Ruby's features make it inherently slow.
* Smalltalk was bought, sold, then bought again by Sun. The VM developed for Smalltalk ended up as the Java VM.
* No reason Ruby cannot have the same performance characteristics as Smalltalk
*Having proper REAL debugging is crucial
*Ruby needs better VM tech
*What if Ruby's ObjectSpace was:

• transactional & persistent
  
• transparently distributable
  
• terabyte scale
  
• no restrictions to storage location
  

*GemStone - Portland based VM company, that has a very high performance runtime for Smalltalk. Acording to Avi, GemStone supports over 1000 VM's sharing 100s of GB of object memory. GemStone is free up to 4GB of storage.

*Objects get better with age
*Object lifecycle
*create
*use
*save

During the Q & A at the end, Ola Bini asked Avi about using Rubinous. Rubinious is a runtime for Ruby that is actually written entirely in Ruby itself. I was impressed by the fact that Ola didn't ask what Avi thought about JRuby. Ola is really cool, and his question was right on target after Avi's criticism of Ruby being written in C, and not turtles all the way down. Avi's response included his opinion that the Smalltalk VM is so advanced it would take the Ruby community 10 years to develop, just like it took that long to write the Smalltalk VM.

Personally, I do not buy that argument. Just because it took them 10 years to develop their runtime engine doesn't mean that all such tasks will take the same length of time. But Avi's perspective is a bit more biased than Ola's, I think.

And then came Ze Frank...


Ze Frank

I like Ze's stuff, but I had no idea what we were in for. I knew he was witty and smart, but what I didn't know what that he was a total genius. Ze is to theory of online community what Why The Lucky Stiff is to Ruby. On the surface everything is very amusing, but if you dig in you can go deep. Very deep.

Ze started out with a hilarious bit on "Acceleration Anxiety". He had the crowd laughing so hard that tears were coming out, as he made fun of his own fear of flying, and the idiocy of so many everyday things like the safety instructions and vomit bags on airplanes.

And then, he told us about a second kind of "Acceleration Anxiety". Here are a few of his key points on socialogy and online community:

People are learning a new language (ours)

People are having a media conversation

What can you do with your audience if they do things you don't like
*ignore them
*control them
*shut them down

Facilitating conversation : cultivating the energy

At a really good dinner party, it about the focus of the experience. Most people asked would not remember what had been discussed, but they would have remembered enjoying themselves - all of the different kinds of guests need to engaged.

Ze then shared his history of how he started and become the net culture star that he is today. At first his communications were one-way. But then his audience started becoming participants. He showed us a few examples:

The show
Earth sandwich
Human baton

Complexity of narrative has become much more complex

Technology is a facilitator of basic human emotions

The Core Values
*Conversation
*Community
*Belonging

There are one to one online communication. And there is mss online communication. We we need to do is look to the middle.


All I can say is wow. During Ze's introduction, Chad Fowler said he didn't realize that Ze was like some kind of Malcolm Gladwell. Neither did I.