RubyConf 2008 - Day 1 - Testing Heresies - Francis Hwang

The sessions for RubyConf 2008 day one were nearly over, when I attending this highly entertaining and informative one. I had missed the chance to see a previous presentation by Francis Hwang, and being as test-obsessed as I am, I was not going to make the same mistake twice.

Many people are confused over mocks, stubs, and other fakes. Here are the real definitions.

Gerard Meszaros - Test Doubles
- dummies - passed around but never used
- fakes - work but unstable in production
- stubs - provide canned answers to calls made during test
- mocks - mostly concerned with surface interaction

Classical TDD use real objects.
Mockist TDD will always use a mock for any object with interesting behavior

Problem #1 with mocks
Cannot figure out problem and solution at same time

Problem #2 with mocks
Bugs can slip between layers

Francis Hwang, TDD Clasiscist
- test setup creates records in the db
- no sqlite in test
- when you use test doubles, it's always a big deal

Real world example #1
Ensure that an optimization is actually being used
Redefine the actual surface interaction to 'scream' on error

Real world example #2
Faking a real web service (not written by us)
Best to use stub, to duplicate functionality
Need test, stub, AND test for stub

Dogma - if you tests do not run quickly, there's something wrong with you
Heresy - I'd like my test to run 30 sec. I'd also like a pony

Francis does not use autotest, because he does not want to run all of the tests every time he changes something. Eric Hodel chimed in from the audience says that it can run only what changed. Only if it passes, will it run ALL tests. Me, I loves my autotest... with growl notifications, it's all good.

devver
They say it is a ec2 based distributed test system...

testjour
uses bonjour to trigger a farm of machines in your LAN to perform distributed tests

Dogma - you code should always be well-structured
Heresy - under-structuring can be freeing

Uses example of constantly cleaning small NYC apt. It was living on a boat, where you spent all your time putting things away.

Dogma - Fine grained focus leads to high quality code
Heresy - Never forget the unknown unknowns

System design 101
How likely fail
What will fail look like
How can I find out quickly
How can I limit damage

Dogma - testing makes my job easier
Heresy - testing makes it easier for others to do my job - unless my job keeps growing

Rails has made it so simple to implement simple business logic, so way more is expected now.

More Developer Fun In Los Angeles

Last night was the always entertaining Los Angeles Application Developer Meetup. L.A. is not known for the same massive quantity of developers as the bay area, for example, but having joined forces with the LA Ruby on Rails Meetup crew some time back, the monthly event is still growing.

This month's was held at the impressive new digs of YellowPages.com, who have been busy building out a huge Ruby on Rails based local search replacement for their venerable 100K+ lines of Java code. Thanks for the great sandwiches, boys... next time, buy beer!

Anyhow, four presentations took place on various fun topics of technical interest. First, Jeff Yamada and his associate (sorry I didn't catch your name) showed AIR. I then did a short presentation on Continuous Integration and CruiseControl.rb. Olmo Maldonado talked about his impressive JavaScript library MooTools, and then Ari Lerner showed his just-relased Ruby web development framework, Sinatra.

AIR
Long, long ago, I used to work with Flash and ActionScript. Most of us more "serious" programmers were always very frustrated with the language, the programming environment, pretty much everything except for the final product. Since then Adobe has introduced technologies like Flex and AIR to try to address the shortcomings of using Flash as a "real" development tool. Yamada and friend are the authors of the upcoming book "AIR Bible". AIR nee Apollo is designed to provide a way to create native OS applications, while using the toolset that Flash/Flex developers are used to. If you are into Flash/Flex already, then this is probably great for you, but I am not overly in love with so many angle brackets mixed in my code... let's just say that "it ain't Ruby". One small thing I will suggest to the boys: please try to find some relevant application to show for a demo a of a new technology. Between this, and what all of the Microsofties were show a few months back for Silverlight, no one is going to take this new desktop app paradigm seriously!

How To Get Your Code Together and Keep It That Way
Then I did a presentation myself, called "How To Get Your Code Together and Keep It That Way". It was really an exhortation to use the best practice of Continuous Integration, while showing how easy it is to get started with the ultra-cool CI tool CruiseControl.rb. If you are a Rails dev, and you are not using CruiseControl.rb yet, you should be. If you have a development project with greater than zero developers, it is worth it to setup a CI system, even just to keep yourself in check. You can get a copy of my presentation here. But don't just believe me, it is in yesterday's O'Reilly Radar too.

As I was demoing the CC.rb project page, where they are doing CI builds of both CruiseControl.rb itself and the Ruby on Rails project, I noticed that the Rails build has had failing unit tests since Oct. 3. What is up with that? Let's get on it boys!

MooTools
Next up was Olmo Maldonado, once of the authors of MooTools. MooTools is a JavaScript framework for web development along the lines of Prototype or jQuery. I had been hearing about MooTools, but hadn't really checked it out. Naturally, since both of those other JS frameworks already exist, and with Prototype really having dominated in the acceptance wars, MooTools does have a different approach on several levels to justify its existence.

Heavily optimized for speed, according to Maldonado, Moo performs much faster than either Prototype or jQuery on the slickspeed benchmark test at performing DOM selects, which is the core convenience provided by any JS framework.

Another difference is that MooTools libs are completely orthogonal. As such, the developer only needs to incorporate the specific libs that contain the functionality that is required for that application, unlike both Prototype and jQuery which require downloading the entire framework to use any part of it.

MooTools seems to have a more serious emphasis on good software devlopment practices than the othe JS framework projects. One example of this is that MooTools was developed using JSSpec, which is a Behavior Driven Development framework for JavaScript.

Already up to version 1.2, MooTools has been adopted in some very interesting places. The three demos that Maldonado showed were the iPhone version of the Pop-Cap game Bejeweled, the Ubuntu.com website, and then most curious of all, the Microsoft Expression website! What, M$ couldn't use Silverlight? Oh yeah, everyone already has Javascript... no additional download required. Whatever happened to that famous dogfooding?

MooTools is realy designed for the developer who wants to work in pure JavaScript. Most Rails devs like to use the Ruby on Rails Prototype helpers to do nice AJAX-y things like edit in-place and auto-complete functionality. MooTools does all that, plus has a bunch of UI gadgetry too, but having no inline functions, is not really compatible with the way the Rails helpers work. Perhaps someone will create a Rails plug-in that can collect up all of the MooTools Javascript, kida like what the Unobtrusive Javascript plugin does... for those Rails people who want to take advanatge of the DSL-like syntax, instead of dropping all the way into JavaScript.

Sinatra
Speaking of DSLs, the final presentation of the evening was by Ari Lerner, showing his incredibly cool new little Ruby web framework called Sinatra. In fact, I was so impressed by Sinatra, that I decided to give it its own posting. I suggest you go check it out right now!

Stop The Bug-Based Development

I enjoyed reading Phil Haack's little "you're either with us or against us" diatribe from earlier today, where he rails against non-test-driven development. I only differ in that I prefer to call it Bug-Based Development (BBD).

So are you one of those fifth-column types who is providing aid and comfort to the enemy? Draw a line in the sand... never make your troops take casualties retaking the same ground twice... no bastard ever won a war by dying for his country. You won it by making the other poor dumb bastard die for his country.

Give 'em hell, Phil!

Power Rangers Should Test First

Last night, I got home a bit late. My 8 year old son was still up, watching the Power Rangers. "Sit down and watch with me, Dad," he said. "I'm surprised you still like the Power Rangers," I said, as I sat beside him on the sofa. "I mostly just like the vehicles," he told me.

The Power Rangers were in trouble in this episode. The bad guys had taken remote control of the Rangers' giant robots, and the Power Rangers were helpless without their usual firepower. The Smart Rangers, meaning the guy with the glasses and the Asian girl (WTF!!!), were constructing a jamming device to block the bad guy's signal, while the other Rangers got their behinds handed to them by their own robots.

"OK, that finishes it. Let's go!" exclaimed Glasses Ranger as he attached the last part to the jamming box. They rushed off to the scene of the battle. "I hope this works," said Glasses Ranger and pushed the button. Nothing happened. "Oh no, what could be wrong?!" cried out the Smart Rangers.

My son, who had been quietly watching all of this, suddenly spoke out. "They're stupid. They should have tested it first."

Toward A Test-Driven Culture

We all have to move to a test-driven culture. We cannot think about testing as an afterthought, or as a necessary evil. Testing has to be baked into the culture of software development the same way that testing is baked into Ruby on Rails. It is there right from the beginning, and makes you feel guilty when you DON'T do it. The DNA of software development has to evolve to grow systems with neurons that make it so that a lack of testing feels like sticking your hand into a wall socket; you notice right away when it happens, and you take immediate action to make it stop.

Software testers are considered "lower on the food chain" then software developers or analysts. That means testers are compensated less than their developer or analyst counterparts. Even on many important projects, across both companies and industries, this is still the case. This amazes me, given how much of the world runs on software. If the software that runs your airplane's landing gear, or your electronic voting system, or whatever other ultra-critical system has not been sufficiently tested, lives can be at stake. Despite the fantastic work being done in the area of software quality, and the proliferation of practices that can dramatically reduce the cost of creating and maintaining a system, we still all too often see quality getting shortchanged on required resources.

So testing is good, and we need more of it, surely. But we need to go further, much further. Giving proper attention to test-driven development is great. But even all this is not enough. What we need to do is to apply a test-driven mentality to all aspects of business. Marketers have found that testing and measuring their marketing yields a far greater ROI then marketing based on opinion or theory. Salespeople have been measured by quotas, and the most successful sales organization performing testing for some time.

In particular, we need to move toward management philosophies that apply testing to business decisions, in particular those about processes. All too often, decisions have consequences often than those intended, perhaps failing in one of more objectives. A new process designed to streamline a company, can actually result in adding more burden with less efficiency.

Without the ability to test, management will eventually lose control over itself. With no feedback, failures may be looked at as successes, and vice versa. Management cannot be arbitrary; by carefully adopting a test-driven culture across all disciplines within an organization we can improve both the quality of our work, as well as the quality of the experience we have while doing that work.