Ruby/Microsoft?

Martin Fowler just posted on his bliki a little piece entitled RubyMicrosoft. As I read it, I felt a strange chill run down my spine. Perhaps a few words of explanation are needed...

At RailsConf recently I stood at the Thoughtworks booth for a moment chatting up Fowler, and embarrassing myself by asking for a photo with him (he doesn't pose for photos with people). Luckily, I was rescued by the arrival of Oli Bini, and an ensuing short discussion about JRuby. I expressed my admiration about how they had been "cleaning Microsoft's clock" as far as their much faster progress with a runtime for the Ruby language then Microsoft's efforts with the DLR.

Much to my surprise, Martin was actually paying attention to what we were saying (I mean to what I was saying, after all he knows and speaks with Ola), and chimed in about how Thoughtworks was working with Microsoft on language compatibility for IronRuby. I expressed a poorly articulated skepticism at the prospects.

Lo and behold, the RubyMicrosoft blog posting sums up rather well the half-formed mumblings I was trying to stammer out. Well, this is why he is a keynote speaker, and I am a dead programmer. However, impressed as I am at his intellect, I am not sure I share his optimism about Microsoft.

I certainly don't know if I am an "alpha geek" or anything like that. I'd rather be a jazz programmer than a rock star. But given my long history working with Microsoft technology, and then the seemingly contradictory fact that I was at both RailsConfs, I must really identify with the movement. Chalk it up as just one more developer who Microsoft is losing to the rebellion.

I Would Rather Be A Jazz Programmer

I have reading a bit lately about "rockstar" programmers. Recruiter ads are proclaiming "Rockstar programmer needed". Various websites named rockstar<name of technology> or alternately <name of technology>rockstars are springing up everywhere.

I would rather be a "jazz musician" programmer, myself. Nothing against rock, don't get me wrong. Glam, punk, metal, I can go there. I frequently do. It's the "star" part, that has been starting to bother me.

Here are some differences, as I see them:

Rockstar
- One big hit song, then disappears
- Embarrass themselves as they age
- Claims they wrote the song
- Keeps trying to get back that sound they used to have
- Gets back together with the old band after unsuccessful solo careers
- Wants to marry a model and have a movie cameo
- Won't play without a contract and advance payment

Jazzer
- One big hit, and they become an influence
- Get cooler with age
- Claims the song is just a cool arrangement of a standard
- Keeps trying to produce a new sound
- Records with a variety of musicians over time
- Wants to become a professor at Berkeley School of Music
- Jams on the street corner just because they feel like it

RailsConf 2007 - Day 3

By the time we got to RailsConf Day 3, only the most hard core remained. In other words, almost everyone. And we were glad to have stuck around, because some of the best content of the entire conference was yet to come. David Black, introduced two integral members of the Ruby on Rails team, Jamis Buck and Michael "Koz" Koziarski, the co-authors of the seminal "Rails Way" blog.

The Rails Way
Jamis and Koz and both written and read a lot of Ruby on Rails code. The Rails Way is not just what could be done, but what should be; the best way to build a Rails application. Their knowledge of best practices is second to no one, and they shared a whole bunch of great information. I can't really do their tag team code review justice, but here are the highlights:


Fat controller is anti-pattern
*hard to test
*break apart into a model object

A model is anything that encapsulates data and logic, not just an object that talks to a database.
*one benefit of this encapsulation is testing
*another advantage is the form_for handle can easily use a model

When to use with_scope
*move the logic to the parent class, so as to freely get the scoping with no extra work
*do not go crazy with DRY

ActiveSupport has helpers to make some code self-documenting for dates/times

Asociations
*People make associations, but then fail to use them throughout the rest of the application
*this helps keep code from breaking in the case where database keys change, for example

Cargo culting
*He hates !! (not-not) idiom.
*Ternary operator...likes it sometimes
*Explain exactly why you need boolean returned, again?

Model helpers
*use associations in views instead of instance variables

Routing
*simplifying non restful routes - map.with_options

Override to_param to make nice looking URL

Returning method

 returning new do |booking|
  booking.some_thing
  booking.some_other_thing
 end

The session was really great. Pretty much everyone in the room, no matter how advanced their level, got something good out of it. We could have stayed there all day, with different people bringing up pieces of code for these guys to refactor. If you can, hire them.

Javascript - Fu
Dan Webb is the author of the LowPro extensions to the Prototype JavaScript library. He had a really nice tight Takahashi style presentation. He was also extremely funny, and the information incredibly useful. Once again, the last day of RailsConf really had a lot of good stuff!

JavaScript is the language we all love to hate. A peasant language, meaning that "lower-level" programmer have to work on the JavaScript code while "higher-level" ones get to work on the "architecture".

JavaScript - fu is not easy to master. Developers are forced into refuge by using frameworks and libraries.

The Ancient Manuals of JavaScript-Fu
* The Tao Of The Event Handler (Events)
* Five Method of DOM Fist (DOM)
* Lightning Script Style (Optimization)
* Iron AJAX Technique (Progressive enhancement)

There are big differences in browser implementation of JavaScript.

Two main techniques: inline vs. scripted event handlers

Inline - applied as soon as browser loads

What happen when more than a little bit of inline code

Scripted event handlers
*problem in they attach after the element is loaded

LowPro library is an extension library for Prototype, but Prototype library trunk now has a similar thing.

// Prototype 1.5+ and LowPro
Event.onReady(function() {
  $('item').observe('click', function() {...});
});

// Prototype trunk
Event.observe(window, 'DOMContentLoaded', function() {
  $('item').observe('click', function() {...});
});

// Basic onload
Event.observe(window, 'load', function() {
  $('item').observe('click', function() {...});
});

Attach handler from script to each object you want, in order to stay DRY

Separate your JavaScript out of your pages, just like you separate CSS

Large number of event handlers can choke browser performance

How:
* use script based by default
* in bug page try event delegation
* if all else fails use inline

Event delegation - yahoo tehnique
* event bubbling

Better inline handlers
*a tag needs an actual HREF
*do as little as possible, for example:

 return view(this);
 function delete(elemtn)
 {
  new Ajax.Request(element.href), {
   method: 'delete'
  });

  return false ;
 }

5 methods of DOM fist
*3 oficial W3C methods to modify the DOM
- appendChild
- insertBefore
- replaceChild
*1 non-standard (from IE)
- innerHTML

DOM methods insert with precision - have to create nodes first
InnerHTML shifts large bulk amounts of HTML, but with little control

Really easy to use innerhtml from Rails with Ajax

The bastard son - unholy marriage of DOM methods and HTML

Element.fromHTML = function(html) {
  var root = document.createElement('div');
  root.innerHTML = html;
  return root.firstChild;
}

var node = Element.fromHTML('<div>');
element.insertBefore(thing, node);

Lightning Script Style
* making things as fast as possible
* do not use javascript_include_tag :defaults cause it downloads 5 big files
* browsers take time both to download AND evaluate a script
* the less JS the better
* mooeffects smaller than scriptaculous, but less powerful
* browsers can only load 2 resource at once
* combine js files
* Use gzip instead of JS based minification.

Make sure everything is cacheable

Faster loops

 for(var i - 0, enemy;
  enemy = enemies[i]; i++) {
 defend(enemy);
  }

Be careful with selectors
$$('.anything'); slower
$$('span.anything'); slow
$$('#item .anything'); // css xpath selector

Iron Ajax Technique
* rule #1: browsers suck
* main browsers are getting better quickly
* but what about the others?
* Corporate security blocks Javascript at firewall
* the traditional rails opin: f* you!
* but why turn away users if you don't have to

Progressive Enhancement
1. start with plain HTML
2. test if necessary brwt features are there (XMLHTTPreq, canvas, etc)
3. If present, then apply xtra JS features

Tt's easy to apply to ajax
* HTML
* JS
* RJS

Try progressinve enhancement first

Lastly, Dan recommended a few books on Javascript he said are better than all the others:
* Professional JavaScript Techniques
* Bulletproof Ajax
* DTHML Utopia: Modern Web Design Using Javascript and DOM

Dan's session was full of good stuff. Even the people who thought they knew it all were scribbling down a couple things before the end. And thank you, Dan, for making it really fun as well as informative.

Dave Thomas
Dave's speech was unconventional, in that he doesn't say what people expect him to. I consider that a good thing. Many people in the crowd were perplexed for long moments as he spoke. For those people who need a simple explanation of Dave's speech, the two recurring themes for RailsConf were "give", and "don't get stuck".

"Just to set the record straight, the first RubyConf was 3 people and a dog, and the speaking was in between sips of beer"

Orthodoxy - do what is expected
Orthopraxy - do the "right thing"

It's called Object oriented programming, not class oriented programming. Challenge: write next next program with objects instead of classes

What Are Your Cargo Cults?

Dave is a real thinker, and I hope that our community can rise to the challenge and keep progressing. Embrace Orthopraxy!

RailsConf 2007 - Day 2

After the incredible energy of RailsConf Day 1, Day 2 seemed to slow down the pace a little bit. That was good for me, as I could spend some time catching up with friends both old and new, and not just attending sessions like an information junky. First would come the morning keynote speeches from Cyndi Mitchell of ThoughtWorks, and headliner Tim Bray of Sun Microsystems, with a short interlude from Charles Nutter.

Cyndi Mitchell - Thoughtworks

Ms. Mitchell is UK Managing Director of ThoughtWorks. As such, she is a polished corporate representative, which is exactly the kind of emissary that Rails needs in big organizations. She had a few interesting points:

* 40% of new work at ThoughtWorks is coming from Ruby on Rails in the enterprise
* We need to reclaim the word "enterprise" back from meaning "bloated, inefficient, and expensive" to its original meaning of "entrepreneurial"
* The "real" enterprises are using Ruby on Rails to get stuff to market quickly
* ThoughtWorks is offering a complete Rails production stack called RubyWorks, and is will be providing paid 24/7 support for Rails.

Tim Bray - Sun Microsystems
After Cyndi, the irrepressible Tim Bray took the stage. Despite not really being that much into Java, I have been following Tim's blog for quite some time, as he is a smart thinker with plenty to say to the industry at large.

Tim thinks that the Ruby language is as important as Rails (I agree!!). With JRuby, Sun wants to sell lots of hardware to large enterprises that will be choosing Sun Hardware and Solaris as the preferred platform for their Rails applications. Sun is also doing lots of work on NetBeans to do better Rails development.

Tim then briefly brought up Charles Nutter to give him some Rails street cred...er, I mean to explain why Ruby developers should care about JRuby.

Charles made two main points:
* A whole different way to do Ruby dev...using tried and true Java tools that already exist
* Move into enterprise where Java is accepted but Rails is still not

Then back to Tim:
It is not CIO's who lead industry...it is people like us.

How to make money on free products
* first you get adoption
* then you get deployments
* then you can monetize at point of value - no large business will adopt any tech that is not supported

Big companies will either get a handle on web 2.0 or get disintermediated

He assumes Rails will succeed beyond our wildest dreams.

Java, php, and .NET will never go away

The network is the computer...and its a heterogeneous network. And JRuby is how we get everything to talk to each other.

Java really consist of three main parts: language/JVM/Java APIs.
JRuby lets Rails programmers access the latter two.

How do we get everything to talk to each other? REST

Etags are a way to solve the REST problem with different versions of the same data.

ATOM - Everything should have a publish button: cameras, phones, browsers, etc

Is REST all there is? SOAP... WS-* 36 specs, ~1000 pages of documentation.

WSIT - interop with MS WCF

Hot developer issues
*static vs dynamic languages
*maintenance
*tooling
*integration
*concurrency
*time to market
*scaling
*load balancing
*observability
*file IO
*DBMS
*Shared nothing architectures
*CPU
*concurrency
*debugging
*unit test
*functional programming?

Which matter the most?
*Conventional wisdom for each language
- Java: performance, tooling
- php: web scaling, ease of use
- Rails - time to market, maintainability
*Tim says: The most important are Time to market, and maintainability

Extra Action Marching Band
After the speeches, I went to a session I didn't find that interesting at all, and so in the interest of not offending anyone else too badly, I shall let it remain anonymous...you know who you were. But near the end, a massive blast of sound came from the entrance to the conference center, and then proceeded past the sessions rooms toward the main hall.

It was the Extra Action Marching Band, an amazing avant-garde performance troupe. If San Francisco (which is where they are from) had a love-child with New Orleans, it would be something like this band.

Despite an almost-incident with the conference center security staff, they were able to finish their set. The music and performance were quite incredible, showing a high level of musicality as well as serious creativity. The bullhorn with a delay pedal attached was really amazing sounding...I have to make one like that for my harmonica.

After this unique musical interlude, it was back to the conference sessions...thamks to Tim Bray for the photograph.


Xen And The Art of Deployment
Ezra Zygmuntowicz of Engine Yard, Merb, and BackgrounDRb fame gave my favorite presentation of the day. The session was packed, with both information and people. If you wanted information about scalability, at least from the perspective of the stack, then this was the high point of the entire conference. Here are copious notes of what he said:

Serving up Rails right now is all about Mongrel

Mongral uses a Ragel state machine for validation

Why is mongrel better:
*HTTP is well known protocol
*Mongrel easy to setup and use
*Transparent wire protocol

Thread safety
*Rails not thread safe
*1 request at a time per mongrel

New dog, old tricks
*Wide # of options to load balance front end
*pen, pound, HA-proxy
*Lightspeed can serve static files
*Apache 2.2/mod_proxy_balancer can do same

Each current solution has problems

Nginx
*serious performance
*small resource footprint
*no leaks under heavy load
*killer rewrite and proxy mods
*author is cool and responsive
*Nginx + Mongrel
*This is THE stack
*apache for mod_dav_svn
*flexible config allows both serving static and dynamic requests
*fast!!!
*Nginx cannot do upload progress due to buffering
*no connection rate limiting for proxy modules yet

The future for nginx
*mod_rewite going away
*replace with http_script_module
*embed NekoVM directly into nginx

Perfect simple stack
*Linux
*Nginx
*Mongrel (mongrel_cluster)
*Monit

Swiftiply - the cool new solution

Evented Mongrel
*hot patch
*remove ruby treeads
*replace with eventmachine event loop
*mongrel single threaded
*big IO speed increase
*stand up to higher load

How is single threaded faster?
*Ruby green threads are slow
*mutex locks are expensive
*one process can only do so much IO
*event driven means running in a tight loop and firing callbacks
*without context switching between threads, single process has less overhead

Mongrel vs evented mongrel - performance stats
*1 user - 1K more req/sec
*100 users - 758 req/sec vs. 3700+

Swiftiply proxy
*event drive proxy, small memory footprint
*faster than ha-proxy

How is swiftiply different
*In a normal proxy, you need to config each back end server in advance before starting proxy
*swiftiply is a client/server architecture, where new back end servers register with proxy
*keeps persistent connection between proxy and back end servers
*auto-configuration of proxy

This means you can start/stop as many mongrels as you want, and they get configured into proxy, opening the door to auto-scaling

The Zen of Xen
*monolithic linux VS modular lnux
*old model: one box running all services
*new: specific virtual machines setup for specific purpose

We all want to modularize application why not modularize servers

What about more than 1 box?

Old school - more boxes
*mysql box
*another box for other services

New school
*add another xen box
*pick resource intensive parts, and more to another VM
*each VM only runs one thing, but well
*target performance problems
*scale better

Advanced clustering
*boot off dom0 off of USB drives
*SAN for all Xen domU
*red hat clustering suite for fencing and cluster options
*GFS for 100% posix file sys
*hardware load balancer, or dedicated box running Ultra Money or LVS

Create a fabric of compute and storage
*If a node fails, just swap in another
*Move hot VM's to less loaded machines
*Deploy app on 1 node, then bounce mongrels with GFS
*Fragment and page cache across nodes
*Scale from 1 or 2 VMs to as many as traffic requires and then back down again as traffic subsides

RAM RAM RAM
*most rails apps are RAM bound before they become CPU bound
*Avg mongrel serves 70-120 mMP PER mongrel
*RMagick is a typical RAM hog
*:include on active record also very bad performance
*95% of rails apps will leak memory at point or another

Use Rails plugin leakhouse to find leaks

Rails eats DB resources for breakfast

Most people's production apps hav no database indexes! Learn where and when to apply indexes.

ActiveRecord insulates developers so much to the point of massive inefficiencies.

You will need to denormalize data and use custom SQL statements to get performance

Other Tips
*do not use file system sessions
*script/runner is inefficient. do not load rails into bgprocesses.
*Use raw MySQL and Ruby without Rails if you can
*DO NOT use script/runner to pocess incoming email. Run deamon in loop and poll svr with net/pop3 or net/imap

Rails is great for 80/20 rule, but you are on your own for last 20%

Learn how to write custom mongrel handlers

When is optimization NOT premature?

Ruby is plenty fast, its Rails that is on slow side

Cache, cache, cache espcially static HTML files

Do not store full ActiveRecord object in session, just store hash to recreate it

Parting thoughts
*do not take anyone's word as gospel
*test and benchmark for yourself
*trust but verify

The Rest Of The Day
After Ezra'a session he was swarmed like a rock star. I know, cause I was standing nearby having a really nice chat with Zed Shaw the creator of Mongrel and Evan Phoenix, the creator of the Rubini.us Ruby runtime I was mentioning yesterday. You go, Ezra!

Later on, I got to hang out at the Engine Yard booth with Ezra for a few minutes, where Mike Brown and some other brilliant Kiwi whose name I didn't catch were chatting him up. I learned even more about hard-core clustering from listening to those guys go off...all I can say is I'm glad I have hosting at Engine Yard!

I wanted to go to Dr. Nic's RejectConf, but so many people told me they planned on going that I figured I wouldn't get in. I couldn't take the prospect of being rejected twice at the same conference, so I made other plans which turned out to be really fun.

After dinner with the same group of RailsConf friends from last year, with more delicious Portland beer, and excellent conversation, I headed to the BoF sessions, to hear about Amazon's EC2 service. I really wanted to learn more hard-core details about it, but so many people were just hearing about it for the first time, that the session was dominated by introductory questions. Oh, well!

I was about ready to turn in. Lucky for me, Rails Machine was having their party at my hotel. Bradley and company are really smart, cool people, and also offer a fantastic service, so I was glad to hang out with them for a few minutes. The most fun was jamming on harmonica with Joey deVilla the AccordianGuy. After hearing him last year, I had brought a couple of my harps with me just in case, and we finally had a chance to bust out a couple of songs together. That was really fun! Thanks, Joey. If anyone has any photos or video, please post it somewhere...if not, well, you had to be there.

RailsConf 2007 - Day 1

I was primed and ready for RailsConf 2007 Day 1. The excitement was in the air, and the crowd was primed. We awaited a day absolutely packed with mind candy. Starting with Chad Fowler, David Heinemeier Hansson, Robert Martin, Adam Keys, Avi Bryant, and Ze Frank. Wow, what a lineup!

As Chad Fowler took the stage, his ukulele in hand, we eagerly awaited his opening remarks. Chad really did a great thing, by not just giving us the "feel-good rah-rah Rails is taking over the world" intro speech that most people probably expected. Instead, he took a higher path. The virtues of sharing are not limited to open source projects, but need to get into the real world. For example, using Rails to create applications that solve real problems that have not been tackled yet, due to the high cost of developing a solution. We as a community can also do a lot for the world outside of just developing software, no matter how cool it is. The Pragmatic Programmer fund-raising drive is a fantastic example of mobilizing the community, and Chad exhorted the crowd to keep those donations coming.

Is the Rails community more concerned with altruism and spirituality than other tech people? Or are we following the once again popular meme of greater social awareness and activism? That may be fodder for another posting some other time...back to the show.

His opening remarks finished, Chad played a little uke ditty, under a delightful beat poetry introduction given by Rich Kilmer for David Heinemeier Hansson. As David took the stage, we all wondered, "what would he spring on us this year? We just spent the last year RESTifying our applications, what MORE do you want from us?!"

And here is what he said:


Keynote - David Heinemeier Hansson

REST is where it is at, and Rails 2.0 doesn't alter that path. But there are 9 other things he really likes about Rails 2.0.

* Debugging with breakpoints
David showed the ability to put a breakpoint into your Rails controller that opened irb with all of the context and objects ready to be introspected. Although dropping to command line is not exactly what most of us want out of a debugging session, this could easily have a superior UI put on it.

* HTTP Performance
HTTP performance in an area where a couple of small changes can dramatically increase performance. His examples showed very easy combining of all CSS files, and separately all JS files together into a single file, and then compressing of each group of files into a gzip archive for quick download.

Another HTTP performance technique that 2.0 introduces is a way to get around the two connection limit that browsers enforce, by serving static assets from multiple host servers using the new :asset_hosts configuration option.

* Query Cache
Rails 2.0 can look at all of the SQL that ActiveRecord is executing, and if there have not been any inserts or updates, automatically caches that database query.

* Rendering
Rendering of a resource request has now been separated from the MIME type of the request. Now, it is much easier to handle multiple format requests like RSS or ATOM that both return XML.

* Configuration Initializers
Separates the config.rb into separate files, which reduces the effort and extracts the application unique config info into easier to read chunks.

* Sexy migrations
This plug-in, which simplifies the syntax for database migrations, started out as a Rails plug-in, but is so useful that it is now part of Rails core.

* HTTP Authentication
DHH didn't feel the pain of not having easy HTTP authentication till he started using his own ActiveResource. Then he realized that although human users with their web browsers might prefer a forms-based authentication, machine users that wabt to call the REST interface for an application would prefer HTTP authentication.

* MIT License
Generating a new Rails assumes that you want to use the MIT license. The MIT license being simple and appealing.

* Spring Cleaning
Removing things form Rails core, like ActiveWebService, that are not as commonly used by many Rails applications, and turning them into plug-is that can be added separately by applications that require them. In the case of ActiveWebService, DHH also wants to guide users away from SOAP and toward REST, and removing ActiveWebService from Rails core does so in a "subtle" way.

DHH is funny, smart, and yes, opinionated. The Rails community wouldn't have it any other way.

Bob Martin - Writing Clean Code

Next, I had the extreme pleasure of seeing the great Bob Martin. Getting to be up close and personal to a legend of software, who normally is a keynote speaker at massive conferences like SDWEST, was a special treat. And Bob certainly lived up to expectations. He had incredible energy, was funny, and very insightful. Here are some excerpts from his remarks.

*Ward Cunningham noticed that "it was too easy to make a mess in Smalltalk".
*Have you ever been significantly impeded by bad code?
*Why did you write it? - "don't have time to write it correctly"
*Due to the quality of the code, we end up with diminishing returns from productivity over time as the code bloats
*Only solution to the problem is to clean up your code

*So what do we do about bad code?

• grand redesign in the sky, or


• incremental improvement

* Grand redesign in the sky

• all requirements are buried in the old system


• most companies will create a tiger team


• tiger team is hated by old team


• both systems are still under development however, so


• it takes years for the old system to be replaced by the new system


• as a result, the new system needs to be redone by the time it replaces the old system

*Incremental improvement

• always check in code better than when you checked it out


• never let sun set on bad code


• test first


• provides necessary flexibility

*Elizabeth Hendrickson had given Uncle Bob a green wristband labeled "test-obsessed". He feels like he cannot take it off. Ever. Because if he does, he is not really committed to testing.

*How much code coverage should we have? We should attempt to reach 100%, asymptotically.

• rspec is really good. Use it.
  
• Open closed principle is violated by repeating code
  
• Modules should open for extension, but closed for modification
  

*What do you do when it gets yucky?

• Refactor quick when you smell the mess
  
• Do not ignore these rules, or you will end up with a festering pile of code!
  
• Remember, we cannot write the correct code the 1st time. Trying to is egomaniacal.
  

*Iterate, like your teacher showed you how to write your grade school papers. Start with a rough draft, then refine.

*One of the best ways to ruin a program is try to make massive changes all at once
*Keep it running at all times
*You are not allowed to make a change that breaks the system

*How do you refactor? You need tests!!!
*Stick to the principle of least surprise
*Make tiny incremental changes, always running tests each time, minute by minute
*Notice what happens when you shrink the granularity of your development
*Uncle Bob is not sure that designating class methods as private is worth it?
*Software design is about separation of concerns - volatile and non-volatile code should be separated

*What if you make a mess? Was it worth it?
*Bad code gets harder to clean - Clean as soon as it gets messy
*So what about time to market? Isn't that the most important thing? What is the fastest way to finish dinner? Once you are done eating, just leave everything on the table, get up, and walk away. However, the fastest way to finish dinner is also the slowest way to start dinner, as you will have to clean everything before you can start cooking, and the plates will be hard to clean.

*Bad code

• Nothing has worse effect on a project than bad code
  
• Schedule slips
  
• Requirements missed
  
• Team dynamics problems
  
• Bad code rots and ferments
  
• Becomes a weight that drags team down
  

*Professional behavior

• Commitment like the "green band"
  
• Pros write tests first
  
• Pros clean their code
  
• Pros know going fast means going well
  
• Making the mess is NOT faster
  

*Like DHH said earlier today: "it's about making things a little bit nicer"


Standing On The Shoulders Of Giants

I was not really familiar with Adam Keys, but on the advice of a friend, I went to his session. Being a bit of a code archeologist myself, I looked forward to hearing Adam's thoughts about reading Ruby code. It was indeed worthwhile attending the session. Here are some notes of his remarks.

*Rails sucking all of the air in the web space has been a good thing...
*You need to listen or perform a composer's music to really know it...likewise we need to read the code to fully understand any piece of software.
*Reading code requires 5 times more effort than writing code
*We all write legacy code, if you look at a long enough timeframe
*Protect yourself from yourself, 5 days ago - going too fast with a pet technique

*Why reading code could be hard

• Not a lot of tool support for reading ruby
  
• Unfamiliar idioms
  
• Code written by non-heros
  
• You need a helmet with light on it - concentrate on only one part of code
  
• Find the entry points and hooks
  
• Don't chase rabbits into holes
  

Adam's guide to spelunking the code

• Look at URL, look at routes
  
• Controller
  
• Controller filters
  
• Controller actions
  
• Method called by actions
  
• templates and temp. helpers
  

Preface to spelunking

• filters in application controllers or helpers in application helpers
  
• model observers, validations
  
• code in libs or plugins
  

Keynote Introduction
After a rather dry and not that well executed speech by Five Runs CEO Steve Smith, Joey deVilla aka AccordianGuy gave a brilliant rendition of Radiohead's "Creep" as "I'm A Noob" an absolutely hillarious ode to DHH.

Avi Bryant

Avi Bryant is the creator of DabbleDB which was built using Smalltalk and the Seaside framework. As a Smalltalk enthusiast, many people were confused as to why on earth he would be a keynote speaker at a Rails conference. Ah, how we limit ourselves by putting everything into a little conceptual box and leaving it there for the rest of our lives. Getting outside perspective is the only way we ever learn anything and grow.

Avi started out by telling us that "He comes from the future". What he really meant was that Smalltalk has already gone through a similar evolution as Ruby, but that he considers Smalltalk 10-20 years ahead as far as solving certain problems that the Ruby community is still working on.

Avi considers Ruby and Smalltalk really the same language, at the core level. He showed a couple simple examples of their similarities. Then, he explained why he was giving his talk: to help out Smalltalk.

He had a number of other points:

* Ruby's features make it inherently slow.
* Smalltalk was bought, sold, then bought again by Sun. The VM developed for Smalltalk ended up as the Java VM.
* No reason Ruby cannot have the same performance characteristics as Smalltalk
*Having proper REAL debugging is crucial
*Ruby needs better VM tech
*What if Ruby's ObjectSpace was:

• transactional & persistent
  
• transparently distributable
  
• terabyte scale
  
• no restrictions to storage location
  

*GemStone - Portland based VM company, that has a very high performance runtime for Smalltalk. Acording to Avi, GemStone supports over 1000 VM's sharing 100s of GB of object memory. GemStone is free up to 4GB of storage.

*Objects get better with age
*Object lifecycle
*create
*use
*save

During the Q & A at the end, Ola Bini asked Avi about using Rubinous. Rubinious is a runtime for Ruby that is actually written entirely in Ruby itself. I was impressed by the fact that Ola didn't ask what Avi thought about JRuby. Ola is really cool, and his question was right on target after Avi's criticism of Ruby being written in C, and not turtles all the way down. Avi's response included his opinion that the Smalltalk VM is so advanced it would take the Ruby community 10 years to develop, just like it took that long to write the Smalltalk VM.

Personally, I do not buy that argument. Just because it took them 10 years to develop their runtime engine doesn't mean that all such tasks will take the same length of time. But Avi's perspective is a bit more biased than Ola's, I think.

And then came Ze Frank...


Ze Frank

I like Ze's stuff, but I had no idea what we were in for. I knew he was witty and smart, but what I didn't know what that he was a total genius. Ze is to theory of online community what Why The Lucky Stiff is to Ruby. On the surface everything is very amusing, but if you dig in you can go deep. Very deep.

Ze started out with a hilarious bit on "Acceleration Anxiety". He had the crowd laughing so hard that tears were coming out, as he made fun of his own fear of flying, and the idiocy of so many everyday things like the safety instructions and vomit bags on airplanes.

And then, he told us about a second kind of "Acceleration Anxiety". Here are a few of his key points on socialogy and online community:

People are learning a new language (ours)

People are having a media conversation

What can you do with your audience if they do things you don't like
*ignore them
*control them
*shut them down

Facilitating conversation : cultivating the energy

At a really good dinner party, it about the focus of the experience. Most people asked would not remember what had been discussed, but they would have remembered enjoying themselves - all of the different kinds of guests need to engaged.

Ze then shared his history of how he started and become the net culture star that he is today. At first his communications were one-way. But then his audience started becoming participants. He showed us a few examples:

The show
Earth sandwich
Human baton

Complexity of narrative has become much more complex

Technology is a facilitator of basic human emotions

The Core Values
*Conversation
*Community
*Belonging

There are one to one online communication. And there is mss online communication. We we need to do is look to the middle.


All I can say is wow. During Ze's introduction, Chad Fowler said he didn't realize that Ze was like some kind of Malcolm Gladwell. Neither did I.

RailsConf 2007 - Sessions (Day 0)

Well, I had just got back to my room after day 0 of RailsConf 2007, which was the tutorial sessions before the main conference began. After a fun evening of pizza and beer with the Pragmatic Programmers, I thought I would try to get down some thoughts about the day's events.

Yesterday, I attended "Scaling Rails From The Ground Up" followed by "Harnessing Capistrano". It is very hard to have tutorial sessions at a conference. They either are too basic to satisfy advanced users, or too complex for noobs. In the case of RailsConf 2007, they probably achieved both at the same time. Since the Pragmatics had the intro to Rails thing locked up, the other tutorials probably should have gone way advanced. Friday's sessions will be shorter and more focused, so I expect to get more out of them for myself personally. Not that they were bad, but many of us who were at last year's conference were hoping for more advanced material.

The real action was on IRC, of course. Just before last year's RailsConf 2006, my old notebook's screen died. As a result I was not connected to the hive mind where the really interesting communication. This year, my still mostly shiny MacBook Pro served me well, by letting me experience the full RailsConf experience. You could certainly tell who was connected to #railsconf, from the snickers that escaped from the occasionally brilliant witty commentary.

Scaling Rails From The Ground Up
This presentation was given by Jason Hoffman of Joyent. His presentation did indeed deal with scaling Rails from the ground up. Litterally, he mentioned the kind of flooring that you need for a minimal data center (concrete, if you must know). His talk went from 50000 foot view to a close-up view of a blade of grass, but he left out much of what goes in between during the first part of this presentation. Yes, he told us what kinds of power connectors they like to use at Joyent.

Here are a few excerpts from his talk:

*Moving parts of your Ruby code that are extremely performance sensitive to C extensions to Ruby is a very viable optimization technique.
*Power is a real limitation for building out a data center. There are large physical data centers that have a lot of empty rack space, due to being unable to access enough power to ad any additional servers.
*Compared to DNA sequencing, the amount of data on the web isn't that much.
*Scaling goes in both directions...you should be able to either up OR down. From running your app on a USB key, to running on a large server farm.
*Look for weak points in the architecture, for example a powerful box that has an OS limitation due to number of files in a directory.
*Scalable means "Lego"-like adding or removing of elements while keeping the integrity of the system intact.
*Rails is just one part of the scaling solution, the rest of the stack that Rails is running on needs to be well understood and kept as simple as possible, as it is easy to over-engineer fast.
*Use other data stores besides just relation databases where appropriate. His example was using LDAP for your user authentication, J-EAI for a message processing bus, and Postgres for the relational data.

One thing that impressed me was the amount of Java technologies in the stack that Joyent is espousing. For a community that spends so much time dissing Java, a whole lot of integration between Rails and Java has occurred over the last year.

Jason told some pretty amazing stories about hosting and colocation providers that they dealt with in the past that went out of business in spectacular ways. I'm not sure which was my favorite, trying to buy back their own servers from a bankruptcy proceeding in Sweden, or the one about the baker's racks with normal white box PC clones stacked on them when they were being charged for nice rack-mount servers. BTW, it is only legal to use backer's racks for electrical equipment in Texas. Zap!

Overall, the session was about half details about how Joyent has built their data center out, and about half scaring people into using some hosting provider, like Joyent, for example. Of course, can you blame him? That is a big part of what presenting at conferences is all about! Trading some useful information in exchange for getting to do some sales pitching to a crowd of likely customers.

Harnessing Capistrano

Jamis Buck, the erstwhile Rails committer and creator of Capistrano nee Switchtower gave the next session that I attended, an overview of Capistrano, including the upcoming 2.0 release.

Although most of us are using Cap as a deployment tool, it can be used for many other interesting things. Some examples are:

Ad-hoc monitoring
*Query all servers at once
*Uptime
*Operating system version (uname)
*Currently running processes (ps)
*Free disk space (df)
*Reading logs
*Searching logs

Query database server status
Server Maintenence
*nfs mount
*symlinks
*pkg_add $home/imgmagic

Troubleshooting
Custom tasks
Clearing cache files
Grep log files

Capistrano Assumptions:
Capistrano makes certain assumptions that are requirements for its use.
*SSH
*POSIX system
*Public keys or identical passwords on all servers

Other Interesting Things You Can Do With Capistrano
Gateways - when you cannot access the server directly over net, you can use set :gateway to tunnel through SSH to target server
set :gateway, "capistrano.demo"

Invoking Remote Commands
You can invoke arbitrary commands on a server using "cap -e invoke", and this does not even require CAP file. One drawback of invoke is that Cap needs to reconnect on each invocation.

An alternative to this is the "cap -e shell" command. This is similar to invoke, but caches connections, and can execute both tasks and commands.

Cap 2.0 adds namespaces for tasks. This is very important if you were using a third-party plugin that had its own Cap tasks.

Variables
Cap variables can be assign values passed in when Capistrano is called. They can be loaded either before or after recipes are loaded. If loading vars after recipies are loaded, any values assigned in the

Transactions
Cap allows you to wrap one or more deployment commands in a transaction. If a transaction is performed on multiple machines, and fails, Cap auto restores the state of each machine on failure
Example:
on_rollback { task_to_perform_when_transaction_fails }

Deployment Recipes
Capistrano 2 allows you to opt out of deployment recipes, for custom deployment recipes, or for non-deployment uses. The default deployment recipes in Cap do have several assumptions:
*Application code in under source control
*The app os a web application (Rails)
*Production environment is ready (all servers are setup and running)
*Using standalone FCGI listeners

How to Cap 2.0 Your App
Switching to Capistrano 2.0 should not be difficult, according to Jamis. The basic process is:
*goto your application base directory
*run "capify ."
*setup a minimal capfile

There are a couple situations where you might have troubles when upgrading to 2.0:
*3rd party ext. and libs
*Overridden or extended tasks without namespace

As a result, Cap 2.0 has a compatibility mode, which provides tasks with old names they need.

Conclusions
Despite the similarity of Rake and Capistrano syntax, they are nothing alike underneath, nor is Cap intended to replace Rake. The new Net::SSH and Net:SFTP gems give Cap even more power as a remote scripting tool to handle not just deployment, but all sorts of time-consuming tasks. The benefits of using Capistrano increase the more machines you are trying to get your software deployed on. And if you are not using it now, what are you waiting for?

Packing For RailsConf 2007

I am packing my stuff for RailsConf 2007, and I am pretty excited. Last year's conference was the best event I had been to in years! The growth in the momentum and enthusiasm in the Rails community has only accelerated in the last year, and I am giddy at the prospect of more fun brain candy!

I will be blogging from the conference, but not moment to moment like some quicker multitaskers. No, I prefer a more Christian Science Monitor approach, giving a complete picture of relevance sometime after events have taken place. Well, I will try to stay focused, but relevant or not here I come.

I was going to post my planned schedule for events I will be attending, but so is everyone else right now, and Dr. Nic's MyConfPlan is down at the moment.. Also, I will at Ezra'a Merb hackfest, and any parties I get invited to. Hint hint, all you companies hoping to sell me things, or sell me about things...feel free to try to influence me by buying me a beer.

The Robotic Revolution

The upcoming RailsConf 2007 is not the only revolutionary conference that is going on this week. At that epicenter of robotics and artificial intelligence research Boston, MA is the RoboBusiness Conference and Expo featuring everybody who is anybody in the biz such as iRobot.

CNET has a pretty good interview with Rodney Brooks, the CTO of iRobot. Despite the popularity of the Roomba, the robotics revolution has hardly begun. Brooks makes the interesting observation that not only will we see more autonomous robots wandering around like the Roomba, but even more embedded robotics like the self-parking feature on the Lexus.

Ultimately, he sees a cyborg society, where we are merged with our robots. It's a rather utopian vision, like that of Ray Kurzweil and the Singularity. I am also an optimist...at worst, the robots will keep us as pets.

Why I Haven't Posted Anything Lately...

I have been enjoying some real world fun with my wife and son on the north shore of Hawaii the last few days, and so I have not had time or inclination to post anything.

But I am back in action now, and there will be lots of high-tech fun at RailsConf next week.

Oh, but island life is beautiful. I already miss it...

Become An Initiate Of The Bayesian Conspiracy

Jeff Atwood at the Coding Horror blog links to a great introduction to Bayes' Theorem.

If indeed as has been claimed "Google uses Bayesian filtering the way Microsoft uses the if statement," we should all be well versed in the work of the mysterious Reverend Bayes...

I have been playing around intermittently with an interesting application of Bayesian classification for a couple of years. Perhaps this will inspire me to actually finish it up enough to put it up on the web.