RubyConf 2008 - Day 1 - Testing Heresies - Francis Hwang

The sessions for RubyConf 2008 day one were nearly over, when I attending this highly entertaining and informative one. I had missed the chance to see a previous presentation by Francis Hwang, and being as test-obsessed as I am, I was not going to make the same mistake twice.

Many people are confused over mocks, stubs, and other fakes. Here are the real definitions.

Gerard Meszaros - Test Doubles
- dummies - passed around but never used
- fakes - work but unstable in production
- stubs - provide canned answers to calls made during test
- mocks - mostly concerned with surface interaction

Classical TDD use real objects.
Mockist TDD will always use a mock for any object with interesting behavior

Problem #1 with mocks
Cannot figure out problem and solution at same time

Problem #2 with mocks
Bugs can slip between layers

Francis Hwang, TDD Clasiscist
- test setup creates records in the db
- no sqlite in test
- when you use test doubles, it's always a big deal

Real world example #1
Ensure that an optimization is actually being used
Redefine the actual surface interaction to 'scream' on error

Real world example #2
Faking a real web service (not written by us)
Best to use stub, to duplicate functionality
Need test, stub, AND test for stub

Dogma - if you tests do not run quickly, there's something wrong with you
Heresy - I'd like my test to run 30 sec. I'd also like a pony

Francis does not use autotest, because he does not want to run all of the tests every time he changes something. Eric Hodel chimed in from the audience says that it can run only what changed. Only if it passes, will it run ALL tests. Me, I loves my autotest... with growl notifications, it's all good.

devver
They say it is a ec2 based distributed test system...

testjour
uses bonjour to trigger a farm of machines in your LAN to perform distributed tests

Dogma - you code should always be well-structured
Heresy - under-structuring can be freeing

Uses example of constantly cleaning small NYC apt. It was living on a boat, where you spent all your time putting things away.

Dogma - Fine grained focus leads to high quality code
Heresy - Never forget the unknown unknowns

System design 101
How likely fail
What will fail look like
How can I find out quickly
How can I limit damage

Dogma - testing makes my job easier
Heresy - testing makes it easier for others to do my job - unless my job keeps growing

Rails has made it so simple to implement simple business logic, so way more is expected now.

RubyConf 2008 - Day 1 - rush, a shell that will yield to you - Nicholas Schlueter

I had previously blogged a bit about Rush, the Ruby shell. Well, now Rush must now be its own force to be reckoned with, because this presentation was NOT by Adam Wiggins, the creator of Rush, but was instead by Nicholas Schlueter, the creator of RushMate. If you've not heard of it, RushMate allows you to use Rush from within TextMate, which translates to being able to write custom commands much more easily.

He showed some neat examples, including one that auto-generates an HTML page with links to all of the RDocs for all of the RubyGems you have installed locally. I have wanted something like that for times that I was not able to connect to the InterTubes, but still wanted to do some development.

The entirety of his presentation is online here with a whole bunch of code examples that will be better than my ramblings, so you should rush over and check it out.

RubyConf 2008 - Rubinius - Evan Phoenix

The Ruby VM wonkiness continued at RubyConf 2008 - Day 1. Next up we heard the "State of Rubinius" address on the status of the anointed heir to the Ruby federation. Evan Phoenix is a fun guy, and it proved to be a both informative and enjoyable session.

Evan says Rubinius is "meta-circular-ish", and slow, but getting faster. They have a new VM that is written in C++, which is a sharp tool, and as such, it is possible to poke your eye out. However, he says it allows them to better model work. The three main advantages are:

- type safety
- organization
- architecture

What is type safety, may ask many Rubyists. Well, sometimes a duck actually IS a duck. The Rubinius code just feels better with OO nature of C++, as opposed to just plain C. Also, it is possible to make the VM class hierarchy that mirrors the Ruby class hierarchy.

Exporting Methods - a day in the life of a primitive operation

- things you cannot do in Ruby
- small example

class String : Object {
  Fixnum* size();
}

class String
  def size
    Ruby.primitive :string_size
  end
end

There was interesting question about why flatten out ala :string_size aka no namespacing. Evan did not seem to give a definitive answer...

Method Dispatch - From resolution to execution

Resolution
- going from receiver and method name to actual method
- three mechanisms
- hierarchical lookup
- global cache
- inline cache

Execution
- every method provide an execute function pointer
- each primitive is an executor
- Ruby methods use executor specialization

Ruby Methods
- Specialized executors based on argument info
- Very simple fast version for majority of cases
- Slower fallback case for all cases

Critical Path
- Populate message object
- Call rep over trampoline
- Fill in more of message object
- Call method executor

Execution/Framing
- Method context objects store info about each method
- Chained together using refs (spaghetti stack)
- MethodContext creation speed is crucial

- Default contexts are allocated in special section of memory
- Normal execution follows simple stack pattern
- We can exploit to allow for fast creation and deallocation

Calling to C
Can you imaging life without ruby gems that use C extensions? For example:
- hpricot
- mongrel
- mysql
- sqlite
Not a problem. A simple recompile == usable in Rubinius

Tricky
- GC interaction requires indirection
- Uses 'quantum leap" stack jumping
- Allows for semi-graceful recovery from extension segfaults

Performance
Rubinius is doing well on micro-benchmarks
- method dispatch
- GC
- OK on macro-benchmarks
- Poorly on mega-benchmarks

Solutions
- LLVM
- JIT
- AOT
- Improve algorithmic efficiency

Rubinius is concentrating on compatibility right now, but will then address performance. Just like Koichi-san said, it is a curve of improvements.

Evan ended up with a nice quote from Woody Allen - "If you're not failing every now and again, it's a sign you're not doing anything very innovative."

RubyConf 2008 - Future of RubyVM - Koichi Sasada

The first session I attended at RubyConf 2008 was called "Future of RubyVM", and it began my ascent into "VM Wonkiness" as I started to call it. Koichi-san is responsible for leading the team of Ruby 1.9 implementers, and is the authoritative source of info on its status. I was hoping to learn a few things, as well as to run into new acquaintance Nobuyoshi Nakada, one of the core team members.

Techniques for VM Performance
- Simple techniques
- C level VM

- Advanced techniques
- Dynamic code generation
- Native compilation
- JIT compilation
- Polymorphic inline cache
- Selective inlining

- Online feedback opt.
- Hotspot JIT
- Tracing JIT

Pros/Cons of JRuby/IronRuby
- Using an awesome VM
- Pros
- Many clever people working on the project
- No code is good code
- Many libraries already exist on each environment
- Easy to use parallelism
- Cons
- Not only focused on ruby, so there is a semantics gap.
- Cannot use C ext direct

Pros/Cons of Rubinius
- Most code in ruby
- Pros
- Ruby in Ruby
- Says this is the *best way* to improve performance in the long run because we can easily analyze and make specific performance improvements.
- Cons
- Still a LONG way to get a high performance VM

Pros/Cons of C Ruby
- Pros
- Portability, using GCC
- C is well known language already
- Extensibility
- Performance improvement
- Easy to write simple extensions.
Cons of C Ruby
- Extension libraries are written in C
- GC problem
- Inlining problem
- Limitations on program analysis

Our Performance Policy
- CRuby is not best solution, but it is a GOOD one
- They continue to improve the CRuby implementation
- It is a pragmatic, practical selection
- It will be the main implementation still for at least several years

Keywords for Success
- Embedding
- Parallelism

Research
The team is working on taking advantage of C some projects are running
- Hidden optimization techniques on YARV
- Ricsin

Hidden Optimization Techniques
- turned off in 1.9.1 by default
- Tail call optimization
- Optimizing using unification
- Stack caching

- Left easy optimizations
- Efficient method caching
- Efficient fiber implementation using platform dependent way suvh as makcontext()
- These optimizations will be merged in 1.9.2

Ricsin; Mix-in C Ruby
- Embed part of a C program into ruby
- Like a RubyInline, but directly embedded
- Usage
- Use C libs direct
- Replace all built-in classes and methods
- Test Ruby C API
- Continuous performance improvements

Ruby to C AOT Compiler
- Translate Ruby scripts to C code ahead of time

Related work
- ruby2c
- yajit
- yarv2llvm

Atomic-Ruby project
- Issue: ruby is too fat
- EMBEDDED
- Embedded system can have special problems, such as resource limitations
- Application embedded ruby
- Some applications need just a Ruby DSL engine, not the full ruby distribution.
- We need a slim Ruby interpreter
- Utilize CRuby portability
- 3 subprojects
- Plugin/out
- Precompilation
- Swift core features

Auto-write barrier detection
- write barrier needed for several GC algorithms.
- automatic WB detection system

Generational GC
1 but ref count

Multi-VM (MVM) Project
- Multi VM in 1 process
- Watch runs in parallel
- High speed inter VM communication
- Sponsored by Sun

Summary
- CRuby/YARV is not "best solution", but it is best for right now. But eventually, Rubinius is the future. Which just so happened to be the next session I attended...

RubyConf 2008 - Day 1

The crowd of over 400 was happily awaiting the kickoff in Orlando of RubyConf 2008. After brief introductions from Rich Kilmer and David Black, Matz took the stage for his opening keynote.

Opening Keynote - Matz

Reasons Behind Ruby
8 years ago was the 1st International Ruby Conference, and there were only 33 people there. Obviously since then, the community has grown. Matz said he was not going to talk about technical stuff.

Question - Why Ruby?
Matz joked that it is his beloved masterpiece. Really though, Matz loves languages, and also loves freedom. His reason for creating Ruby was to maximize freedom.

There are still a few problems. Ruby is imperfect, slow, lacks features, and is complex. And there's more. Ruby is inconsistent, has a poor implementation, has memory issues, and embedding issues.

So why do we love Ruby? "Programmers Strange Love, Or How I learned to Stop Worrying and Love Ruby"

The greatest Reason behind Ruby is Love. It made programming fun again. Again? It used to be fun.

In 1980, Matz began programming in BASIC. He liked that he could make intelligence, and felt a sense of creation. But there was also great pain.

The Basic Aristocracy
There was a big difference between language designers, vs. lowly programmers. In BASIC you cannot add to the language, and there is no abstraction.

Lisp
Everything is opposite in Lisp from BASIC. In Lisp, there is no discrimination and extreme abstraction.

As Eric Raymond said, "Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days..."

However, Lisp was great in a book, not in reality. Programming in Lisp did not make him happy. Was it all the parentheses? No. Was it the macros? Partially.

It came down to the fact that smart people just underestimate the "ordinarity" of ordinary people.

Aristocracy is OK, as long as I have POWER. Matz wants to be between "owelty" and freedom (none of us in the audience knew what he had meant by "owelty" either, but Matz was on a roll, so we went with it).

BASIC gives you no power, Lisp gives you all power. But at both ends we lose popularity. If you stay in BASIC you are a coward, and if you go extreme, you are brave but you risk your life (or popularity).

We all love power. You see, policy matters. We need philosophy and attitude.

The community cannot live by love alone, power matters.

Then along came Rails which turned Ruby into a web DSL. You see, Ruby is about vocabulary. It has rules, or syntax. There are others, for example Rake and RSpec.

As Dave Thomas said - "Programming is a process of designing DSL for your own application"

Ruby is really like a meta-DSL. It provides a foundation for DSL's, which increases our overall productivity.

But beware of commercial success, because they will try to take your freedom away.

Now there are better implementations or Ruby than MRI - YARV, JRuby, Rubinius. Matz says that the various implementations are competing, which is a good thing.

Matz then mentions the 4X growth expected in Ruby programmers from the current 1 million to over 4 millions. There will be NEW people coming into the community, so welcome them and nourish them. Ruby can have the power to raise up people in the programming field, so feed them, and love them.

He loves us all.

We love you too, Matz.

Sessions - Day 1

I attended a number of interesting sessions throughout RubyConf 2008 day 1. Per previous reader feedback (keep each post to a manageable size!), I will put each into a separate post. Here is what I will be covering:

Future of RubyVM - Koichi Sasada
State Of Rubinius - Evan Phoenix
rush, a shell that yields to you - Nicholas Schlueter
Testing Heresies - Francis Hwang
Lightning Talks

I missed a few of the lightning talks while having a nice dinner with David Black, Jim Freeze, and Paul Brannan. Hanging out with really smart people is like a continuous lightning talk, so I didn't feel deprived by any means. Thanks, guys!

After the lightning, as it were, we had the second ever Ruby Jam Session. This time, about 5 people brought guitars, including Ruby Jams veterans like Jim Weirich and Forrest Chang, and lots of new friends showed up as well. I was better prepared this time around, and had brought about 10 harmonicas with me. But the tastiest part of the jam, was at the very end when David Chelimsky and Diego Scataglini played some really great jazz standards. A couple of tasty beers followed the music, and then it was off to rest up for RubyConf Day 2.

My Vote Was Easy

Despite a massive line of people waiting for the polls to open this morning, when I went to vote just now I discovered... no line whatsoever. What a letdown! I wanted to do some sacrificing like hard struggling adherents to the cause of freedom the world over have done, just to get to the polling place, let alone cast their vote. But no, everything had to go quickly and smoothly.

After accepting my offering in Halloween chocolate (stolen from my son, I might add) the friendly poll-workers ushered me to my individualized voting booth, and afterward counted my vote extra-specially well before bidding me farewell.

Democracy is great, and kudos to the people that humbly and with no applause watch over our freedoms at the ground level... your friendly neighborhood poll-workers.

If you haven't done it yet... go! Now!

I Think I'm Comatose

This last weekend, I upgraded a pretty complex application to the latest Ruby on Rails, and consequently had to upgrade a bunch of the plugins. In the case of Comatose 2.0, this was well timed, as there is a new release.

If you have never heard of Comatose, it is a very cool mini-CMS that you add to your Rails app as a plugin. For cases where you have an applcation, and you want to put the 'static' pages under the control of the users, while still integrating with your application, Comatose is the best option I have discovered.

If you are just discovering Comatose, just go check it out. Now. If, on the other hand, you are running into any problems going to the new version, read on.

Since this application had been running the 0.8 version of Comatose, I ran into a little problem, which has been documented on Google Groups here.

If you are running the older version of Comatose, you have a migration that contains this:

 module Comatose
   class Page < ActiveRecord::Base
     set_table_name 'comatose_pages'
     acts_as_versioned :if_changed => [:title, :slug, :keywords, :body]
   end
 end

The new version of Comatose, however requires this:

 class ComatosePage < ActiveRecord::Base
   set_table_name 'comatose_pages'
   acts_as_versioned :if_changed => [:title, :slug, :keywords, :body]
 end

What I did was, go and change the original migration to match the newly required format. This will allow a new clean installation to work correctly. Then, to handle upgrade installs, I created a separate rake task to perform the migration steps as described here.

Voila. Now this app is happily running the latest Rails, Comatose, ActiveScaffold, ActiveMerchant, and a few other plugins too. Life is good.