People are communicating beings

I ran into this older article from Alistair Cockburn the other day called Characterizing people as non-linear, first-order components in software development.

What Dr. Cockburn is talking about, is that very agile of ideas that people are the central focus for any software development effort. Developing software is not just a sausage factory of requirement documents, source code documents, and unit test plan documents. The people are what is important, not the documents.

Deep within it is a fantastic section called "People are communicating beings". In it, Dr. Cockburn says that "The most significant single factor is 'communication'", and presents an informal chart of the efficiency of communication that he uses to "inform my methodological considerations":

This chart shows that real-time, face to face communications is the most efficient. Conversely, written documents are the least efficient format.

The article goes on to say:

If this model is useful, it should inform us as to how better to work. Indeed it does, and we find the busiest projects making use of those suggestions.

'Put all the people into one room.' 'Don't give me more than 4 people, that's all I can get into one room and talking together.' These are standard recommendations of successful project leaders. They plan on using the highest communication mode, people face-to-face...'Make sure there are whiteboards and coffee corners all over the building.' Hewlett-Packard and IBM were early to observe the effectiveness of informal meeting places, and by now it has become a standard idiom in our industry that an effective design environment actively encourage and permit ad hoc meetings of small groups. Weinberg documented specific instances where casual conversations had significant effect on the overall group output [Wei]. Much progress comes when people 'just talk together.'.

A colleague of mine argued on behalf of written documents, and in his case, he took a very conventional view and insisted that the documents were the most important thing to "provide a record of what happened". Dr. Cockburn does not dismiss the need, and addresses this in an excellent manner:

The above model also allows us to make a recommendation for archival documentation:

Have the designer give a short (5-20 minute) description of the design to one or two colleagues who are not familiar with the work. They will act as ombudsmen for the viewers of the videotape. Let the people simply have a discussion of the design, with the colleagues asking questions as they need. Video the discussion. At the end, produce drawings of the examples used in the discussion, or the design drawings used, to act as mnemonic anchors of the discussion.

I was pleased to hear from Lizette Velasquez of Lucent Technologies that not only had she profitably already used that technique, but that I had forgotten to mention something important. She said it is also important to mark and index places where 'something interesting happened'. While much of the discussion proceeds at a relatively slow pace, occasionally a question triggers a flurry of significant discussion, and the viewers will want to refer back to those sections.

These days, it is possible to put the talk online with hyperlinked media.

For those who still think a book is best, consider the excellent but difficult book Design Patterns. Imagine that instead of trying to extract the meaning of the 'Decorator' pattern from the paper, you could click on the page and see one of the authors explaining the pattern in a video clip. They would, of course, rely on tonal inflections, gestures, and timing to get the idea across.

The lesson of this human characteristic is that we should try to move team communications up the curve as far as possible, given the situation at hand.

As I said, Dr. Cockburn does not dismiss the need for archival documentation, he just puts it in its place. Thank you, Doctor! I'm just glad I have some of your wisdom on video.

Programming Language Family Tree

My colleague Michael Hocter just sent me a great link to a diagram showing the history of the evolutions of the major programming languages. I just love this kind of visual representation of information!

Note how C# has taken ideas from Java...then Java goes and draws inspiration from C#. Also note how C# 2.0 takes its inspiration from Ruby. The two programming languages that drew from the most other languages (five) were Oak, which turned into Java, and Ruby.

What To Do When The Flu Comes For You

My dear friend and former colleague Dan Rasmus has some really important points to make regarding IT operations and emergency preparedness in his blog entry Work Interrupted - Rapid Response in a Crisis.

Thank about how many warnings we have already had about potential pandemics. If it hits, it will be too late to make emergency plans...we will need to be carrying out those plans, not talking about what they should be!

I have a supply of masks, gloves, bottled water and canned food at my home. Do you? I hope I never need them...but I have them.

Beating a Dead Horse

Last week, there was a very funny post from the extreme programming mailing list, published here.

This is quite emblematic of what dead programmers see every day, as we are often ordered to ride dead horses...

We need to add a couple to the list:

- Offshore the dead horse so we can ride it 24 hours a day
- Mandate all riding be done on site, so we can keep an eye on our dead horse

The Folly of Accountabalism

The Harvard Business Review has published their list of 20 breakthrough ideas for 2007. There are several ideas on this list with particular appeal to me. One that stands out is David Weinberger's "Folly of Accountabalism" which insists that "Accountability has gone horribly wrong. It has become 'accountabalism,' the practice of eating sacrificial victims in an attempt to magically ward off evil."

Dr. Weinberger is referring to the emphasis on corporate management and compliance that has come to dominate modern American business. Software development within corporate IT departments are just as bound by this kind of thinking as marketing or sales is. Weinberger argues that this manifests itself as several common practices.

"Deciding that the problem can be solved at the next level of detail - looking at complex systems that have gone wrong for complex reasons and writing another set of work procedures, and printing up more forms." A common approach with software development organizations is to try to become too abstract, creating a solution to every possible problem in advance.

"Assuming perfection - if anything goes wrong, it’s a sign that the system is broken." This does not allow for the "good enough" solutions that allows for the mismatches between what we can do, and what we can do now.

"Being blind to human nature - by overly formalizing processes, accountabalism refuses to acknowledge that people work and think differently. It eliminates the human variations that move institutions forward and provide a check on the monoculture that accounts for most disastrous decisions. It also makes work no fun." Need I say more!

"Bureaucratizing and atomizing responsibility - While claiming to increase individual responsibility, it drives out human judgment. When a sign-off is required for every step in the work flow, those closest to a process lack the leeway to optimize or rectify it." I was just following orders...we know where that road leads.

No wonder so many organizations try to force all of their software development processes into a waterfall! It just fits perfectly within the practices that have already fallen into due to this overemphasis on idealized management, however wrong minded it may be.

Fabbers Gone Wild

Why stop at fabbing a chocolate bar when you can just fab your own whole factory?
According to Behrokh Khoshnevis, a USC Engineering researcher, "The goal is to be able to completely construct a one-story, 2000-square foot home on site, in one day and without using human hands."
Seems like a scene from Manna to me, doesn't it?

One Man Army

The continued rise of the individual as a viable economic unit is being covered by several bloggers, starting with Paul Kedrosky and continuing with Tim O'Reily.
The trend in human societies has been toward increasing decentralization. We see this in terms of computers (70's = mainframes, 80's = PCs, 90's = Internet, 2000's = Wireless Internet), as well as in terms of business (70's = corporate HQ, 80's = independent business units, 90's = spin-off independent companies, 2000's = supply chain integration). The numbers show that the economic impact of this has become HUGE, at 70% of US businesses.
The difference now is that individuals can impact beyond just their local geographic area. No longer just being limited to the corner store, individuals can reach beyond their neighborhood or city, to play within the broader global market.

Speaking The Same Language

The art of successful software development is matching the delivered software to the needs of the users. Achieving this requires that the developers and the users speak the same language regarding the problem to be solved. James Shore has a new chapter called Ubiquitous Language in his upcoming agile development book that summarizes this idea nicely.
Martin Fowler's seminal article on Language Workbenches also has really influenced my thinking. I guess the difference to me is that where Ubiquitous Language is about getting the developers to understand what the users are talking about, Language Workbenches are about building software systems using tools to translate the domain specific language into runnable code.
The foundations of these ideas have to do with human communication as much or more than they have to do with technology. For decades, good consultants have been taught "learn to speak the customer's language". The movement behind domain specific languages is just a formalization of something that has been going on for a long time. I'm pretty sure that Peter Drucker was doing something like this back in 1954 or so...just not using computers.

The Anti-Team

Jeremy Miller and Ayende have each captured some hysterical and yet poignant slices of life from the depths of the seventh pit of hell...or your typical IT project team, as most of us more commonly know it.
If you are guilty, you know who you are. If you are innocent, then you haven't been working very long in this game...

When We Were Fab

A fabber is a machine that can make three-dimensional objects just like a printer can print in two-dimensions on a a piece of paper. Only recently had this technology escaped the realm of science fiction into the still unreal world of government and large-budget research labs...which is ironic given that the purpose of fabbers is to produce highly specialized items at very low cost.
Well, thanks to an open-source hardware/software project called Fab@Home, the technology is reaching out to the home enthusiast. You can now make anything you want, as long as it is constructed of silicone, or perhaps chocolate.

Optimus Mini Three is Prime

Time again for another incredibly cool looking device... the Optimus Mini Three Keyboard. The description on their site says "Optimus mini is an auxiliary keyboard with three keys, each complete with an OLED screen displaying the current function."
They call it a keyboard, but I think that is quite the understatement. It is really like an external display surface that you can press. Drivers for Win32, MacOS and Linux are available. The software allows you to configure each of the three mini-screens to display the results of a "plug-in". Plug-ins display whatever they want, and there are already plug-ins that grab data from commonly used applications like iTunes to display summary info on the tiny screens.
Imagine your continuous integration system, server farm health, or stats on your plans for world domination displayed on three tiny screens. If the indicator shows something interesting, the display is actually a button that can launch your desired application. How cool is that!

Ruby Domain Specific Languages - The Basics (Part 4)

In the previous three entries in this series of postings, I have been exploring the basics of creating domain specific languages using Ruby. Since I cannot disclose details of my big financial service client's DSL, I am using this made up example to illustrate the techniques that are useful when building your own Ruby DSL. So far we have created a "PetShop" domain that allows us to interact with the Pets. Defining the behavior of new Pets is the purpose of this DSL.

For example, let's say we now want each Pet to be able to do some trick, when called on to do so. When you are defining behaviors, it is very useful to define a recipe-like syntax in a DSL.

Here is an example:

 pet "Toto" do
  when_performing_tricks do
   sit
   speak
  end
 end

This is just a description or recipe of what the animal is supposed to do when asked to perform it's trick. When we really want our Pet to perform, we simply say:

  pet.perform

This produces the following output:

Toto will now perform...
Toto is sitting...
Toto says 'Woof!'
Let's hear some applause for Toto!
Slugworth will now perform...
Slugworth is sitting...
Let's hear some applause for Slugworth!
Tweety will now perform...
Tweety is sitting...
Tweety says 'I thought I saw a putty tat!
Tweety is flying...
Let's hear some applause for Tweety!

The Pet will do whatever tricks it knows, using this little simple bit of Ruby magic:

 def self.when_performing_tricks(&routine)
  @routine = routine
 end

 def perform
  puts "#{name} will now perform..."
  @routine.call
  puts "Let's hear some applause for #{name}!"
 end

The "when_performing_tricks" method simply stores the block, and executes it only when the time comes to "perform".

So how does the Pet know what is entailed in each trick? I have put the "sit" trick into the base Pet class (every Pet knows how to sit):

 def self.sit
  puts "#{name} is sitting..."
 end

We can also more importantly define custom tricks per type of Pet. A simple bit of Ruby metaprogramming goodness helps us out:

 def self.define_trick(name, &trick_definition)
  singleton_class.class_eval do
   define_method name, &trick_definition
  end
 end

The "class_eval" methos lets us evaluate the inside expression in the context of the class object, not just a particular instance of the object. Another bit worth noting is the helper method that I have added to the DSLThing base class called "singleton_class" that looks like this:

 def self.singleton_class
  class << self; self; end
 end  

This is just a short cut to get to the class instance object, known better to Rubyists as the "singleton_class".

Lastly, the "define_method" method then lets us define a new method for the trick. When we want to define a new trick for a particular Pet, we can simply define it like this:

  define_trick "speak" do
   puts "Toto says 'Woof!'"
  end

Putting it all together, here are our new definitions of the tricks our Pets can do:

 pet "Toto" do
  when_performing_tricks do
   sit
   speak
  end
  
  define_trick "speak" do
   puts "Toto says 'Woof!'"
  end
 end

 pet "Tweety" do
  when_performing_tricks do
   sit
   speak
   fly
  end
  
  define_trick "speak" do
   puts "Tweety says 'I thought I saw a putty tat!"
  end
  
  define_trick "fly" do
   puts "Tweety is flying..."
  end
 end

 pet "Slugworth" do
  when_performing_tricks do
   sit
  end
 end

One other interesting technique of note is that we are actually defining a new type of Pet by dynamically declaring a new class based on the Pet class. Otherwise, our custom tricks for one type of Pet might interfer with the custom tricks of another. The solution to this is using the "Object.const_set" and "Object.const_get". Here is the code I used:

 def self.pet(name, &blk)
  @pets ||= Hash.new
  klass = Class.new(Pet)
  Object.const_set(name, klass) if not Object.const_defined?(name)
  p = Object.const_get(name).new
  p.name = name
  p.class.class_eval(&blk) if block_given?
  p.copyvars  
  @pets[name] = p
 end

In this posting I have used the DSL recipe technique, and created dynamic methods and classes. Combining these techniques can allow for a very powerful and yet concise syntax when creating your own Ruby domain specific languages.

Here is the complete listing of the code from this post:

class DSLThing
 def copyvars
  self.class.instance_variables.each do |var|
   instance_variable_set(var, self.class.instance_variable_get(var))
  end 
 end
 
 def self.singleton_class
  class << self; self; end
 end  
end

class PetShop < DSLThing
 attr_accessor :pets, :people
 
 def self.create(&block)
  f = PetShop.new
  f.class.instance_eval(&block) if block_given?
  f.copyvars  
  return f
 end
 
 def self.pet(name, &blk)
  @pets ||= Hash.new
  klass = Class.new(Pet)
  Object.const_set(name, klass) if not Object.const_defined?(name)
  p = Object.const_get(name).new
  p.name = name
  p.class.class_eval(&blk) if block_given?
  p.copyvars  
  @pets[name] = p
 end
end

class Animal < DSLThing
 attr_accessor :name
 
 def initialize(name=nil)
  @name = name
 end
end

class Pet < Animal
 def initialize(name=nil)
  @name = name
  super
 end
 
 def self.when_performing_tricks(&routine)
  @routine = routine
 end

 def self.define_trick(name, &trick_definition)
  singleton_class.class_eval do
   define_method name, &trick_definition
  end
 end

 def perform
  puts "#{name} will now perform..."
  @routine.call
  puts "Let's hear some applause for #{name}!"
 end

 def self.sit
  puts "#{name} is sitting..."
 end
end

shop = PetShop.create do
 pet "Toto" do
  when_performing_tricks do
   sit
   speak
  end
  
  define_trick "speak" do
   puts "Toto says 'Woof!'"
  end
 end

 pet "Tweety" do
  when_performing_tricks do
   sit
   speak
   fly
  end
  
  define_trick "speak" do
   puts "Tweety says 'I thought I saw a putty tat!"
  end
  
  define_trick "fly" do
   puts "Tweety is flying..."
  end
 end

 pet "Slugworth" do
  when_performing_tricks do
   sit
  end
 end
end

shop.pets.each_value do |pet|
 pet.perform 
end

Rewrite Of No Return

I am really into Kevin Barnes' article "To Rewrite Or Not To Rewrite". He is on the right track about the challenges and tactics for a rewrite of any complexity. Chad Fowler is also exploring the big rewrite, and has fun things to say in a whole series of postings.
Rewriting a system too soon can be a waste of resources. One of the points of SOA is to allows services to be rewritten as needed, not just because a cool new technology can come out, but because there is value to be gained from doing so. For example, one of the best reasons to rewrite a system in a new language is if that the cost of extending the rewritten system will be less than the cost of maintaining the old one.
However, another factor is that near the end of the lifecycle of any system, a time comes when a rewrite may no longer be possible, without impacting the current operations of the existing system. Once you have reached a "development box canyon", your options are limited, and all of them painful. What do you mean, "we", kimosabe?