First off, his name is pronounced “Coh-burn” so let’s just get that out of the way now ;-) I finished Agile Software Development tonight and wanted to give a book report here while it’s still fresh to edify some of the points that resonated with me. This will probably be a lengthy post but it’s ripe with good concepts for anyone who manages a team of developers.

The term “AGILE” refers to a style of development that was coined by a committee of seventeen software architects in 2001 at a summit in which they drew up a document they named The Agile Manifesto. It proposed a value system that encouraged certain activities over others in the pursuit of creating flexible, well-constructed software. While Cockburn’s book is relevant to anyone involved in the software development process (engineer, QA person, manager), it probably makes more sense to be digested by a team lead and disseminated to the other developers. With so much reading to keep up with on various techniques, methodologies and languages, a programmer in a team would be better served by devoting her “extracurricular cycles” to material with more tangible, immediate rewards and receiving this type of abstract, work-style advice second-hand via a senior developer. If you’re a lone-wolf developer you may not find this book useful at all but if you deal with other developers in any capacity (internal or external to your organization), it contains some valuable lessons. In short, it gives a holistic, high-level overview of the qualities of methodologies and environments that are used by successful teams. And this is the strongest feature of the book: the fact it is entirely rooted in real experiences. It makes ample use of non-fiction anecdotes to make concepts more memorable and it draws heavily upon hundreds of interviews with real teams that deliver real software under real deadlines. The theories conveyed in technical books can sometimes be refined to the point of “PhD purity and correctness” yet be impractical in application. Cockburn instead offers “blunt, field-tested tools” that are actually getting things done everyday on the front lines of software development. Here are the main takeaways I extracted from this book:

1. Agile Manifesto – as short and simple as this document is (read it – 15secs), it has a profound message- basically, when push comes to shove, delivering good software under deadline is the ultimate goal, good communication amongst a team trumps process and documentation every time, and the only thing consistent across all projects is change and therefore your development style had better be the flexible reed and not the brittle stick when it comes time to deal with change. This manifesto sets the tone for the entire book, the chapters are filled mostly concrete practices that support the Agile manifesto.

   My Takeaway – Prior to buying this book I had heard Alistair
   speak on ITconversations.com . He struck me as being very practical-minded and not high & mighty on theory (which I liked). Having worked informally under both styles of development, I agree with the main values of Agile for being adaptive and goal-focused rather than process-oriented. Agile encompasses a bunch of formal styles like XP, SCRUM, DSDM and others – the tenets are very similar in all though. I especially related to a story about a team that was given strict requirements and micro-managed on the process and documentation required and they ended up ignoring orders and doing it the way they knew it needed to be done in order to get it delivered on time. As so often is the case for developers, you juggle what the client “tells you they want” vs. what you know they truly need and when pressed to favor one over the other, you have to diplomatically choose the latter and make it look like you choose the former.

2. “Microtouch Intervention”: big results from small changes in each member – This is not rocket science but enacting a very small change in each person of a large team can yield massive results with very little effort. It’s the idea that if the project as a whole is like a giant boulder that each team member is trying to move in a certain direction (towards project completion), the vector of force for each person is always somewhat misaligned from the desired direction such that people end up inadvertently wasting some amount of energy pulling against each other. Any steps that can be taken to conserve this energy by: a) communicating the intended direction more clearly b) revealing to each participant their actual vector of pull or c) producing a more compelling reason for people to pull towards the goal, will ultimately yield major gains.
   My Takeaway – This positive effect of this concept is obviously amplified in larger teams. Conversely, the larger the team, the easier it is to dodge responsibility and the more inertia that a stationary group will have. I’m a big proponent of this idea of “paint the vision clearly and then get out of people’s way.” I’ve always thought that energy expended cultivating the enthusiasm and passion of a group is a good buy (ie. if you can get the “WHY” part firmly planted in people’s hearts and minds, the “WHO WHAT WHERE WHEN HOW” questions will find a way to answer themselves). Cockburn validates this notion with a wonderful story from Dee Hock of Visa (second section from the end) and how this type of laissez faire leadership and simple microtouch intervention established a virtuous circle amongst his team and enabled them to successfully deliver a complex system under an ambitious sixty-day deadline. Just read this sentence- “Leaders spontaneously emerged and reemerged, none in control, but all in order. Ingenuity exploded. Individuality and diversity flourished. People astonished themselves at what they could accomplish and were amazed at the suppressed talents that emerged in others.” That’s the flavor of a team operating at peak performance driven by intrinsic motivations from within and that’s what you want to engender.
3. “Erg-seconds” – the energy/time required to transfer important info
   The idea of “erg-seconds” is an analogy Cockburn makes between the transmission of critical information in a project environment and fluid mechanics, or the dispersion of gasses and liquids in a container. In a workplace at any given time there are “convection currents” of info that are venting info like geysers amongst the team members. Anything you can do to narrow the distance this information must travel to reach the necessary recipients pays off exponentially in progress, morale, fewer errors, etc. Mole-meters/sec is the metric used in the realm of fluid mechanics, but that unit does not directly translate over since distance traveled is not the key factor whereas “time necessary to achieve clear communication” is. The optimal situation is to have the people that require the most back-and-forth communication sitting in the same room so they can tap each other for help and overhear and interject when they have valuable advice to contribute. Overkill of this setup creates what Cockburn calls “Drafty cubes” or too much interference with irrelevant info bombarding the programmer. Like anything it’s a balance that must be optimized. Moving further down the continuum of proximity, adding cubes, walls, floors, buildings or even separate geographical locations into the mix gradually introduces more and more friction that adds “erg-seconds” to each interaction in a project and (like we determined in #2) gets amplified in larger teams.
   My Takeaway – This is a very valuable way of thinking about the flow of info on a project. Being a solo consultant, I encounter situations where project delays get compounded by unnecessary “erg-seconds” on interactions not within my development team (of one), but when there are multiple entities involved as clients, and especially when there are tiered teams in each entity. I generally mitigate this problem by occasionally working onsite to reap the benefit of some of the “osmosis” and “convection currents” Cockburn mentions. There is admittedly no substitute for face-to-face interaction- body language, tone of voice, gestures, proximity cues and interjective feedback are lost or hindered when communication occurs via any other method. Neglecting the value of being there to absorb the situation from time to time is a mistake. I have explored the idea of off shoring discrete portions of my projects lately but this concept warns that this geographic dispersion of team members might magnify the “erg-seconds” effect on even simple tasks to the point where it defeats the utility of off shoring I will have to re-analyze the tradeoffs with an eye towards how this effect could be nullified.
4. “Teeing up work” to the bottleneck player
   If we think of software development as an ongoing game of interactive cooperation and communication as Cockburn wishes, then the principle he espouses “#7 Efficiency is expendable in non bottleneck activities” should take care of itself. It’s the concept that if you have say five cooks and one delivery person in your pizza joint, the driver is the bottleneck and your cooks had better “tee up” those pizzas by double-checking that the orders are correct, printing and verifying the directions, perhaps calling when the directions are unclear and making sure the boxes are pristinely stacked in their warming sleeves on the counter for the driver when he/she shows up.
   My Takeaway – Again, I don’t encounter this problem as much since I am solo- every time as the sole developer, I’ll be the bottleneck. The idea that occurs to me though is that I need to utilize the workers in the client’s company better if they can be at my disposal. In even the “best-packaged” project there is probably some added level of prep-work and optimization that can be done to “tee up” the materials for execution. Decentralizing these efforts and pushing as much of this type of activity to the client’s side definitely bides me more time to allocate to activities for which I truly am the bottleneck, plus it has a by-product of #3 and getting some convection currents working to my advantage.
5. The right amount of documentation: light but sufficient
   So there are two goals in this ongoing game of invention and communication- 1) execute well on the current iteration 2) set yourself up for success on the next “game.” If you’re like most developers, documentation is always an afterthought and it’s something you only do when it’s mandated by a manager. Documentation (in the world of Agile) takes on a completely different meaning – there is no strict format for what constitutes appropriate documentation. It’s simply “whatever is necessary to communicate the essence of the theory or project to those that follow.” That may sound too foofy to some but it’s important because (consistent w/ the Manifesto) it’s not constraining implementation to a particular convention or process but rather focusing on the goal of communicating the important aspects. Cockburn talks about methods like “progress radiators” and videotaped whiteboard sessions as simpler ways of preserving and conveying this understanding. The traditional view of documentation is one of a detailed functional spec that precedes the project and captures every requirement and then some system for documenting the code afterwards to again capture every detail of execution so that it can be understood by programmer’s that follow. According to Cockburn’s advice, a picture is a thousand words and a video is worth a kajillion.
   My Takeaway – I found this to be extremely insightful and especially relevant to what I’m proposing to do with Grid7. Too often developers receive a mandate from management to perform documentation in some nit-picky format that distracts them from valuable time they could spend just making things better. A simple picture of a whiteboard to accompany a paragraph of instructions can go miles towards bridging the knowledge gap and that’s exactly what we used to do at a place I worked called Interactive Sites. I’m all about the idea of videotaped whiteboard sessions though and I have already made it a habit of using my iPod along with my Griffin iTalk to audiotape meetings so that I can refer back to them and bring someone new up to speed by giving them an MP3 of a previous conversation rather than regurgitating it. I plan to hold monthly status meetings via Skype on the Grid7 stuff and you bet we will be videotaping them and archiving them for future developers that enter the project.
6. A “Shared Metaphor” as a common anchor point for developers
   The idea is that when working with complex systems and many developers, if you can develop a “shared metaphor” or a more tangible scenario that parallels the problem domain, people will be able to draw upon these parallels to make decisions and therefore remain more cohesive in their approach than if they lacked the shared metaphor. This contribution was from Kent Beck, one of the originators of the Extreme Programming methodology. I cannot comment on the efficacy of this tactic amongst a dev team having never used it myself, but it certainly sounds like it has merit. I think anything you can do to align the vision of team members to some landmark will have value, especially when the project involves highly abstract concepts.
   My Takeaway – “Having everyone on the same page” is important and there is absolutely value to using metaphors to assist in scenarios when the concepts are abstract. In fact, just last week in my meeting with Value Options and Arizona Behavioral Health regarding the construction of the extranet they will use to collaborate on housing homeless folks, I was able to frame the abstract idea of our anonymous wait list in terms of the red ticket dispenser you see at the post office that dispenses numbered tickets anonymously. We were able to collectively work from that image much easier than talking in abstractions and it helped not only in communicating but we were actually able to discover additional ways of simplifying how the wait list works by drawing on parallels of how the ticket dispenser functions.
7. Tradition vs. Transcendence as the foundation of good design
   In other words, “how much do you balance making the interface for your word processing program look like a typewriter in order to make sense for old school folks vs. making it the cleanest-possible, sensible interface for getting the job done at the risk of alienating people who are less savvy?” This is the age-old question of design (and not just in computer apps). There is some optimal balance that exists in a newly-designed apparatus that meshes tradition with transcending innovation to push the state of the art forward and test the boundaries of what’s possible today.
   My Takeaway -With graphical interfaces especially I know big companies like Microsoft spend millions of dollars in research to find that balance. We had Russ Ruggia, a Developer Evangelist for MS come and talk to our CFUG awhile back and he mentioned that MS had spent something like $7bn in psychology research in developing the forthcoming interface for Windows Longhorn/Vista).
8. Criticality and Team size determine methodology weight
   Any project can be evaluated on a 2D graph that shows team size on the X-axis and system criticality on the Y-axis (system criticality refers to the severity of loss in the event of a bug, lowest being an inconvenience and highest being loss of life). Cockburn breaks down the graph into quadrants and explains why projects that stray further from the origin of this graph require heavier methodologies. Even small projects at NASA demand a heavier-weight methodology than the typical web 2.0 app due to criticality. Likewise, even a large non-critical project with many team members like say, the latest Ubuntu Linux distribution, requires a heaver methodology due to the organization required to manage that many team members.
   My Takeaway – Again, the right tools for the right time. It’s not enough to pick one methodology and stick with it in all situations. The skilled developer will recognize that each tool and methodology has validity in different circumstances and will understand the factors at work and when it makes sense to choose one over the other. I’ve never worked on any of the “L stage” critical projects like medical devices where lives are at stake. I have worked on “Discretionary Monies” and “Essential Monies” projects however while running integration with some of the major credit card processors at Apriva. I can certainly testify to the fact that credit card companies have strict methodologies in place for how things happen, to the point where it can become crushingly burdensome on development. Given the position on the criticality axis of Cockburn’s suggested chart, this decision makes sense however.

Well that was a lot of rambling but I hope this condensed version of Agile Software Development is useful. It was a decent read and not as dense as it may sound. I recommend checking out Cockburn’s talk on ITconversations.com at the link above if you have interest in this stuff. It’s definitely given me some ideas for how to approach things in the forthcoming developer co-op.

sean

© 2005 Lights Out Production – All Rights Reserved Worldwide
© 2005 Lights Out Production – All Rights Reserved Worldwide

I recently finished a solid book called “The Innovator’s Solution ” co-authored by Harvard PhD Clayton Christiensen and Michael E. Raynor. It was a sequel to a previous book “The Innovator’s Dilemna” which I never read, but this one was self-contained enough to where I could appreciate its lessons without having read the prequel. Lemme first say that Clayton Christiensen is a _brilliant_ thinker and a persuasive public speaker and that (not having studied for an MBA myself) some of the ideas presented in this book sailed over my head. I like to think that “bidness” can ultimately be understood in terms of common sense once all relevant perspectives are identified though, and this book definitely appeals to that concept. The gist of the book was to examine different businesses across multiple industries and identify recurring “design patterns” or common tendencies with the hopes of offering strategies to up & coming startups for usurping an entrenched, incumbent competitor. Although it was dense reading material, the real-life case studies and examples helped to make the material more digestible and memorable. I’ll try and summarize the salient points that I took from the book and draw parallels to the web development industry:

1. How to pick the right fights – In business (or in any situation for that matter) consider your opponent’s perspective and don’t pick fights where it’s in his/her best interest to stay and go head-to-head with you, but rather pick the fights where it makes more sense for your opponent to flee. The examples referenced in the book showed the tendency of corporations to move “up market” tackling products and services which yield higher returns and leaving the “low hanging fruit” to be commoditized and gobbled up by the newcommers.
   My Takeaway–
   In thinking more about this natural tendency and applying it to the field of software development, I realized he’s right- coding (to some extent) is the “brick-laying” of our industry. While the architecture of an application will always demand skillful consideration and experience, blueprints and prototypes can be created and handed over to a lesser-skilled worker to implement, provided they have suffiicient skills to implement the code according to the plan. To me this was a revolutionary breakthrough because, as an indy developer I had always seen myself as a “one man show” handling the process from initial requirements gathering through the architecture and planning all the way to “hanging the drywall” and code implementation. Borrowing validation from Christiensen’s book, I’ve already looked into farming out the actual coding of applications to a third-party to allow me to focus more on the business development and architecture portions (which I enjoy more anyways).
2. “Hiring” a product to do a job – there was a really cool example early in the book that talked about this recurring tendency of managers in corporations to inappropriately conduct what he termed “attribute-based assesment” when attempting to improve an existing product or service. These assesments often led to perplexingly little or no measured improvement in sales or perceived value by the target market. With all the extensive research, how couldn’t this empirically-determined prescription for success given to us from our customers result in better sales The more effective paradigm for conducting the analysis he proposes is a “situational assesment” in which you think of the product or service as being “hired” by the customer to do a specific “job.” The example he used was the purchase of a milkshake at a fastfood restaurant. The attribute-based studies did things like taste tests to determine the most desirable viscosity of the shake, the optimum sugar content and the most popular flavors. But when all was said and done, the sales of the supposed “perfect milkshake” were not significantly better than the previous one (btw, this type of failure due to the aggregation of massive user feedback reminds me of Kathy Sierra’s “Keep the sharp edges” advice). When they conducted a situational-type study of their consumers to elicit buying trends they learned that there were two major segments that purchased milkshakes, the morning commuters and the evening parents. Both “hired” their milkshakes for completely different reasons though – the morning people wanted a quick breakfast that would give them energy, be relatively easy to consume in the car and give them something to keep them busy on the way to work. The evening parents just wanted a treat to give their kids so they wouldn’t feel guilty about having not bought them the toy they wanted or not letting them play as long as they wanted- the milkshake was a guilt-negating compromise. Attributes of the shakes were adjusted situationally according to the job for which they were being hired to do (ie, fruit chunks were added for the morning commuters to give them some interesting surprises on the way to work, smaller kid-sizes were created for the evening parents) and sales greatly improved.
   My takeaway – how many times have you been tasked with a one-size-fits-all redesign of a web site and are given an”attribute-based” assesment on how it should look without having a full “situational-based” understanding of what the user is truly trying to achieve when they “hire” the site? This shift in thinking to me is HUGE because all the great CSS and Flash and aesthetic enhancements you make are useless if you don’t align everything with the intended “job” the site is being hired to do. We web application developers are in a unique realm in that we can dynamically change our “product” realtime to suit the role for which it is hired by the visitor using personalization techniques as they interact with the site – that’s powerful and often under-utilized. We can essentially serve the milkshake plain vanilla initially and once it’s in hand being consumed, switch to chocolate and add strawberries or even change the shape of the container to better fit the visitor’s intended use. I will be making an attempt to incorporate this type of strategy in my apps from now on.
3. The concept of “The Law of Conservation of Modularity” – Basically, go after under-served markets or un-served markets and compete against non-consumption. All throughout the book he talks about this predictable cycle all businesses experience in which at first, the customers in a market are under-served. One example he used that sticks in mind (perhaps because I had the first one) is the Sony Walkman – when it came out it was a bulky, crappy-sounding portable radio and yet kids had never had the ability to listen to their own music in their parents’ house yet OUT OF EARSHOT of their parents. This feature was mind-blowingly cool because it offered an experience previously out of reach and kids were happy to put up with a crappy product because it was competeing with non-consumption- they’d never had this capability before. I can remember getting my teeth pulled and getting the Walkman as a present and forgetting all about the miserable pain being engrossed in this musical experience with these things called “headphones.” Eventually though, enough competitors will arrive onscene and drive the quality of a product up to the point where the customer base becomes “over-served.” Basically this means that they are being delivered a product with more functionality, more performance or more ***fill-in-the-blank*** than they need. It’s not that the customer doesn’t appreciate the better product, but just that they would be perfectly happy with less. At this point, the law of Conservation of Modularity kicks in and this magical threshold is crossed in which the game flips 180deg: whereas the initial climate of under-served-ness rewarded proprietary architectures and was entirely focused on eeking out maximum performance, the new charge is for open standards, modular architectures, interoperability and convenience, speed of delivery and customizability of the product. The classic example of the shift was in Apple’s early dominance of the PC market with their proprietary systems and then the subsequent coup by makers of the PC’s. The key is when the “low-hanging fruit” gets commoditized, profits do not evaporate into some black hole like people might think (–gasp– outsourcing, sending all our profits overseas?! no.) but rather gets flipped to a different layer. Christiensen repeatedly brings up ex-hockey player Wayne Gretzky and his intuitive ability to skate not to where the puck is but where it will be- businesses must learn to do the same and it takes a confident CEO to assess the current market and go against the grain striking out in a different direction than the pack is headed towards a niche where he/she knows the money will be.
   My takeaway – the parallel here can be easily drawn to any of the big players like Google, Yahoo, Ebay, Amazon- they began with proprietary systems competing against others on the axis of performance but eventually it all shifted and now they are comitted to open API’s and using agreed-upon standards for getting more and more stuff dependent on using their systems and consequently entrenching their position. It’s like a well-digger’s mad dash to first burrow deep vertically to the best water source and then immediately shift the focus to extending his/her pipelines horizontally in every direction to maximize distribution and entrenchment in the marketplace.
4. Business units with disruptive goals must be separated from traditional ones and motivations of your distribution channels must be aligned with your own – this idea made a great deal of sense for me: ask a salesperson to sell a product with 1/3rd the profit potential of a product they currently sell and realistically, they’ll tell you to f*** off. “Market disruption” is the term Christiensen gives to a new product or service that wildly upsets an existing landscape by shattering some kind of preconceived barrier. Think of the angioplasty procedure coming onto the medical scene in an environment where the incumbent technology was expensive bypass procedures performed only by heart surgeons. Originally, the proprietors of the angioplasty procedure tried to sell surgeons on the benefits of performing this procedure over traditional bypass surgery. After all, it was safer, had a quicker recovery time for the patient and was an order of magnitude cheaper- do you think the heart surgeon’s embraced it though? Hellz no- it would have taken the place of their lucrative surgeries they had trained all their lives to perform and shrank a valuable stream of revenue for them. Angioplasty sellers were bummed by the surgeon’s failure to embrace this radically-better technology but after rethinking the situations with respect to the motivations of the intended distributors, they found that they could market their procedure to the smaller clinics that didn’t have high-dollar heart surgeons on staff and therefore had never had the capability to serve this market. By choosing a channel that was filled with motivated distributors, the makers of the angioplasty procedure were able to work with folks whose interests were aligned with their own and secured a foothold in the industry and eventually disrupted the incumbent bypass procedure, but ONLY after they had failed first by trying to recruit the wrong salespeople to replace an existing lucrative service with a less lucrative one.
   My takeaway – This lesson has great relevance to a project I’m currently setting up called Grid7. I’m creating a network of application developers to help execute some business ideas I’ve been tinkering with that now need coding muscle to bring them to fruition. Each project will need to be segmented into a separate business unit because each will have different revenue potential and motivational factors such that if we were to lump them all under one cross-functional umbrella development company, “all the weight would slide to one end of the boat” and development would occur very assymetrically (sales efforts would be even be more skewed). By isolating the business units and structuring the incentives in such a way that motivations are appropriately aligned within each unit, everyone is happy and we can still leverage certain services/assets across all units to best capitalize on what we have.

There are plenty more valuable lessons in this book, but these are the main points that really resonated with me- all highly-relevant to the web development industry and specifically to startup ventures. I’m looking forward to establishing the Grid7 incubator in the coming months- if you are an application developer and have interest in being involved, be sure to get in touch with me so I know who you are and where your skills/interests lie.

-sean

© 2005 Lights Out Production – All Rights Reserved Worldwide