How to build a distributed application: Why Theory Matters

After looking at the comments on my last post, I have decided to write a few words about why I think it's important to have a theoretical framework for how to build distributed applications. Our industry is beset with faddism, magical thinking, and tribalism. We spend an enormous amount of time debating best practices, patterns, and "the right tool for the job". Unfortunately, we usually end up with nothing more than slogans, silver bullets and shibboleths. Without a coherent mental model for what makes things work, we can easily get lost in a jungle of received wisdom, industry trends, and vendor claims. I like to think of the OODA loop as my "machete of truth". It's no guarantee that I'll find the right path, but at least I can clear out the most obnoxious undergrowth.

 

A few specific examples will probably help explain this more clearly. Suppose you hear about a new tool or technique that is supposed to help improve your application development process or application architecture. How do you decide if its potential value outweighs the costs of changing the way you do things? Here's one I've started looking at: FIT. I take a quick look around the web and I like what I see. I have a lot of respect for the work of Ward Cunningham and James Shore. On the other hand, I also know that the real superstars of development are much more productive than I and my team members are. Superstars often make the mistake of assuming that it's their tools or techniques that make them productive, but that is usually not the case. What works for Ward and James may not work for me. If the OODA Loop has predictive power and can tell me why some tools and techniques work better than others, I am in a much better position to decide whether or not to push for incorporating FIT into my preferred process. In this case, FIT fits the OODA loop model very well. Now, I'm faced with the hard work of really learning about it, figuring out how hard to push my organization to accept it, and when to introduce it into the process. Has anybody done any work to hook FIT into Team System? Sam? Anyone?

 

Using the OODA Loop as a mental model can also help you understand how to use a potentially dangerous tool (and all powerful tools are potentially dangerous). As you might expect from a VB-loving Mort, I absolutely adore "Edit and Continue". I'm certainly aware of the dangers of that feature, but I've also used its power in extremely productive ways. Writing code in a late-bound, loosely typed, verbose language with an IDE that supports auto completion, edit and continue, and excellent framework support allows me to engage in exploratory coding that approaches "coding as thinking". This technique allows me to iterate through a variety of potential solutions to a particular problem without the friction inherent in the more traditional method I use in developing production quality code.

 

I would like to posit the "spinning balls" from MTS/COM+ as an example from distributed application architecture. Back in the day, I spent many hours becoming intimately acquainted with the ins and outs of MTS. At the time, I didn't understand why IT managers and business users were so fixated on those darn spinning balls. Now I understand that, for most of them, this was their first exposure to distributed applications that provided them with insight into what was happening as the application ran. Understanding the OODA loop allowed me to appreciate the importance of this feature. The vast majority of distributed applications are completely opaque to the people who pay for them and the people who support them day in and day out. As application architects considering code hosting environments (e.g. ASP.Net, BizTalk, Windows Activation Services), the first question we should be asking is "Where are the spinning balls?" The ability to see what's happening as your systems run is so vital that no modern distributed application can afford to be without it. Building that capability on your own can be devilishly difficult.

 

These examples are really just about how I use the OODA loop in the small. My main goal is to develop and propound a theory that unifies the process and the architecture of distributed applications. I don't think that's really been attempted before (this statement is my attempt to elicit evidence to the contrary). Ali Pasha asked what other frameworks had I considered. The closest thing to what I'm trying to do that I'm aware of is feature driven development. Unfortunately, FDD is very closely tied to OO in ways that I think are inappropriate for distributed applications. Of course, that doesn't stop me from appropriating all the things about FDD that I like. On the process side of the fence, I think Scrum has the best theoretical underpinnings. I will borrow heavily from it and the ideas of David Anderson. I would be remiss if I didn't point out that one of my favorite bloggers has considered the idea of connecting agile development with SOA and pretty much dismissed it out of hand.