Serendipitous, Indeed

Wherein our narrator makes a fortuitous discovery in a most unexpected place.
 
One of the struggles I face in trying to explain my objections to the Indigo API is creating concrete examples of the abstract, and sometimes seemingly abstruse, notion I have that the Indigo API itself leads developers down the wrong path in building distributed applications. It's easy to come up with scenarios that demonstrate the ideas, but these scenarios often seem forced and are always open to the criticism that they're contrived specifically to prove my point. Imagine my delight and surprise when a post showed up in my aggregator that perfectly illustrates my argument and that post is written by a Microsoft expert in API usability. Please take a few moments to read and ponder the significance of this post by Steven Clarke. Now, I think Steven drew the wrong lesson from his experience, but before we get to that, let's take a look at the situation he set up. He has some UI programmers using data binding to link the UI to a backend Indigo service. This is a key scenario for SOA. In this case the problem domain is an album list. I'm going to assume (based on the names of the methods in his interface) that this is a personal list of music albums. Steven's understanding of his problem is that he made errors in the way he maintained state within his service and the assumptions that the UI programmers made. To understand how fundamentally wrong this is, we need to think a bit about the problem domain, the interface he exposed, and how the Indigo API enabled him to create the situation.
 
An album list is rather obviously a document. Except that to many software architects and developers, that's not obvious at all. The database folks will immediately think of a relational model and the OO crowd will think in terms of objects and their methods (state + behavior), but normal people will naturally have a conceptual model of a document. It is absolutely critical to maintain the document oriented view if you want to be build excellent distributed applications. The main reason that service orientation is the best way to build distributed applications is that it provides an effective way to map document-oriented business processes onto the physical constructs of a computer network. The basic principles of service orientation have been around as long as the pyramids of Giza. Indeed, they enabled the ancient Egyptians to build the pyramids. The system they used was based on the same asynchronous messaging architecture that I use to build systems today. Granted, their messages were either verbal or written on papyrus in hieroglyphics, their network used people to physically move those messages, and the services were all performed manually, but from an abstract systems view,  it is the same.
 
Unfortunately, the current Indigo API doesn't encourage a document oriented approach to service design. Take a look at Steven's service contract:
 
[ServiceContract(Namespace = "http://Usability.Task")]
interface IAlbumService
{
 [OperationContract]
 Album[] GetAlbumList();
 [OperationContract]
 void AddAlbum(string title);
 [OperationContract]
 int GetNumberOfAlbums();
 [OperationContract]
 void SellAlbum(string title);
}
 
Now, if you're a veteran of the global struggle against rampant unscalability, you'll immediately see issues with this design. The AddAlbum method is the very definition of a chatty interface. And if you don't think scalability matters for an album list, I'd just point you this guy. More fundamentally, this interface confused the UI developers it was supposed to support. I don't think you place the blame for these problems with Steven Clarke. He's clearly a very intelligent guy. I've learned a lot from his writings on API usability. Nor can we write this off to inexperience. He's been working with Indigo as long as anybody. I think the fault here clearly lies with the API itself. What I find most interesting about Steven's post is Steve Swartz's advice. Unlike most Indigo developers, Steven has direct access to the Indigo architects. Steve's advice is very standard RPC thinking. He presents two choices. The first is add Open and Close methods to the API to represent the state management. The second is to add a parameter to each function to represent a session identifier. Both of these solutions pollute the service interface with details that are irrelevant to the problem domain and force the service to relinquish some its autonomy to the service consumer.
 
There is a better way. Microsoft's own Patterns and Practices group calls it the Document Processor pattern. If you apply that pattern, you get an interface that would look more like this:
 
[ServiceContract(Namespace = "http://Usability.Task")]
interface IAlbumService
{
 [OperationContract]
 AlbumList GetAlbumList(string Owner);
 [OperationContract(IsOneWay = true)]
 void StoreAlbumList(AlbumList Albums);
 }
 
This is a huge improvement over the original API. No need for Open and Close methods, no extraneous session id parameters, and no chatty calls. But there is still a problem. I wouldn't want to tie up my pretty Avalon UI while waiting on Brad's 1000+ album list to come back from the service. Other implementers of the service caller might choose to block on that call, especially if the service caller isn't a UI. Unfortunately, the async design of Indigo is totally messed up. It's based on the async pattern of the .Net framework which is a pattern for calling synchronous local methods asynchronously. There is a much simpler approach that's possible when you realize that all network traffic is fundamentally asynchronous and that only the caller is able to make the choice whether or not its code should block waiting for the response. I'll outline this approach in my next post.
 

Posted Sep 04 2005, 01:18 PM by john-cavnar-johnson

Comments

Tom Fuller wrote re: Serendipitous, Indeed
on 09-04-2005 12:27 PM
Mr. Cavnar Johnson,

As you are aware I have not always fully understood your objections to the Indigo programming model. However after a few more weeks of research I'm really starting to follow your concerns completely. Specifically I have read two outstanding pieces of work in the past two weeks. The first from John Evdemon who is right out of the Microsoft Architcture Strategy group. The article is titled "Principles of Service Design: Service Patterns and Anti-Patterns". In this piece two things stand out that reinforce your points above. First of all the first anti pattern titled "CRUDy Interface" warns against chatty interfaces. This completely makes sense in my mind, how can we avoid the pitfalls of tight coupling if we don't get away from fine grained interfacing. Second Mr. Evdemon talks about never trusting the client to do the right thing. I would think you support this notion by not agreeing to Mr. Swartz's idea of implementing an Open() and Close() to manage state through an Indigo service. I couldn't agree with you more on this point.

The other item I recently read was from November of 1994. That's right 1994, It is from Sun Microsystems and is something I'm sure you distributed guys are familiar with. But for a guy who's first true introduction to distributed development was .Net remoting and web services I found the parallels to our recent conversations jaw dropping. I'm referring to "A Note on Distributed Computing" by Jim Waldo, Geoff Wyant, Ann Wollrath, and Sam Kendall. Shockingly enough they write about the cyclical nature of our attempts to unify programming models for distributed application development. They talk about client trust issues and partial failure recovery. They talk about "Quality of Service" and the importance of not treating remote objects as though they are local objects.

These things sound awfully familiar to the grumblings I hear from you and your other contract-first colleagues. I guess what I'm still not convinced on is whether or not we can do this with Indigo. I can't help but picture services defined that are explicit in their [DataContract] and in their [MessageContract] definitions along with a [ServiceContract] that more carefully presents the exposed behavior in an explicit way (like the one you have defined above). Many of the key issues they talk about in "A Note on Distributed Computing" are being handled in the Indigo programming model. They talk about the differences in memory access and Indigo introduces "out" parameters. I guess there in lies part of the problem in your opinon .... right? You basically are capable of doing the things that you can do with local objects without taking specific steps to control it when you go across process boundaries.

When I was researching all of the capabilities of the Indigo service stack I couldn't help but think that the idea of "out" parameters was very anti Message Oriented. Would you agree? Also I have to say that the message out of Microsoft does appear to be somewhat fragmented. In one camp they are attempting to showcase all of the cool new features for building RPC style services (like private sessions for example) and in another camp they are showing some of the best practices for doing SO (John Evdemon's article for example). I suppose the responsiblity falls back on enterprise architects to understand this and even more so on trainers like PluralSight to continue to reinforce the differences. What else do you think Microsoft needs to do? Aren't they just giving us an aresenal of features that we can use or not use. If we as architects and designers build an enterprise SOA using RPC style objects then we should expect to pay a penalty (probably a big one). I suppose I still struggle with why would I ever build an RPC style distributed object if not for Fat client business logic centralization (of course I come from a web application background so that might contribute to my short sided opinions). Can you think of some other scenarios? I've heard of security being used as a reason for doing this too.

Great thought provoking post for a Sunday afternoon. Also very timely for me since I just read those two articles. I look forward to your response.
Rich Turner wrote re: Serendipitous, Indeed
on 09-06-2005 1:10 PM
John - all you've done here in your "solution" is abstract the process a little more be sending across a list of albums rather than just one. Big deal - this is a design issue, not a technology/platform issue.

Further - your "huge improvement" is only a little more scalable than Steve's original design - the calling thread still blocks while your album list downloads. If this truly was a concern, then you would, of course, implement the beginxxx-endxxx async method pattern that you so readily discounted.

Also, you have not changed the programming model - you're still doing what you call "RPC" style programming. I'm shocked. Why not illustrate the code that would form up the necessary message, squirt it across the wire, and implement a response handler that de-serializes the response ... as you seem so fond of promoting?

Finally, what Steven was illustrating was how one COULD build sessionful communications using Indigo. He was not proposing that this be the design for a fully-fledged Google-scale service - this would, of course, require an entirely different approach.

John, I think you're letting yourself get a little carried away with your anti-WCF jihad and the gaps in your argument are starting to show.

I would encourage you to, instead of simply thinking up new ways to bash a given technology, to instead come up with very clear, consistent and actionable improvements that could actually benefit the design of WCF and the customers who use it.
John Cavnar-Johnson wrote re: Serendipitous, Indeed
on 09-06-2005 2:07 PM
Rich,

Get a grip and actually read my post. You guys always want to equate messaging with bit-level handling of the communications. That's not what I'm talking about at all. I think it's your job to do "the code that would form up the necessary message, squirt it across the wire, and implement a response handler that de-serializes the response". I just want you to expose a simple messaging model that encourages developers to do the right thing.

Also, I specifically pointed out the problem with the GetAlbumList method and mentioned that I'd be addressing it in my next post.

In case I wasn't clear, the huge improvement in the API had nothing to do with scalability and everything to do with matching the API to the problem domain. The biggest problem with Steven's API was that it didn't match the expectation of his users.

Tom Fuller wrote re: Serendipitous, Indeed
on 09-06-2005 3:19 PM
Wow, a true blog battle royale. I'll gracefully step aside with these final comments. I still don't understand what it is that would make WCF more message-oriented. I think there are features that reinforce that approach (message contracts for example) and things that basically contradict message-orientation (out parameters for example).

In my mind the responsiblity falls back on the architect/designer to implement a services that will prove to be flexible. The technology set appears on the surface to give me what I need to respond to the changing needs of our business community. If it is RPC stylings that creates all of this horrible tight coupling then simply avoid it by using typed message services. Am I over-simplifying this?

Either way, I enjoy your postings John, they make me think through my own opinions on these manners and they've pointed me to some great literature.
Erik Johnson wrote re: Serendipitous, Indeed
on 09-07-2005 6:25 AM
It looks like I am moderating a BOF session at PDC about "Designing Apps for SOA". It's a brainstorming session to look into how we think applications designs are changing to be more amenable to service-orientation.

I drew the newbie-moderator timeslot: Tuesday, 13 September at 10:15PM Pacific. I can post the notes when it's done. But now that the slot is confirmed, I thought I would drum up some interest.
Ice wrote re: Serendipitous, Indeed
on 09-07-2005 11:15 AM
I've really enjoyed reading a lot of the posts here. I do believe that I may be missing some important points however.

First of all, at what point do you make the decision to use "services" or a "service-oriented" approach.

What do you consider "RPC"-style of programming? Is then when the client has "knowledge" of the server in the form of a proxy? If you remove a "proxy" what agnostic solution do you provide for the client to connect to service?

Tom Guinther wrote re: Serendipitous, Indeed
on 09-10-2005 4:15 AM
You guys are having a lot of wonderful arguments and everyone is making good points (altough Rich seems to be loosing patience and simply bashing away.) but I think the key point is that the original album list interface is extremely poor. It is so poor that you can't even have an intelligent argument about it, and actually using it as part of an API Usability study is astounding (and somewhat revealing...) to me.

No one really mentioned that a contract (service) or otherwise has important components that might not be refelected in the interface itself. For example, if Steve had provided some actual documentation on the purpose of the service, how the service worked (authentication, statefulness, synchronicity, etc...) at least good developers would be able to work around a poor interface. And although they shouldn't have to, in the real world that is the 90/10 and we can't blame Indigo for that (yet!)
Steve Swartz wrote re: Serendipitous, Indeed
on 09-20-2005 12:10 PM
John, I stopped reading your blog once I got to tenth thing you hate about Indigo. I found it both mean-spirited and ill-informed.

A friend noted that you had misquoted me in this post; I wanted to note that here.

Steven's usability study was around the interaction between WCF and the "databinding" part of WPF. The problem people were having was that they assumed that WCF worked like a database -- they figured they'd get the same data if they opened many proxies. I was suggesting an alternative pattern for that particular service that would make the service's operation clear to his experimental users (who do not get a lot of documentation). For better or worse, the databinding APIs are all request-response -- it was out of scope for the particular exercise to redesign WPF.

For some reason, you felt inclined to infer my general architectural inclination from that single example. You were wrong to do so, and you got me wrong to boot.

Add a Comment

(required)  
(optional)
(required)  
Remember Me?