The XML in your objects is closer than it appears

DaveO and MarkNot replied to last night's post. Both raised interesting points, to which I'm compelled to respond.

To start with, though, let me be clear: I want interoperable web services just as much as anyone else. If web services don't interoperate, then we spent the last 5 years working on them for nothing. I just disagree with the path being proposed.

Thinking about posts from the last couple of weeks, I couldn't help noticing the odd path they've taken... I said I want RelaxNG. Some agreed with me. Aaron (and others) said, too bad, there's industry concensus on XSD and that's what we're using. Dare said that XSD was easier for object mapping than RNG. Then Dave responded to my original post about not profiling indicating that in fact people can't get XSD-based object mappers to work right and WS-I should act. So what exactly is the concensus on XSD; that we have to use it, but it doesn't work so we shouldn't use it really, we should derive a new language from it instead? (And, as Mark asks, whose concensus is it anyway?) If, just as I was starting to accept using XSD again (as I periodically do), people say “well, actually not XSD but something else derived from it“, I have to wonder why, again, we can't move to RNG.

Now, to comments (my favorite part of the morning :-)...

Mark rejects my concern about subsetting XSD pointing us toward RPC:

That doesn't follow. Just because we're limited to a subset of Schema doesn't mean that it magically becomes RPC; it's possible to have document-oriented messages without using the Infoset and Schema; that was precisely the point of my original entry.

When we worked on the WS-I Basic Profile, a lot of people focused on mapping operations to methods and messages to objects. Some of us viewed that as an implementation detail and while we wanted to support it, we didn't want to restrict people to that. We talked at the time about subsetting XSD to remove, for instance, redefine and element substitution groups. These were specific concerns to people working on object serialization plumbing. I argued at the time that you could always simply surface XML whenever you couldn't do a reasonable object mapping, but most people rejected that because they don't want developers to have to see any angle brackets.

So, frankly, the heart of my concern about WS-I subsetting XSD is that the subset will be restricted to the things that map obviously to modern OO languages. For someone like me, who is willing to (and even likes to) deal with XML and happens to work in a document-centric problem space, that's troubling. I don't want to lose options for describing certain XML structures just because someone else wants to pretend that Web services are about methods and objects. I also don't want to be forced to have a very loose schema definition and then some additional document that says what I really mean (don't think I haven't thought about making all my schemas wildcards with an RNG definition in appInfo ;-), as Mark seems to suggest. Why should I have to do that, just because someone else has a tool that isn't working for them? If users care, as DaveO suggests, do they care about people like me?

The case in point is my argument about using XHTML. Mark thinks I'm not gaining much because there aren't any semantics captured there anyway. There are several counter arguments. First, as a system that publishes documents for presentation, that is the semantic of my system. Next, even if there is no semantic information, the syntax validation is helpful (especially since we use an XHTML 1.1 modular doctype that does not allow certain elements, like script). Finally, does XSD ever capture semantics, or is it just a complicated way to define XML syntax? Anyway, if my system consumes and produces XHTML, why shouldn't I be able to use the W3C XSD definition as part of my Web service's contract? Why should I have to do something looser and non-standard and non-interoperable because of how someone's object mapper works? The pushback is only coming from people who want to program against my service with a certain tool and find that it doesn't work well. If using XHTML as an example confuses the issue, consider the other vertical languages I mentioned in my last post: XBRL and HL7. They convey data, not presentation. Are they to be restricted away or redefined if they don't fit the profile? Everyone agreed (so they tell me) to use XSD, and they did. We can't rescind that now.

In short, I connect subsetting XSD to RPC because I believe that's the force that's really driving it. It's because developers writing classes and exposing them as Web services are having trouble with their tools. If people were willing to move to a simpler language like RNG, instead of trying to derive such from XSD, I could get on board. But RNG doesn't necessarily map all that well to objects either. The real problem is that XML doesn't map that well to objects, because the content model of an XML element is much more flexible than the content model of a class with fields and properties.

That brings me around to Dave's latest post on this thread. He feels customer pain and is frustrated that all the vendors can say is “wait until the tools get better“. I agree. Since that position is unacceptable for developers, profiling XSD seems like the best bet, unless someone has a concrete counter-proposal. I disagree. So, here's my counter proposal: don't build better tools, build different tools.

Steve chimed in last night and said something I've felt and said myself for years: the object/XML mapping problem is akin to the object/relational mapping problem, and just as hard. We've spent more than a decade trying to get O/R mappings to work correctly with very little success. We're 5 years into O/X mappings and we're seeing the same problems. In fact, XML and SQL are more alike in their ability to construct flexible data “shapes“ than objects are. It kills me that people want to build systems that store data on the back-end in a relational database and sends it out on the wire as XML, but in between it has to all be objects. There are two reasons for this that I see. One is that “developers shouldn't have to know“, which is a fantasy, at least at this point. You only get to not know when the tooling actually does an effective job for you (for instance, I'm so confident a good optimizing compiler can generate more efficient assembly than I can that I don't even think about it anymore). We've never gotten anywhere near that with these data mappings. The other is that they need a good way to implement their systems logic and it's hard to do on top of raw SQL data and raw XML data. Steve's suggestion is that we have to stop trying to hide the XML inside compile-time-generated classes and I totally agree. That doesn't mean (to me at least) that you should never map XML to objects if that's what you want. But it should be a deliberate act when desired, not the default behavior of the tools you use. You should treat it as one of many options you have for implementing your system, and one with significant limitations at that.

I often hear people say that if you don't map Web services to something familiar, i.e., objects and methods, developers won't adopt them. I don't buy it. The success of ASP and it's descendants on all platforms say that's not true. ASP used objects and script to provide a model for constructing pages that was different from what had gone before. It was easy and effective and developers loved it. So it's not familiarity but simplicity that matters. I also often hear people say that XML is too hard and that objects make it easy for people. Well, based on this thread, clearly the second part is false. It isn't XSD that's at fault though, it's the two layers of OO serialization plumbing between the author of a service and the author of its client.

The solution, I believe, is to make tools and languages that surface XML and make it easier to deal with directly. XPath, XSLT, XSD are languages that allow you to act on XML in particular ways and you can use them to implement portions of your system. What's missing is good, simple ways to manage and deploy them, invoke them from code, etc. Yes, this implies that people have to learn something about these languages, the same way they had to learn about HTML and JScript and SQL, etc. (this is one of the reasons I think RNG is the place to go). But this shouldn't surprise us. We can't build an entire distributed system infrastructure on XML while at the same time saying that no one should have to know anything about it. It just won't work.

I still meet people who've designed a Web service that passes one string in and one string out. The strings contain XML. They do this to bypass their marshalers, and it works. Yes, they have to read and write XML, but so what? It's effective and interoperable; and because they are deliberate about the data they consume and produce, it opens the door to loose-coupling in ways that static object mappers never will.

Finally, Dave asks what it means for WS-I if it can't solve this problem. I worry about what it means if it does. If WS-I restricts XSD to a subset that makes OO programmers happy, but XML programmers sad, we run the risk of derailing the whole train. Companies adopt standards as a matter of policy. Where I work, people have asserted that all services should be WS-I BP compliant. If, someday, they also asserted that all services be WS-I XSD subset compliant, we'd might well be forced to either abandon plans for some services or abandon the WS stack and go back to raw XML over HTTP. Frankly, for the work I do, I'd rather that WS-I was irrelevant than WS-* was irrelevant.