REST vs XML-RPC vs SOAP

REST vs XML-RPC vs SOAP
[
home
] [
zone
] [
weblog
]
This is an excerpt from a mail conversation with someone who
was a bit surprised to learn that I really don't like XML-based
RPC protocols.  Slightly edited for readability, and to protect
the innocent.

I was just surprised after all of
your work with xmlrpclib and soaplib. I mean you did do that work long
before I came to the conclusion that RPC isn't scalable so I would
hardly blame you for changing your mind.

Well, I spent about 30 minutes on the first
xmlrpclib
version,
mostly because Dave Winer flamed me over some irrelevant
topic, and told me I couldn't program my way out of a brown
paper bag (or something similar)...
The first version was client only, just enough to talk to a
Frontier echo server.  It took a few more hours to prepare
the second release, and I've probably spent 2-3 days since
then on bug fixes, documentation, and support.  Plus a couple
of days on _xmlrpclib, our commercial XML-RPC accelerator.
About a week in total.  Mostly by accident.
:::
In contrast, SOAP was development hell, with moving specs, no
working server to test against (we spent over a week trying to
get IBM's first Java implementation to run on a server at the labs,
only to find that they had implemented an earlier version of the
specification).  The first version was based on the xmlrpclib design,
but that architecture broke down, and I did at least two complete
overhauls on the way to 0.9 (which we still haven't released;
nobody can motivate themselves to fix the last few bugs...).
:::
The first piece of the REST-puzzle came from the second
overhaul -- to hold the SOAP body during decoding, we designed
a light-weight DOM-like data structure.  Only took a day before
I realized that it was an excellent way to work with almost any
kind of XML data:
http://effbot.org/zone/element.htm
The second piece came from an XML-RPC based project where
we needed to work with large chunks of binary data.  Base64
just won't cut it when you have to transport 1-100 megabyte
entities mixed with metadata records (and other structured
data).  We solved this by letting an XML-RPC service return
dynamically generated URLs, which the client used to fetch
the resulting data via HTTP:
    rpc = ServerProxy(...)
    ticket = rpc.login(...)
    result = rpc.search(ticket, "meteosat")
    for scene in result:
        url = rpc.geturl(ticket, scene)
        async_http(url, file_consumer(scene + ".im"))
Moving to pure HTTP allowed us to throw away the code for
the XML-RPC service.  (I love throwing away old code ;-)
The third piece was a deeper understanding of XML, with
emphasis on how to think in
Infoset
terms.
:::
So when we started working on the design for a large image
distribution and processing system, we already had a simple
and scalable design, and the tools to support it.  Just send
XML documents representing objects back and forth over
HTTP, and use the lightweight DOM structure to hold parsed
versions of them inside the application.  Add some glue code
to let application code access the DOM structures as ordinary
Python objects, and you have a complete and scalable system.
The result was a much nicer specification (very few buzzwords)
that anyone can understand, far less code, and most importantly,
a much more robust design.
And we got the contract.
(now that I'm writing this, I realize that the result is not
that different from an HTTP-based image archive I built back when
Mosaic was state of the art in browing.  IIRC, we shipped INI-
files back and forth... oh well, one never learns ;-)

Plus, I still think [RPC] is incredibly easy for small apps...

Oh, sure -- XML-RPC gives you a lot of power, and anyone can
understand how it works, and understand what the limitations
are (Dave W. might not know the limitations, but that's another
story ;-)
SOAP is something completely different; lots of additional
complexity
,
but very few additional benefits.  Some people love complexity
(especially if they see a chance to make a living out of it,
like Don Box ;-).  But I don't.  Wouldn't use Python if I did.
Cheers /F