//Request
GET /shop HTTP/1.1
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Date: Mon, 26 Jul 2010 10:00:00 GMT
Cache-Control: public, max-age=86400
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<shop xmlns="http://schemas.restbucks.com/shop">
  <link rel="http://relations.restbucks.com/rfq"
        href="http://restbucks.com/request-for-quote"
        type="application/vnd.restbucks+xml"/>
</shop>

The link here is typed with the link relation value http://relations.restbucks.com/rfq, which indicates that the link points to a resource where a request for a quote can be submitted. Following the link, the client retrieves a form:

//Request
GET /request-for-quote HTTP/1.1
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Date: Mon, 26 Jul 2010 10:00:05 GMT
Cache-Control: public, max-age=86400
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<model xmlns="http://www.w3.org/2002/xforms"
  schema="http://schemas.restbucks.com/rfq.xsd">
  <submission
    resource="http://restbucks.com/quotes" 
    method="post" 
    mediatype="application/vnd.restbucks+xml"/>
</model>

The response entity body comprises an XForms form model. (Our custom media type definition for application/vnd.restbucks+xml says that one of the things a client can expect to receive in a response is an XForms form model.) The form’s <submission> element includes several pieces of control data: it indicates which verb to use when submitting the form (POST), where to submit the form (http://restbucks.com/quotes), and which content or media type to use when submitting the form (application/vnd.restbucks+xml). Because the definition of our custom media type includes more than one XML schema (much as the Atom specification defines two schemas, one for feeds and one for entries), the form’s control data needs to further clarify what /quotes expects to receive in a POST request. To clarify which schema the POST request body should adhere to, the <model> element’s schema attribute references the rfq.xsd schema.

Here, then, we have all the information a client needs to construct and submit a valid request to the /quotes resource. And all without any fields for the client to fill out.

Heavily typed links

A heavily typed link is one where the link relation describes not only the relationship to the linked resource, but also the HTTP idioms – the control data – necessary to manipulate that resource.

Using a heavily typed link our shop could link directly to the /quotes resource, instead of to a form:

<shop xmlns="http://schemas.restbucks.com/shop">
  <link rel="http://relations.restbucks.com/quotes"
        href="http://restbucks.com/quotes"
        type="application/vnd.restbucks+xml"/>
</shop>

Here, the definition of the link relation http://relations.restbucks.com/quotes might be something like: “Indicates a collection of quotes. To request a quote, POST a <request-for-quote> with a Content-Type header of application/vnd.restbucks+xml to the linked resource.”

That’s a perfectly respectable way of using links and link relations; in fact, it’s the strategy we employ in REST in Practice. But it can have its downsides. Most importantly, it can increase the coupling between the client and any resources that employ link relations.

Make no mistake: link relation semantics comprise out-of-band knowledge. There’s no magic here: link relations introduce a degree of coupling between a client and any server-governed resources that adopt them. The trick is to keep this coupling as low as possible. By putting control data in the link relation definition, we perhaps introduce more coupling than is strictly necessary.

Out-of-band data is less visible, and more difficult and more costly to change, than data that is inlined in the message. Whilst the control data may not change all that often, changes can and do sometimes happen; inlining the data allows these changes to be propagated to clients sooner rather than later. Control data produced at the time the response is generated is generally more recent than control data defined through some out-of-band mechanism.

Using lightly typed links helps separate semantics from control data. A “light” link relation indicates what the linked resource means in the context of the current representation – that’s all. This helps mitigate against a second, somewhat more subtle, downside of adding control data to link relations: the tendency to introduce action semantics. It’s no great step to shorten the link relation value above to http://relations.restbucks.com/quote, and to rewrite its semantic to read “Indicates an opportunity to request a quote by POSTing a <request-for-quote> with a Content-Type header of application/vnd.restbucks+xml to the linked resource.” At this point, our link has effectively become an operation. The typed-link-to-form strategy helps us concentrate on describing what a linked resource is, rather than what a link does. Link relations do not necessarily imply action semantics, but they can very easily be made to do so.

(Note: when adding links to representations, I still prefer to use a <link rel="..." href="..."> construct, or something similar, rather than <order-form href="...">. The reason for this is that elements such as <link> separate link syntax from semantic context, as explained here; and this is a good thing, because what a link ought look like – its syntax – changes far less than what a link might mean in a particular context. By separating these concerns, we make it easier to evolve a distributed application.)

Interestingly, on the wire, the result of submitting an XForm form looks exactly the same as if we’d simply POSTed a <request-for-quote> directly to /quotes:

//Request
POST /quotes HTTP/1.1
Host: restbucks.com
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<request-for-quote xmlns="http://schemas.restbucks.com/rfq">
  <items>
    <item>
      <description>Costa Rica Tarrazu</description>
      <amount>250g</amount>
    </item>
    <item>
      <description>Guatemala Elephant Beans</description>
      <amount>250g</amount>
    </item>
  </items>
</request-for-quote>

Knowing this, we could always add both a lightly typed link to a form and a heavily typed link to our shop representation:

<shop xmlns="http://schemas.restbucks.com/shop">
  <link rel="http://relations.restbucks.com/rfq"
        href="http://restbucks.com/request-for-quote"
        type="application/vnd.restbucks+xml"/>
  <link rel="http://relations.restbucks.com/quotes"
        href="http://restbucks.com/quotes"
        type="application/vnd.restbucks+xml"/>
</shop>

Two paths to the same result. I don’t recommend doing this: I include simply to highlight how a linked form achieves the exact same result, but with the added beenfit of having inlined control data.

Pre-filled forms/self-describing requests

Here’s another example of using a typed link to a form:

//Request
GET /quotes/1234
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Cache-Control: public
Date: Mon, 26 Jul 2010 10:01:00 GMT
Expires: Mon, 02 Aug 2010 10:01:00 GMT
Content-Type: application/vnd.restbucks+xml
Content-Length: ...

<quote xmlns="http://schemas.restbucks.com/quote">
  <items>
    <item>
      <description>Costa Rica Tarrazu</description>
      <amount>250g</amount>
      <price currency="GBP">4.40</price>
    </item>
    <item>
      <description>Guatemala Elephant Beans</description>
      <amount>250g</amount>
      <price currency="GBP">5.30</price>
    </item>
  </items>
  <link rel="http://relations.restbucks.com/order-form"
        href="http://restbucks.com/order-forms/1234"
        type="application/vnd.restbucks+xml"/>
</quote>

The link relation http://relations.restbucks.com/order-form indicates that the linked resource is something that allows an order to be submitted. Following this link, the client retrieves an order form:

//Request
GET /order-forms/1234
Host: restbucks.com
Accept: application/vnd.restbucks+xml

//Response
HTTP/1.1 200 OK
Cache-Control: public
Date: Mon, 26 Jul 2010 10:01:05 GMT
Expires: Mon, 02 Aug 2010 10:01:00 GMT
Content-Type: application/vnd.restbucks+xml
Content-Length: ...
Content-Location: http://restbucks.com/quotes/1234

<model xmlns="http://www.w3.org/2002/xforms">
  <instance>
    <quote xmlns="http://schemas.restbucks.com/quote">
      <items>
        <item>
          <description>Costa Rica Tarrazu</description>
          <amount>250g</amount>
          <price currency="GBP">4.40</price>
        </item>
        <item>
          <description>Guatemala Elephant Beans</description>
          <amount>250g</amount>
          <price currency="GBP">5.30</price>
        </item>
      </items>
      <link rel="self"
            href="http://restbucks.com/quotes/1234"
            type="application/vnd.restbucks+xml"/>
    </quote>
  </instance>
  <submission
    resource="http://restbucks.com/orders" 
    method="post" 
    mediatype="application/vnd.restbucks+xml"/>
</model>

Once again, I’ve used an XForms form model, but this time I’ve pre-filled it with an <instance> element. Even so, there are no form fields to fill in; all the client needs to do is operate the form according to the inlined control data.

The interesting thing here is that /order-forms/1234 simply returns a different representation of the quote resource identified by /quotes/1234 (the response indicates as much in its Content-Location header). By supplying a forms-based representation of the quote, we inline all the information necessary to submit an order to an order processing engine. The client doesn’t need to compose an entity body; it simply needs to operate the form according to its control data. This results in a self-describing message being sent to /orders.

(In a real-world application I’d likely include a signature in the form body. This signature would guarantee that the client hasn’t tampered with the form contents prior to submitting the form to the order processing engine. The effectiveness of the signature depends on an out-of-band trust relationship having been established between the quoting engine and the order processing engine.)

The result of first following the link to the form, and then submitting the form, is to transition the overall state of the distributed application from Quote Requested to Order Placed, as illustrated in the following diagram:

Every request-response pair transforms application state. Retrieving the form enriches the client’s understanding of the current application state; that is, it opens up new opportunities for interacting with other resources. POSTing the form causes the application state to transition from Quote Requested to Order Placed.

The fact that the overall state of the application has changed is of no consequence to the server resources involved; the application state model is nowhere baked into the server resources. As far as the quote resource is concerned, it has simply been asked to surface a forms-based representation of itself. As far as the orders resource is concerned, it has simply created a new, subordinate order resource. This change in application state is, however, important to the client.

Summary

Understand the tradeoffs between inlining and putting control data in an out-of-band mechanism. Use the right controls for the job; understand the many different hypermedia capabilities at your disposal. The best resource for this is Mike Amundsen’s in-depth study of the hypermedia capabilities of many different kinds of hypermedia control. Recently, Andrew Wahbe started examining the need for machine-to-machine hypermedia, and the differences between controls for machines and controls for humans. Watch his blog for further discussion.

My understanding of hypermedia controls has been heavily influenced by my experience of the human web, where links and forms predominate. But when we talk about forms in a machine-to-machine context, it’s not the form field elements that are of interest, it’s the control data elements. These control data elements help program the client on the fly. The term “form” as it applies in a machine-to-machine context is likely an inappropriate metaphor; nonetheless, it does emphasize the fact that unsafe requests – and requests that have an entity body – require different hypermedia capabilities from simple GETs.

Because in the past I’ve tended to think form fields are redundant in machine-to-machine scenarios, I’ve avoided using forms at all, and have instead overloaded link relations with control data. But link relations are not a “get out of jail free” card. In overloading link relations, we add unnecessary coupling.

Coupling, of course, is not an all-or-nothing affair. There are degrees of coupling. We choose to accept some coupling because of the benefits it brings. But that doesn’t mean we should accept more coupling than is necessary; doing so can only inhibit our ability to evolve a distributed application.

• Following JavaZone, we’ll be at Oslo’s Henning Solberg on Friday, 10th September. You can register for JavaZone here, and for the tutorial here.
• At the beginning of October, we’re at the wonderful JAOO Aarhus, with the tutorial running throughout the day on Friday 8th October. Register here, with a 20% discount if you use the JAOOspeakerfollower promo code.
• Closer to home, we’ll be at Software Architect 2010 in London on Friday, 22nd October. You can register for the conference and tutorial here.

The tutorial agenda closely follows the structure of REST in Practice, which hits the shelves on Sept 24th:

• The Web Architecture: HTTP and URIs
• The Richardson Maturity Model
• CRUD Services using URI templates and HTTP
• Hypermedia and the REST architectural style
• Implementing domain application protocols
• Atom- and AtomPub-based event-driven systems
• Scalability through caching
• Semantics using Microformats and RDF
• Security and the -ilities

With ThoughtWorks recently having opened an office in Germany, I’ll also be presenting at Herbstcampus in Nuremberg, 12-15th October.

And finally, if your want to enjoy some REST in Practice from the comfort of your own office (or home), there’s still time to sign up for the ThoughtWorks’ Webinar, Designing and Implementing RESTful Application Protocols, which takes place on Wednesday, 1st September, at 6.30pm IST (that’s, er, sometime in the afternoon GMT).

I understand that someone recorded the panel session at the end of the day: I’ll post details as soon as the recording becomes available.

In the talk I described how we can implement rich and interesting business processes in (RESTful) Web applications, but only if we think in terms of protocol resources, not coarse-grained domain resources. By embracing the Web as first and foremost a web of data, an open set of resource representations manipulated in the same-old-same-old ways using a closed set of verbs, our designs capture the behaviours most CRUD-based, data-centric applications so sorely lack.

Many thanks to Dan North for inviting me to speak on his Irresponsible Architectures and Unusual Architects track.

Thank you to everyone who came along to the London Geek Night last night. There were many good questions and comments throughout the evening. I didn’t respond to all of them satisfactorily at the time, so I thought I’d expand on a few of them here.

Enforcing the protocol

Alex Scordellis asked a very challenging question: how does the server prevent the client from “teleporting” to locations that aren’t immediately accessible from its current location? In other words, how do we stop the client jumping around the app, ignoring the advertised URIs, going off the rails, and interacting with resources in a way that contravenes the application protocol?

One solution to this problem is to use ephemeral URIs. Remember, other than the entry point URI, every URI the client encounters will have been minted by the server as it generates representations. By appending an expiry time, signed using a private key held only by the server, to each URI, we can ensure that each URI the client is given is valid only for a short period of time. Subbu Allamaraju and Mike Amundsen talk about ephemeral URIs in their forthcoming book, RESTful Web Services Cookbook. Amazon S3 offers the capability to sign URIs in just this fashion so as to limit access to resources.

Of course, this still leaves a short “teleport” window open, which the client can use to make multiple requests of a resource that it ought no longer access. To close this window, we might consider maintaining a secret resource access counter per ephemeral URI inside the server implementation. When the counter limit is reached, the server replies with 204 No Content. The only way for the client to access this resource again would be for it to play fair by the application protocol, navigating advertised transitions until it comes upon a representation containing a link (with new ephemeral URI) to the desired resource.

It can be argued that a GET that increments a secret counter associated with a resource (at an ephemeral URI) is no longer safe. Is the incrementing of a secret counter an unintended user-visible side-effect? The argument is played out in the comments to a post from Sam Ruby from 2002. Read the comments here and make up your own mind.

Resource design

Adewale Oshineye asked how a design ought accommodate making small changes to large resources – is this necessarily an inefficient operation that requires the client to PUT the entire representation back? He also asked how one might support large batch operations in an efficient manner.

HTTP PATCH supplies one mechanism for dealing with partial updates, which fall under Adewale’s first question. Taken together, however, I feel Adewale’s questions lead us to reflect on the role of resource design in the overall design and implementation of a RESTful application. If we model our resources based simply on an understanding of business resources, we can end up with a resource landscape that’s not amenable to being manipulate din the way we require.

The first piece of advice usually given a would-be service designer is: identify your resources and assign them URIs. But this can all too easily lead us to identify only business resources – customer, product, order, etc – and equate these business resources with the resources our service exposes. But remember, the resources we deal with on the Web are information resources, which are somewhat more abstract than the business resources we typically capture in a domain model.

The design and implementation strategy that Jim, Savas and I recommend in REST in Practice is:

1. Design applications in terms of application protocol state machines
2. Implement them in terms of resource lifecycles
3. Advertise/document them using media types, link relation values and HTTP idioms

The transition from 1. to 2. here requires the service designer to decompose an application protocol into whatever information resources and information resource lifecycles are necessary to realise the protocol. The kinds of resources you identify using this approach may look a little different from the ones you would have identified had you taken a business domain resource approach.

I’m being a little vague here, but it’s a subject I plan to develop in more detail at QCon London.

And finally…

In other comments, Bruce Durling pointed out I was mixing up 3rd edition D&D and 1st edition AD&D rules. Guilty as charged. -1 Credibility, no saving throw.

On February 11th I’ll be presenting Hydras and Hypermedia at London Geek Night.

Do you know what your enterprise apps get up to in their time off? Fighting fantasy, pick-your-path, hypermedia-driven, RESTful Web application adventures – of course.

In this speculative dungeon delve I’ll show how we can use hypermedia-driven Web applications to model rich workflows. We’ll tackle the many-headed Hydra of HATEOAS, the “Hypermedia as the Engine of Application State” monster; level up through the Web services maturity heuristic; and meet the dwarves with grudges. On the way, we’ll learn how to model business processes as domain application protocols, implement them in terms of resource lifecycles, and advertise them using HTTP idioms, media types and link relation values.

Event

London Geek Night

Description

Hydras and Hypermedia

Date

11th February

Time

7 pm – 10 pm

Location

ThoughtWorks UK Office

Berkshire House 168-173 High Holborn London WC1V 7AA

A little while ago, Bill de hÓra expressed his preference for direct extension over enveloping. At its worst, enveloping results in near anaemic Atom entries, with very few of the Atom metadata elements put to good use, rightly prompting Bill’s question: “why bother using Atom at all?”

Like Bill, I’ve gone back and forth between these two options. At present, I’m inclined to use enveloping more than direct extension, but only if I can still put the Atom metadata to good use. If I find I’m having to fill out Atom metadata elements with lots of dummy information, just so that I can use Atom to transfer some opaque data between applications, I quickly turn to another format.

To my mind, Atom’s “meta-purpose” is to establish a domain processing context for some content. In AtomPub, Atom artefacts establish a publishing context for Web content. When used to establish this publishing context, the Atom artefacts are called collections and members, rather than feeds and entries.

I prefer not to extend Atom beyond the domain processing context I’m trying to model: instead, I push the thing to be processed down into the content. This allows for media type composition, whereby one media type processor hands off to another as it works its way through a representation. A reasonably “generic” Atom client can resurrect the domain processing context, which then remains in force whilst a specialized media type handler deals with the content (according to its media type).

In the past I’ve illustrated this point by showing how Atom feeds can be used to represent streams of events. The event metadata maps nicely to Atom metadata. The content itself contains a snapshot of some other resource’s state at the point the event occurred. In other words, the Atom metadata establishes an event-ish processing context for the content. When the client invokes a specialised handler for the content, it does so in the knowledge that its dealing with a representation of state at a particular point of time.

If I can’t separate a given problem into 1) an activity and accompanying processing context (e.g. “eventing”, “publishing”), and 2) the thing to be acted on, I’ll consider using something other than Atom.

Call for Papers

The First International Workshop on RESTful Design (WS-REST 2010) aims to provide a forum for discussion and dissemination of research on the emerging resource-oriented style of Web service design.

Background

Over the past few years, several discussions between advocates of the two major architectural styles for designing and implementing Web services (the RPC/ESB-oriented approach and the resource-oriented approach) have been mainly held outside of the research and academic community, within dedicated mailing lists, forums and practitioner communities. The RESTful approach to Web services has also received a significant amount of attention from industry as indicated by the numerous technical books being published on the topic.

This first edition of WS-REST, co-located with the WWW2010 conference, aims at providing an academic forum for discussing current emerging research topics centered around the application of REST, as well as advanced application scenarios for building large scale distributed systems.

In addition to presentations on novel applications of RESTful Web services technologies, the workshop program will also include discussions on the limits of the applicability of the REST architectural style, as well as recent advances in research that aim at tackling new problems that may require to extend the basic REST architectural style. The organizers are seeking novel and original, high quality paper submissions on research contributions focusing on the following topics:

• Applications of the REST architectural style to novel domains
• Design Patterns and Anti-Patterns for RESTful services
• RESTful service composition
• Inverted REST (REST for push events)
• Integration of Pub/Sub with REST
• Performance and QoS Evaluations of RESTful services
• REST compliant transaction models
• Mashups
• Frameworks and toolkits for RESTful service implementations
• Frameworks and toolkits for RESTful service consumption
• Modeling RESTful services
• Resource Design and Granularity
• Evolution of RESTful services
• Versioning and Extension of REST APIs
• HTTP extensions and replacements
• REST compliant protocols beyond HTTP
• Multi-Protocol REST (REST architectures across protocols)

All workshop papers are peer-reviewed and accepted papers will be published as part of the ACM Digital Library. Two kinds of contributions are sought: short position papers (not to exceed 4 pages in ACM style format) describing particular challenges or experiences relevant to the scope of the workshop, and full research papers (not to exceed 8 pages in the ACM style format) describing novel solutions to relevant problems. Technology demonstrations are particularly welcome, and we encourage authors to focus on “lessons learned” rather than describing an implementation.

Papers must be submitted electronically in PDF format. Submit at the WS-REST 2010 EasyChair installation.

Important Dates

• Submission deadline: February 8, 2010, 23.59 Hawaii time
• Notification of acceptance: March 1, 2010
• Camera-ready versions of accepted papers: March 14, 2010
• WS-REST 2010 Workshop: April 26, 2010

Program Committee Chairs

• Cesare Pautasso, Faculty of Informatics, USI Lugano, Switzerland
• Erik Wilde, School of Information, UC Berkeley, USA
• Alexandros Marinos, Faculty of Engineering & Physical Sciences, University of Surrey, UK

Program Committee

• Rosa Alarcon, Pontificia Universidad Catolica de Chile
• Subbu Allamaraju, Yahoo Inc., USA
• Tim Bray, Sun Microsystems, USA
• Bill Burke, Red Hat, USA
• Benjamin Carlyle, Australia
• Stuart Charlton, Elastra, USA
• Joe Gregorio, Google, USA
• Michael Hausenblas, DERI, Ireland
• Rohit Khare, 4K Associates, USA
• Frank Leymann, University of Stuttgart, Germany
• Mark Nottingham, Yahoo Inc., Australia
• Aristotle Pagaltzis, Germany
• Ian Robinson, Thoughtworks, USA
• Richard Taylor, UC Irvine, USA
• Stefan Tilkov, innoQ, Germany
• Steve Vinoski, Verivue, USA
• Jim Webber, Thoughtworks, USA
• Olaf Zimmermann, IBM Zurich Research Lab, Switzerland

Contact

• WS-REST Web site: http://ws-rest.org/
• WS-REST Email: chairs@ws-rest.org

Also as part of QCon, I gave a talk, Beginning an SOA Initiative, the slides of which can be downloaded here.

Jim and I will be giving the REST tutorial at QCon London next year. We’re already planning plenty of changes so as to make it a little more hands-on and practical.