I used to introduce this topic with a quote from Beckett’s Unnameable – “You must go on, I can’t go on, I’ll go on” – which nicely sums up the situation I see many organisations facing today: burdened with a legacy systems estate, unable to stop the world, sorely in need of a change in strategy to face the future.

Based on case study material from the last few years, talk suggests some future proofing strategies that can arise out of an assessment of today’s business operations.

Event Details

• Manchester 16th February, Radisson Hotel, Free Trade Hall, Peter Street, M1 6pm-9pm Register here
• London 19th February, Liverpool Street, EC2 8am-10.30am Register here

Also as part of QCon, I gave a talk, Beginning an SOA Initiative, the slides of which can be downloaded here.

Jim and I will be giving the REST tutorial at QCon London next year. We’re already planning plenty of changes so as to make it a little more hands-on and practical.

The session will be on Live Meeting, at http://snipr.com/virtualaltnet.

Please join us on Monday, times below:

• France/Germany/Belgium: 8:00PM
• UK is: 7:00PM
• EST in the US: 2:00PM
• PST in the US: 11:00AM

Every apostate technician on his or her descent to whiteboard irrelevancy ought produce at least one “magic quadrant” diagram. This is mine: the coupling matrix. Whilst sadly not a career-making architectural silver bullet, this diagram may help you characterize the several parts of your extant solutions, envision direction for future initiatives, and understand the constraints – both legitimate and inadvertent – that in part determine the design options available to you.

First, a brief overview of these two types of coupling:

Temporal coupling

Temporal coupling refers to the degree to which the sending and handling of a message are connected in time. If a sender is dependent on a receiver being available when a message is sent, we have high temporal coupling: if the provider is not available, the interaction fails. Processes whose activities are strictly ordered or whose results carry forward, leaving subsequent activities unable to start until a response to a prior request has been received, are similarly temporally coupled. In situations of high temporal coupling, the time taken to handle the message and return a response increases the processing time on the sender side.

Behavioral coupling

Behavioral coupling refers to the degree to which parties share assumptions regarding behaviors, more specifically, to the implications of assigning the twin responsibilities for determining what action to execute and how to execute that action to the parties in a distributed interaction. In systems that exhibit an extremely high degree of behavioral coupling, the sender of a message determines what to do, and also knows something of how the receiver ought satisfy its request. Such coupling is typically evinced by, for example, database foreign keys and/or platform-dependent artifacts such as collection types leaking into messages and operations. If a sender determines what to do, and a receiver determines how to satisfy the sender’s request, the participants similarly exhibit a high degree of behavioral coupling. If a receiver alone determines both what to do and how to do it in reaction to a received message, the sender and receiver have low behavioral coupling. High behavioral coupling requires a provider to evolve its service offerings in response to changed consumer requirements.

And then some sketchy characterizations of each of the quadrants:

Distributed 3-layer

Traditional 3-layer application architecture blown up to distributed proportions. Characterized by call stack-like, imperative behaviour (high temporal and behavioural coupling) and synchronous request-response interactions. Includes systems that layer synchronous interactions on top of asynchronous message exchanges. Senders tell receivers what to do; receivers execute the sender’s orders. Sender and all intermediaries block until the call stack unwinds, effectively locking and/or consuming system resources further up the call chain. This blocking behaviour undermines the autonomy of upstream components and at the same time increases the availability requirements of downstream components. As Michael Nygard’s Release It! reminds us, in these circumstances the availability of the overall system can be no more than that of the least available participant, and the probability of failure is the joint probability of failure in any component or service.

Command-oriented

“Good”, “orthodox” SOA. Low degree of temporal coupling characterised by asynchronous interactions, deferred state and a resumable programming model (process or activity instances are dehydrated between remote invocations in order to conserve resources, and then rehydrated based on correlated responses). Senders typically determine what needs to be done, but rely on receivers to determine how to execute their instructions. This behavioural coupling can require providers to evolve (message formats, supported operations) in lockstep with changing consumer demands.

Event-oriented

Low temporal and behavioural coupling. Receivers determine both what needs to be done and how to do it based on the content of received messages. Resumable programming model: processes are suspended or dehydrated, waiting for events. Can be difficult to trace the execution path of an end-to-end transaction or activity. Exposing an ExtinguishFire operation is a command-oriented way of executing a business process; acting on FireStarted notifications an event-oriented approach.

Emergency services

So called because you tell them what happened, and they decide what to do, but if there’s no one to take your call, you’re hosed. Low behavioural coupling, which allows for the independent evolution of system components, but a degree of temporal coupling, impacting availability requirements of participants. Many RESTful solutions occupy this quadrant. URI-templated solutions have a higher degree of behavioural coupling than hypermedia-driven solutions (where servers constrain and guide what a client can do next, and determine how best to satisfy requests); client polling and caching can mitigate some of these temporal coupling issues.

Endnote

These coupling issues are well understood by many developers and architects – to the extent they comprise a general base of knowledge without specific authority or provenance. I would, however, like to draw your attention to one of very many excellent posts by Bill Poole, Avoid Command Messages, that some time ago helped bring behavioural coupling into focus for me.

Over the course of a piece of analysis some things, some entities, will emerge as being of particular significance to the business. Customer and order, for example. Don’t refuse to acknowledge or model these things, but don’t get hung up on them either: they’re just a part of a whole that includes not only concept models, but lifecycle models and capability maps.

Take customer for example. We can, very quickly, create a high-level picture of what a customer might look like – an overarching, company-wide view of all the parts of a customer that are of interest to the business, or the several parts of the business. But this isn’t the domain model: rather it’s an approximation, a temporary stabilizing of multiple shifting perspectives, a flawed starting point – just good enough – for further investigation.

Side-by-side with this high-level conceptual view of a customer, we can also model the customer lifecycle; that is, the story of a customer’s relationship with our company. I’ll often model this in the form of a state machine. We then see that our interest in a customer can be expressed in terms of our transitioning the customer through several different states.

For each transition in this state machine, we can now ask: what business capabilities do we need in order to transition the customer from, eg. lead to applicant? How well do we implement these capabilities today? And so on.

Capabilities operate on the customer in order to transition it through different states, in so doing releasing value both to the customer and the business.

What likely starts to emerge are business-meaningful bounded contexts, each context encapsulating part of the high-level picture, plus one or more state transitions, plus the capabilities necessary to effect those transitions. Each bounded context projects a different view of the customer onto the business landscape – suggesting there’s not necessarily an organization-wide canonical model of a customer, just many “interested” views. That’s why I say our original high-level picture isn’t the domain model: it’s simply a starting point for identifying the bounded contexts – and the processes – that transition the overarching concept “customer” through states in the customer lifecycle.

What we end up with is a high-level model/picture divided by bounded contexts, a customer lifecycle model, and a capability map. Together these comprise part of an operating model – a useful representation of the organization that we can use to identify desirable business outcomes and guide the identification, planning and prioritization of projects.

I also had a couple of requests to make my hand-drawn UML stencil for OmniGraffle available. And so, for your delight: Napkin UML. Be warned: it’s very rough and ready. The font I used in my presentation was FG Jayne Print.

On Friday, I’ll be catching up with Stefan again when he interviews Jim Webber and me on REST for SOA: Using the Web for Integration. I’m currently going through some back issue of Viz so that I can keep up with Jim’s “pattern language.”

At the end of the afternoon I’ll be joining the panel – Martin Fowler, Michael T. Nygard and Steve Vinoski – for Game show: It’s a Bullseye! with Jim Webber. After all, every quiz show needs one fake.

Speaking of all things QCon-wise, InfoQ have recently published Ryan Slobojan’s interview with me from QCon San Francisco last year. I’m surprised that there was so much to show from this. Ryan asked some great questions around REST, WS-* and SOA, but jet-lag had the better of me: thankfully, InfoQ have edited out the parts where I give up on putting one word in front of another.

The answer was close to hand: user stories. The added depth that the story format brought to the table really invigorated the capability mapping process – not only does a story describe a goal, it also describes a user (or organisational) role and the value attached to realising a goal. Used judiciously – after all, it’s not always wise to insist that every utterance use the story format – story writing can greatly help in discovering and mapping capabilities.

And in pairing user stories and capabilities, I learnt something interesting about stories.

Goals or Features?

I’ve written briefly in the past about using stories and capabilities to create a business-meaningful model of an organisation. When I pair stories and capabilities, I say that stories describe strategic organisational goals; capabilities describe what an organisation must provide to satisfy those goals. How an organisation meets its strategic goals is another matter: capabilities can be implemented in many different ways, and the how varies over time. The capability to source replacement parts for used cars, for example, might today be implemented by harried customer service staff leafing through papers in a filing cabinet; tomorrow it might be implemented using an online application backed by a relational database.

If you’re familiar with any kind of agile requirements analysis process, whether based on Mike Cohn’s “user stories”, or Behaviour-Driven Development’s example-based specifications, you might think it a little odd when I say that stories describe goals. User stories and BDD exemplars typically describe features. Moreover, they do so in way that is estimable and testable. The purpose of stories is to guide software development in a way that is meaningful to end-users and stakeholders. In the course of describing features they might allude to or otherwise encode user goals, but the goals would not appear to be ends in themselves: they simply motivate the features that drive specific instances of software development.

So in saying that stories describe goals we appear to have reduced the usefulness of the story format. Without features, user stories feel a little too sparse, and of insufficient value to feature-focussed software development.

Nonetheless, when I use stories to drive out capabilities, I use a weak form of the story format – one that aims at describing only goals not features. In effect, I unpack a feature into its constituent parts: on the one hand a goal (or need), on the other, a capability.

Why would I do that? Because goals can be satisfied in many different ways. Just as capabilities are amenable to multiple implementations, so high-level goals can be satisfied through a range of possible capabilities.

This isn’t really news to the agile community, of course. Both Dan North and Martin Fowler have discussed the double-edged power of the user story, the benefits and dangers that come from baking features into user stories. The only novelty here concerns the degree to which I make explicit and control the split between goals and capabilities. By distinguishing between the two, I can “slow down” the analysis process and tease goals from capabilities so as to focus on the capabilities, or I can speed things up and get at the features.

When the purpose of the exercise is to identify and map capabilities, I enforce this split quite rigorously. The fine-grained control over the analysis process this gives me helps emphasize the distinctive nature of the capability concept, differentiating it from goals on the one hand, processes on the other. The power of variability comes into play here, helping us progress the conversation and get value from the workshop. In practical terms, this means we can anchor a goal and then iterate capabilities until we’re happy we’ve found the right set of capabilities and an appropriate language to describe them.

When gunning for development, however, I’m often quite happy to roll goals and capabilities back up into features, though always with the acknowledgement that I’m assuming a more one-to-one relationship between a goal and a capability.

In both situations, I’ll stress the need to specify a meaningful value for the story. I’ll also be on the lookout for inadvertent lapses into specifying solutions rather than goals, and I’ll quite happily employ the shift left approach to highlight poorly constructed stories.

From Operating Model to Business Architecture

A little more context may be useful here. I use the story and capability pairing when running high-level capability discovery workshops with business unit heads or their delegates, heads of the lines of business, strategy makers, etc. These people are typically responsible for specifying what their part of the organisation must achieve, what capabilities it must possess; they have little direct responsibility for determining how those capabilities are implemented on a day-by-day basis. (For insight into the lower-level capabilities and the how of execution, I speak to operational staff.)

Used in this way, the stories are not necessarily estimable or testable. But because they’re not being used to guide software development, that’s not an issue. The stories are vehicles to get to capabilities.

The purpose of these capability discovery workshops is to create a network map of an organisation’s high-level capabilities. These maps link an organisation’s operating model to its business architecture. Subsequent iterations of the capability map lay the groundwork for service discovery. The workshops themselves help establish context and create consensus amongst stakeholders; at the same time, they serve as a platform for interrogating and challenging an organisation’s goals and benefits. In this way they provide for early course corrections towards the beginning of an IT or business initiative.

• Stories – in the form of Behaviour-Driven Development exemplars – describe goals and desired outcomes;
• Capabilities encapsulate the resources and abilities an organisation needs to satisfy those goals;
• Services host capabilities;
• Contracts – particularly consumer-driven contracts – assert the interactions between services.

We iterate over this model many times in the course of an engagement, emphasising different parts as we go. We concentrate on organisation-level stories and capabilities when establishing an organisation-wide context, then project-level stories, capabilities and services whilst planning projects and delivery streams, and then release- and iteration-level artefacts during specific instances of delivery.

Such an approach represents a “thin-slice” way to:

• establish context, business goals and consensus amongst stakeholders;
• create a long-term vision that joins up the business, architectural and technology views of an SOA initiative;
• describe and challenge an organisation’s goals and the benefits attached to those goals;
• describe the capabilities needed to meet those goals;
• identify the quality-of-service expectations the business has of those capabilities;
• identify services and assign capabilities to services;
• describe and test the externally visible interactions between services;
• identify, plan and develop slices of service functionality that deliver business benefits early and often.