David Ingham

Video of SuperMondays Service Bus Presentation

Here's the video from my Service Bus presentation from last week's SuperMondays session. The slides are up on the presentations page and over at SlideShare.

Enterprise Messaging Session at SuperMondays

Yesterday I had the opportunity to attend my first SuperMondays event. To quote the Web site, SuperMondays is "a strong and vibrant IT community based in the North-East of England." Based on last night's experience, I'd agree with that description, it does indeed appear to be a strong community and now that I'm back in Newcastle, I'll be attending regularly. 

Ross Cooney, one of the organizers, took the opportunity to recruit several members of the AMQP working group, who were in town for a connect-a-thon event, to create an event focused on enterprise messaging. I gave a demo of Windows Azure AppFabric Service Bus, highlighting the new message queuing and publish/subscribe features that we've added as part of the May CTP. I think it went down pretty well with the audience. I've put my slides up on the presentations page but they're not very useful without the accompanying demo. I think a video of the session will be put up on the SuperMondays site soon.

 

An Introduction to Service Bus Topics

In the May CTP of Service Bus, we’ve added a brand-new set of cloud-based, message-oriented-middleware technologies including reliable message queuing and durable publish/subscribe messaging. Last week I posted the Introduction to Service Bus Queues blog entry. This post follows on from that and provides an introduction to the publish/subscribe capabilities offered by Service Bus Topics. Again, I’m not going to cover all the features in this article, I just want to give you enough information to get started with the new feature. We’ll have follow-up posts that drill into some of the details.

I’m going to continue with the retail scenario that I started in the queues blog post. Recall that sales data from individual Point of Sale (POS) terminals needs to be routed to an inventory management system which uses that data to determine when stock needs to be replenished. Each POS terminal reports its sales data by sending messages to the DataCollectionQueue where they sit until they are received by the inventory management system as shown below:

Now let’s evolve this scenario. A new requirement has been added to the system: the store owner wants to be able to monitor how the store is performing in real-time.

To address this requirement we need to take a “tap” off the sales data stream. We still want each message sent by the POS terminals to be sent to the Inventory Management System as before but we want another copy of each message that we can use to present the dashboard view to the store owner.

In any situation like this, where you need each message to be consumed by multiple parties, you need the Service Bus Topic feature. Topics provide the publish/subscribe pattern in which each published message is made available to each subscription registered with the Topic. Contrast this with the queue where each message is consumed by a single consumer. That’s the key difference between the two models.

Messages are sent to a topic in exactly the same way as they are sent to a queue but messages aren’t received from the topic directly, instead they are received from subscriptions. You can think of a topic subscription like a virtual queue that gets copies of the messages that are sent to the topic. Messages are received from a subscription in exactly the same way as they are received from a queue.

So, going back to the scenario, the first thing to do is to switch out the queue for a topic and add a subscription that will be used by the Inventory Management System. So, the system would now look like this:

The above configuration would perform identically to the previous queue-based design. That is, messages sent to the topic would be routed to the Inventory subscription from where the Inventory Management System would consume them.

Now, in order to support the management dashboard, we need to create a second subscription on the topic as shown below:

Now, with the above configuration, each message from the POS terminals will be made available to both the Dashboard and Inventory subscriptions.

Show Me the Code

I described how to sign-up for a Service Bus account and create a namespace in the queues blog post so I won’t cover that again here. Recall that to use the Service Bus namespace, an application needs to reference the AppFabric Service Bus DLLs, namely Microsoft.ServiceBus.dll and Microsoft.ServiceBus.Messaging.dll. You can find these as part of the SDK download.

Creating the Topic and Subscriptions

Management operations for Service Bus messaging entities (queues and topics) are performed via the ServiceBusNamespaceClient which is constructed with the base address of the Service Bus namespace and the user credentials. The ServiceBusNamespaceClient provides methods to create, enumerate and delete messaging entities. The snippet below shows how the ServiceBusNamespaceClient is used to create the DataCollectionTopic.

    Uri ServiceBusEnvironment.CreateServiceUri("sb", "ingham-blog", string.Empty);

    string name = "owner";

    string key = "abcdefghijklmopqrstuvwxyz";

  

    ServiceBusNamespaceClient namespaceClient = new ServiceBusNamespaceClient(

        baseAddress, TransportClientCredentialBase.CreateSharedSecretCredential(name, key) );

 

    Topic dataCollectionTopic = namespaceClient.CreateTopic("DataCollectionTopic"); 

Note that there are overloads of the CreateTopic method that allow properties of the topic to be tuned, for example, to set the default time-to-live to be applied to messages sent to the topic. Next, let’s add the Inventory and Dashboard subscriptions.

    dataCollectionTopic.AddSubscription("Inventory");
    dataCollectionTopic.AddSubscription("Dashboard");

Sending Messages to the Topic

As I mentioned earlier, applications send messages to a topic in the same way that they send to a queue so the code below will look very familiar if you read the queues blog post. The difference is the application creates a TopicClient instead of a QueueClient.

For runtime operations on Service Bus entities, i.e., sending and receiving messages, an application first needs to create a MessagingFactory. The base address of the ServiceBus namespace and the user credentials are required.

    Uri ServiceBusEnvironment.CreateServiceUri("sb", "ingham-blog", string.Empty);

    string name = "owner";

    string key = "abcdefghijklmopqrstuvwxyz";

 

    MessagingFactory factory = MessagingFactory.Create(

        baseAddress, TransportClientCredentialBase.CreateSharedSecretCredential(name, key) );

  

From the factory, a TopicClient is created for the particular topic of interest, in our case, the DataCollectionTopic.

 

    TopicClient topicClient = factory.CreateTopicClient("DataCollectionTopic");

 

A MessageSender is created from the TopicClient to perform the send operations.

 

    MessageSender ms = topicClient.CreateSender();

 

Messages sent to, and received from, Service Bus topics (and queues) are instances of the BrokeredMessage class which consists of a set of standard properties (such as Label and TimeToLive), a dictionary that is used to hold application properties, and a body of arbitrary application data. An application can set the body by passing in any serializable object into CreateMessage (the example below passes in a SalesData object representing the sales data from the POS terminal) which will use the DataContractSerializer to serialize the object. Alternatively, a System.IO.Stream can be provided.

 

    BrokeredMessage bm = BrokeredMessage.CreateMessage(salesData);

    bm.Label = "SalesReport";

    bm.Properties["StoreName"] = "Redmond";

    bm.Properties["MachineID"] = "POS_1";

 

    ms.Send(bm);

Receiving Messages from a Subscription

Just like when using queues, messages are received from a subscription using a MessageReceiver. The difference is that the MessageReceiver is created from a SubscriptionClient rather than a QueueClient. Everything else remains the same including support for the two different receive modes (ReceiveAndDelete and PeekLock) that I discussed in the queues blog post.

So, first we create the SubscriptionClient, passing the name of the topic and the name of the subscription as parameters. Here I’m using the Inventory subscription.

    SubscriptionClient subClient = factory.CreateSubscriptionClient("DataCollectionTopic", "Inventory"); 

Next we create the MessageReceiver and receive a message.

    MessageReceiver mr = subClient.CreateReceiver();

    BrokeredMessage receivedMessage = mr.Receive();

    try

    {

        ProcessMessage(receivedMessage);

        receivedMessage.Complete();

    }

    catch (Exception e)

    {

        receivedMessage.Abandon();

    }

Subscription Filters

So far, I’ve said that all messages sent to the topic are made available to all registered subscriptions. The key phrase there is “made available”. While Service Bus subscriptions see all messages sent to the topic, it is possible to only copy a subset of those messages to the virtual subscription queue. This is done using subscription filters. When a subscription is created, it’s possible to supply a filter expression in the form of a SQL92 style predicate that can operate over the properties of the message, both the system properties (e.g., Label) and the application properties, such as StoreName in the above example.

Let’s evolve the scenario a little to illustrate this. A second store is to be added to our retail scenario. Sales data from all of the POS terminals from both stores still need to be routed to the centralized Inventory Management System but a store manager using the Dashboard tool is only interested in the performance of her store. We can use subscription filtering to achieve this. Note that when the POS terminals publish messages, they set the StoreName application property on the message. Now we have two stores, let’s say Redmond and Seattle, the POS terminals in the Redmond store stamp their sales data messages with a StoreName of Redmond while the Seattle store POS terminals use a StoreName of Seattle. The store manager of the Redmond store only wishes to see data from its POS terminals. Here’s how the system would look:

To set this routing up, we need to make a simple change to how we’re creating the Dashboard subscription as follows:

    dataCollectionTopic.AddSubscription("Dashboard", new SqlFilterExpression("StoreName = 'Redmond'"); 

With this subscription filter in place, only messages with the StoreName property set to Redmond will be copied to the virtual queue for the Dashboard subscription.

There is a bigger story to tell around subscription filtering. Applications have an option to have multiple filter rules per subscription and there’s also the ability to modify the properties of a message as it passes in to a subscription’s virtual queue. We’ll cover these advanced topics in a separate blog post.

Wrapping up

Hopefully this post has shown you how to get started with the topic-based publish/subscribe feature being introduced in the new May CTP of Service Bus.

It’s worth noting that all of the reasons for using queuing that I mentioned in the introduction to queuing blog post also apply to topics, namely:

• Temporal decoupling – message producers and consumers do not have to be online at the same time.
• Load leveling – peaks in load are smoothed out by the topic allowing consuming applications to be provisioned for average load rather than peak load.
• Load balancing – just like with a queue, it’s possible to have multiple competing consumers listening on a single subscription with each message being handed off to only one of the consumers, thereby balancing load.
• Loose coupling – it’s possible to evolve the messaging network without impacting existing endpoints, e.g., adding subscriptions or changing filters to a topic to accommodate new consumers.

We’ve only really just scratched the surface here; we’ll go in to more depth in future posts.

Finally, remember one of the main goals of our CTP release is to get feedback on the service. We’re interested to hear what you think of the Service Bus messaging features. We’re particularly keen to get your opinion of the API. So, if you have suggestions, critique, praise, or questions, please let us know at http://social.msdn.microsoft.com/Forums/en-US/appfabricctp/. Your feedback will help us improve the service for you and other users like you.

 

An Introduction to Service Bus Queues

In the new May CTP of Service Bus, we’re adding a brand-new set of cloud-based, message-oriented-middleware technologies including reliable message queuing and durable publish/subscribe messaging. We’ll walk through the full set of capabilities over a series of blog posts but I’m going to begin by focusing on the basic concepts of the message queuing feature. This post will explain the usefulness of queues and show how to write a simple queue-based application using Service Bus.

Let’s consider a scenario from the world of retail in which sales data from individual Point of Sale (POS) terminals needs to be routed to an inventory management system which uses that data to determine when stock needs to be replenished. I’m going to walk through a solution that uses Service Bus messaging for the communication between the terminals and the inventory management system as illustrated below:

Each POS terminal reports its sales data by sending messages to the DataCollectionQueue. These messages sit in this queue until they are received by the inventory management system. This pattern is often termed asynchronous messaging because the POS terminal doesn’t need to wait for a reply from the inventory management system to continue processing.

Why Queuing?

Before we look at the code required to set up this application, let’s consider the advantages of using a queue in this scenario rather than having the POS terminals talk directly (synchronously) to the inventory management system.

Temporal decoupling

With the asynchronous messaging pattern, producers and consumers need not be online at the same time. The messaging infrastructure reliably stores messages until the consuming party is ready to receive them. This allows the components of the distributed application to be disconnected, either voluntarily, e.g., for maintenance, or due to a component crash, without impacting the system as a whole. Furthermore, the consuming application may only need to come online during certain times of the day, for example, in this retail scenario, the inventory management system may only need to come online after the end of the business day.

Load leveling

In many applications system load varies over time whereas the processing time required for each unit of work is typically constant. Intermediating message producers and consumers with a queue means that the consuming application (the worker) only needs to be provisioned to accommodate average load rather than peak load. The depth of the queue will grow and contract as the incoming load varies. This directly saves money in terms of the amount of infrastructure required to service the application load.

Load balancing

As load increases, more worker processes can be added to read from the work queue. Each message is processed by only one of the worker processes. Furthermore, this pull-based load balancing allows for optimum utilization of the worker machines even if the worker machines differ in terms of processing power as they will pull messages at their own maximum rate. This pattern is often termed the competing consumer pattern.

Loose coupling

Using message queuing to intermediate between message producers and consumers provides an inherent loose coupling between the components. Since producers and consumers are not aware of each other, a consumer can be upgraded without having any effect on the producer. Furthermore, the messaging topology can evolve without impacting the existing endpoints – we’ll discuss this further in a later post when we talk about publish/subscribe.

Show me the Code

Alright, now let’s look at how to use Service Bus to build this application.

Signing up for a Service Bus account and subscription

Before you can begin working with the Service Bus, you’ll first need to sign-up for a Service Bus account within the Service Bus portal at http://portal.appfabriclabs.com/. You will be required to sign-in with a Windows Live ID (WLID), which will be associated with your Service Bus account. Once you’ve done that, you can create a new Service Bus Subscription. In the future, whenever you login with your WLID, you will have access to all of the Service Bus Subscriptions associated with your account.

Creating a namespace

Once you have a Subscription in place, you can create a new service namespace. You’ll need to give your new service namespace a unique name across all Service Bus accounts. Each service namespace acts as a container for a set of Service Bus entities. The screenshot below illustrates what this page looks like when creating the “ingham-blog” service namespace.

Further details regarding account setup and namespace creation can be found in the User Guide accompanying the May CTP release.

Using the Service Bus

To use the Service Bus namespace, an application needs to reference the AppFabric Service Bus DLLs, namely Microsoft.ServiceBus.dll and Microsoft.ServiceBus.Messaging.dll. You can find these in the SDK that can be downloaded from the CTP download page.

Creating the Queue

Management operations for Service Bus messaging entities (queues and publish/subscribe topics) are performed via the ServiceBusNamespaceClient which is constructed with the base address of the Service Bus namespace and the user credentials. The ServiceBusNamespaceClient provides methods to create, enumerate and delete messaging entities. The snippet below shows how the ServiceBusNamespaceClient is used to create the DataCollectionQueue.

    Uri ServiceBusEnvironment.CreateServiceUri("sb", "ingham-blog", string.Empty); 

    string name = "owner";
    string key = "abcdefghijklmopqrstuvwxyz";

    ServiceBusNamespaceClient namespaceClient = new ServiceBusNamespaceClient(
         baseAddress, TransportClientCredentialBase.CreateSharedSecretCredential(name, key) );

    namespaceClient.CreateQueue("DataCollectionQueue");

Note that there are overloads of the CreateQueue method that allow properties of the queue to be tuned, for example, to set the default time-to-live to be applied to messages sent to the queue.

Sending Messages to the Queue

For runtime operations on Service Bus entities, i.e., sending and receiving messages, an application first needs to create a MessagingFactory. The base address of the ServiceBus namespace and the user credentials are required.

    Uri ServiceBusEnvironment.CreateServiceUri("sb", "ingham-blog", string.Empty); 

    string name = "owner";
    string key = "abcdefghijklmopqrstuvwxyz"; 

     MessagingFactory factory = MessagingFactory.Create(
        baseAddress, TransportClientCredentialBase.CreateSharedSecretCredential(name, key) );

From the factory, a QueueClient is created for the particular queue of interest, in our case, the DataCollectionQueue. 

    QueueClient queueClient = factory.CreateQueueClient("DataCollectionQueue");

 A MessageSender is created from the QueueClient to perform the send operations.

    MessageSender ms = queueClient.CreateSender();

Messages sent to, and received from, Service Bus queues are instances of the BrokeredMessage class which consists of a set of standard properties (such as Label and TimeToLive), a dictionary that is used to hold application properties, and a body of arbitrary application data. An application can set the body by passing in any serializable object into CreateMessage (the example below passes in a SalesData object representing the sales data from the POS terminal) which will use the DataContractSerializer to serialize the object. Alternatively, a System.IO.Stream can be provided. 

    BrokeredMessage bm = BrokeredMessage.CreateMessage(salesData); 

    bm.Label = "SalesReport";
    bm.Properties["StoreName"] = "Redmond";
    bm.Properties["MachineID"] = "POS_1";

    ms.Send(bm); 

Receiving Messages from the Queue

Messages are received from the queue using a MessageReceiver which is also created from the QueueClient. MessageReceivers can work in two different modes, named ReceiveAndDelete and PeekLock. The mode is set when the MessageReceiver is created, as a parameter to the CreateReceiver operation.

When using the ReceiveAndDelete mode, receive is a single-shot operation, that is, when Service Bus receives the request, it marks the message as being consumed and returns it to the application. ReceiveAndDelete mode is the simplest model and works best for scenarios in which the application can tolerate not processing a message in the event of a failure. To understand this, consider a scenario in which the consumer issues the receive request and then crashes before processing it. Since Service Bus will have marked the message as being consumed then when the application restarts and begins consuming messages again, it will have missed the message that was consumed prior to the crash.

In PeekLock mode, receive becomes a two stage operation which makes it possible to support applications that cannot tolerate missing messages. When Service Bus receives the request, it finds the next message to be consumed, locks it to prevent other consumers receiving it, and then returns it to the application. After the application finishes processing the message (or stores it reliably for future processing), it completes the second stage of the receive process by calling Complete on the received message. When Service Bus sees the Complete, it will mark the message as being consumed.

Two other outcomes are possible. Firstly, if the application is unable to process the message for some reason then it can call Abandon on the received message (instead of Complete). This will cause Service Bus to unlock the message and make it available to be received again, either by the same consumer or by another completing consumer. Secondly, there is a timeout associated with the lock and if the application fails to process the message before the lock timeout expires (e.g., if the application crashes), then Service Bus will unlock the message and make it available to be received again.

One thing to note here is that in the event that the application crashes after processing the message but before the Complete request was issued then the message will be redelivered to the application when it restarts. This is often termed At Least Once processing, that is, each message will be processed at least once but in certain situations the same message may be redelivered. If the scenario cannot tolerate duplicate processing then additional logic is required in the application to detect duplicates which can be achieved based upon the MessageId property of the message which will remain constant across delivery attempts. This is termed Exactly Once processing.

Back to the code, the snippet below illustrates how a message can be received and processed using the PeekLock mode which is the default if no ReceiveMode is explicitly provided.

    MessageReceiver mr = queueClient.CreateReceiver();
    BrokeredMessage receivedMessage = mr.Receive();

    try
    {
        ProcessMessage(receivedMessage);
        receivedMessage.Complete();
    }
    catch (Exception e)
    {
        receivedMessage.Abandon();
    }

Wrapping up and request for feedback

Hopefully this post has shown you how to get started with the queuing feature being introduced in the new May CTP of Service Bus. We’ve only really just scratched the surface here; we’ll go in to more depth in future posts.

Finally, remember one of the main goals of our CTP release is to get feedback on the service. We’re interested to hear what you think of the Service Bus messaging features.

We’re particularly keen to get your opinion of the API, for example, do you think it makes sense to have PeekLock be the default mode for receivers? We have a survey for that question.

For other suggestions, critique, praise, or questions, please let us know at the AppFabric CTP forum. Your feedback will help us improve the service for you and other users like you.

My SXSW 2010

3/16: Boxcar Preachers, Ha Ha Tonka, The Silos, The Savage Trip, Tom Freund. 3/17: Drive-By Truckers / The Beauties / Justin Rutledge / Hollerado / Honeyhoney / Cocktail Slippers / Gin Wigmore / Billy Bragg. 3/18: Codeine Velvet Club / Dr Dog / Gin Wigmore / The Blue Aeroplanes / Joe Purdy / Titus Andronicus / The Mother Hips / Street Sweeper Social Club / Deer Tick. 3/19: The Broderick / Bear Hands / Jakob Dylan and Three Legs (Featuring Neko Case and Kelly Hogan) / Band of Skulls / Harper Simon & friends / Murder By Death / Black Rebel Motorcycle Club / Chuck Prophet & Mission Express. 3/20: Frightened Rabbit / Ten Bears / John Smith / Lovely Eggs / The Wave Machines / Ben Weaver / Exene Cervenka, Jon Allen / Big Star – Alex Chilton Tribute with Curt Kirkwood, Mike Mills, John Doe, M Ward, Sondre Lerche, Chris Stamey, Chuck Prophet, Evan Dando, Amy Speace, The Watson Twins, Susan Cowsill, Andy Hummel.

ApacheCon US 2009

I'm writing this from ApacheCon in Oakland. Apache is celebrating the 10 year anniversary of the Apache Software Foundation (ASF). My first involvement with Apache was way back in 1995 when I wrote a plug-in module for the Apache HTTP Server as part of my W3Objects research work. It's amazing how ASF has grown in the intervening years.

My main reason for being here is to talk about AMQP and Apache Qpid.

Microsoft joined the AMQP Working Group a year or so ago and I'm our representative there. I'm also the technical lead for the work we've been doing to make Windows a great platform for Qpid. Specifically we've made the following contributions to Qpid:

• Ported the C++ broker and client library to Windows
• Added a WCF channel to allow .NET developers to write clients that communicate using AMQP using our standard network programming API
• Added a Microsoft SQL-based persistence store for the Qpid broker
• Packaging for Windows including a Windows installer with samples

I've been spreading the word at every opportunity.

First was BarCamp. It was the first time that I've attended an "unconference" but I enjoyed it very much. If you haven't heard of BarCamp, it works like this - the organizers provide some basic infrastructure (rooms, projectors, etc.) but there is no program defined in advance. The attendees turn up prepared to speak on some topic and then spend the first 30 minutes of the day devising the schedule for the day. I gave a session on the challenges of enterprise messaging in a heterogeneous environment, covering AMQP and Apache Qpid. It was a good session with lots of discussion. [slides here]

A number of Apache project meetups were organized for ApacheCon and a last-minute slot opened up. Jonathan Robie, a Qpid contributor from Red Hat, stepped up and organized a Qpid meetup. Unfortunately there wasn't a big attendance due to the lack of advance notice. We reconvened to the bar ;-)

Yesterday I presented again in a joint session with Kent Brown of Microsoft and Prabath Siriwardena of WSO2. The title was "Interoperability Through Community" and covered Apache Stonehenge and Apache Qpid. Stonehenge is an Incubator project focused on Web services interoperability. They've built a real-world application, StockTrader, that demonstrates through code how to achieve Web services interoperability between different Web services stacks. Today this includes Microsoft, Spring Source, Sun and WSO2. I gave a pitch on AMQP and Apache Qpid as solutions for building heterogeneous message-oriented middleware applications. [slides here]

So, it's been a good few days geeking out at ApacheCon. I plan to attend again.

2009 AMQP Conference

I spent the last 3 days in La Jolla Shores near San Diego for the AMQP Conference. Microsoft joined the AMQP Working Group in October 2008 and I’m our representative in the AMQP Project Management Committee. This was my first opportunity to meet many of the working group members face-to-face; it was good to put faces to the disembodied voices that I’ve been talking with on the phone every week.

The setting for the conference, the Scripps Institution of Oceanography at the University of California San Diego, was just perfect; right on the beach with the sun shining and sound of waves crashing in the background. Many thanks to Matthew Arrott and Rita Bauer for making us feel so welcome.

The first day of the conference was open to the public and I’m pleased to say that we were full to capacity. Representatives from a broad range of companies came to hear about AMQP and the details of the upcoming 1.0 version of the specification. John O’Hara (JPMorgan), the originator and Chairman of the AMQP Working Group, began with an introduction to AMQP that covered the motivation and real-world use cases that it’s designed to address. Mark Blair (Credit Suisse) then presented the findings of the User SIG - the features and characteristics deemed necessary by the end-users participating in the working group. Next up was an “AMQP in detail” session presented by Rob Godfrey (JPMorgan) and Rafi Schloming (Red Hat) that covered the new AMQP 1.0 messaging model and wire-level transport. The vendors of the main three AMQP implementations, iMatix, Rabbit Technologies and Red Hat, then described how their products were being used in real-world deployments today. Finally there was a chance for the audience to provide feedback. I’m pleased to say that response was uniformly positive.

The slides from the session are available:

• Introduction to AMQP and User SIG findings
• AMQP technical overview
• AMQP transport overview

The remaining two days were a closed session of the AMQP Working Group. Good progress was made on a range of subjects. I was impressed how a group of people with diverse opinions were able to work together productively and reach consensus on a range of issues.

The next few months are going to be a busy time for the Working Group as we finalize the 1.0 version of the specification. The pressure is on as if this event is anything to go by, there’s a real demand out there.

My SXSW 2009

3/17: Andy Friedman, Romantica, Freedy Johnston (w/Jon Dee Graham), The Savage Trip (w/Michael Hall), Grant Hart (from Husker Du), The Silos (w/Jon Dee Graham), Amy Cook, The Blue Aeroplanes, 3/18: Rebekah Pulley and the Reluctant Prophets, Ronnie Elliott, Hoots & Hellmouth, Jon Dee Graham, James McMurtry, Randy Weeks, Ladyhawke, The Heartless Bastards, The Avett Brothers, The Decemberists, 3/19: The Magic, Human Highway, The Hard Lessons, Lucero, The Hold Steady, Dead Confederate, Phosphorescent, The Black Lips, Primal Scream, 3/20: Little Steven (keynote), The Mighty Stef, The Brothers Movement, The Thermals, The Hold Steady, Howard Elliot Payne, Hamell On Trial, Ed Harcourt, The Whispertown 2000, The Felice Brothers, The Hold Steady, The Shys, The Chesterfield Kings, 3/21: The Hold Steady (interview), Delta Spirit, Fanfarlo, Wallis Bird, Echo and The Bunnymen, James Harries, Moriarty, Silversun Pickups, News on the March.

Microsoft joins the AMQP working group

I first heard about AMQP (the Advanced Message Queuing Protocol) back in early 2005 when John Davies first announced its existence at the Web Services on Wall Street Conference. I loved the concept of an open standard for messaging middleware and was excited by the potential for wire-level interoperability for enterprise messaging.

I’ve been tracking the progress of AMQP ever since. The formation of the AMQP.org working group in summer 2006 was another great milestone. Today AMQP.org consists of a broad group of software vendors and end-users: Cisco Systems, Credit Suisse, Deutsche Börse Systems, Envoy Technologies, Inc., Goldman Sachs, IONA Technologies PLC, iMatix Corporation sprl., JPMorgan Chase Bank Inc. N.A, Novell, Rabbit Technologies Ltd., Red Hat, Inc., TWIST Process Innovations Ltd, WS02, Inc. and 29West Inc. Last

Last Friday, Microsoft joined the party by becoming a member of AMQP.org. You can get more information from the press release and Sam Ramji’s blog posting.

I’m excited. More on this to come…

Windows Home Server Power Pack 1

Announced yesterday, Power Pack 1 for the excellent Windows Home Server is finally available. If you have automatic updates enabled then you'll eventually get this via Windows Update. If you can't wait, you can get it now at the Microsoft Download Center. For me, the main reason to pick this up is the fix to the data corruption bug but there are a few bonus features too:

• Support for PCs running Windows Vista x64 editions
• Backup of home server Shared Folders
• Easier, enhanced remote access capabilities
• Better energy efficiency
• Improved performance

More info here.

I'm currently working from the UK and being able to access files from our server back in Seattle has been especially useful. It's a really great product.

Sequestered In Memphis

For those who doubt the redemptive power of rock'n'roll, I invite you to check out the new tune by The Hold Steady. It's called Sequestered In Memphis, the first single from their upcoming album, Stay Positive. You can listen to it over at their myspace page. It's got everything, great riffs, horns, Springsteenesque piano, great lyrics and a shout-out-loud chorus. Ben Nichols from Lucero on backing vocals too. Makes you want to turn it up to 11 and jump around. I just can't wait to see this live.

Sequestered In Memphis

It started when we were dancing
It got heavy when we got to the bathroom
We didn't go back to her place
We went to some place where she cat sits
She said "I know I look tired,
But everything's fried here in Memphis"

Now they wanna know exactly which bathroom
Dude it doesn't make any difference, it can't be important
Yeah sure tell my story again

In barlight, she looked alright
In daylight, she looked desperate.
That's alright I was desperate too,
I'm gettin' pretty sick of this interview.
Subpoenaed in Texas
Sequestered in Memphis

I think she drove a new mustang
I guess it might be a rental
And I remember she had satellite radio
I guess she seemed a bit nervous
Do you think I'm that stupid
Well what the hell, tell the story again

In barlight, she looked alright
In daylight, she looked desperate.
That's alright I was desperate too,
I'm gettin' pretty sick of this interview.
Subpoenaed in Texas
Sequestered in Memphis

I went there on business

The album is out on July 14 and they're touring in the UK and the US - details on their myspace page.

Stay Positive!

Bruce ServicePack and the Vista Street Band

I don't know whether to laugh or cry...

[Thanks Robert]

Bruuuce!

Expanding interoperability

In case you missed it, Microsoft made a major announcement today regarding some significant changes being made to foster interoperability. A key piece of this is the publication of some 30000 pages of API and protocol documentation; here's an extract from the press release:

To enhance connections with third-party products, Microsoft will publish on its Web site documentation for all application programming interfaces (APIs) and communications protocols in its high-volume products that are used by other Microsoft products. Developers do not need to take a license or pay a royalty or other fee to access this information. Open access to this documentation will ensure that third-party developers can connect to Microsoft’s high-volume products just as Microsoft’s other products do.

There has been a massive amount of effort invested in this documentation work. I've had the pleasure of working on some of these documents myself, specifically some of the message queuing ones, and so it's great to see them out in the open.

U23D

You’ve got to see this. It’s a U2 concert movie filmed over several shows on their recent tour of Latin America…and it’s filmed in 3D! We saw it on the huge screen at the IMAX in Seattle and it is just jaw dropping. One minute you’re on the stage, next you’re in the mosh pit, next you’re flying over the stadium. You’re all sat there with your oversized glasses on, looking a little like Bono himself, grinning from ear-to-ear as Edge plays his guitar solo right in your face or Adam’s bass lines thump your chest. It’s just an amazing experience.

Windows Home Server

Like many geeks I have a server at home that we use as a central place to store music, photos and documents. I used to use an old Fedora Core linux box to do the job, based on a vintage dual processor PIII. This trusty old box had served me well for several years and accompanied me on the trip across the Atlantic from Newcastle to Seattle. Well, a few weeks back it finally gave up the ghost, leaving me to look for a replacement.

My initial thought was to buy a few new components and resurrect the box, again as a linux-based server. Before I did though, I thought I’d check out the newly released Windows Home Server, you know, being a company man and all ;-)

For those that haven’t heard of it, Windows Home Server (WHS) is pretty different from the other flavours of Windows in that, rather than being a general-purpose platform, it’s squarely focused on being an appliance for the home. It’s intended to run as a headless thing that you can stuff in the closet, or, if you prefer, in the wardrobe.

So what does it do? Well, first it acts as a central file store. You can create a bunch of shares and make them accessible over the network by your client PCs. Not much new there I hear you say. True, not on the surface of it, but if you look under the covers you’ll see a few smarts in how the server is storing the data. Instead of using RAID-based storage for data redundancy, WHS allows a bunch of different sized discs work to work together as a single volume. If you mark a particular share as requiring duplication, WHS will make sure that each of the files are stored on multiple physical disks. If you need more storage just go ahead and add some more drives and WHS will add it to the pool. If you have a disc failure, pull it out, stick in a replacement and WHS will do the necessary re-shuffling to restore the duplication. Very cool.

Another very useful feature is the integrated support for automated backups of the other computers in the house. By installing some ‘connector’ software on each client box, WHS will automate their backup using a nice incremental backup scheme. Using the server console you can configure the backup schedule that you want. The machines then wake up in the middle of the night and back themselves up to the server. You can open any of the backups and restore individual files or you can restore a complete machine using a special recovery disk. As the guy from The Fast Show’s Jazz Club would say, nice!

But wait, there’s more. WHS also provides a remote access feature that allows you to get at all your files or remote desktop into any of your home machines through a remote desktop protocol tunneled through HTTP.

One of the nice things about WHS is that it’s based on Windows Server 2003 so, if you know what you’re doing, you can get under the covers of the very simple-to-use interface and get to the real server underneath, allowing you to install other software on the box. I installed my streaming music server which works great; I’ll talk about this in another post.

Anyway, I’m sold, WHS is a really great product. If you’re looking to centralize your family’s photos and music or implement a backup strategy for all your precious data, I’d recommend that you take a look at WHS. You can buy pre-built, typically headless, machines from the likes of HP and smaller specialist providers, or you can buy the software and install it yourself. I’m running mine on a relatvely beefy system (2.33GHz E6650 Core2Duo) but others have reported respectable performance on some really old boxes. You know you’ve been looking for something to do with that old PC you’ve got lying around. You can register for an evaluation at the Windows Home Server website.

The Hold Steady are the saviours of rock and roll

If you read The Guardian you’re probably well aware that they love The Hold Steady. A couple of weeks ago they proclaimed their Glastonbury appearance to be “The gig of the festival.” Earlier they gave rave reviews to their records, Separation Sunday and Boys and Girls in America.

For me, The Hold Steady is just the most exciting band around. A month or so ago, Helen and I went down to Portland to see them play at the Crystal Ballroom. They played the Monday of the Memorial Day long weekend. We’d spent the weekend over in Hood River hiking and relaxing and when it came to the evening of the show I’d almost forgotten we were going to see a band. They weren’t playing til late and we sat in the downstairs bar having a few drinks til around 10 or so. We made our way upstairs and jostled for position, still pretty relaxed. When they finally came on stage and started playing, I was overcome by this complete sense of energy, excitement and joy. I was leaping around like an idiot (along with many other idiots!). The show was everything I expected and more; I was 18 again, feeling how I felt when I first heard bootlegs of Bruce’s live shows from the 70s.

Craig Finn, singer with The Hold Steady, understands this feeling too - check him out at the Springsteen tribute show from back in April.

In The Guardian’s latest piece on the band (from where the title to this post comes), Michael Hann shares his experiences:

But in the end, I guess, I love the Hold Steady because they make me feel like I did when I was 18. I don't mean girlfriendless, uncertain about the future, and prone to adolescent moodiness. I mean excited about the possibilities that music can offer, that someone you've never met can sing to your soul.

Don’t miss them if you get the chance.

The Hold Steady

From this week's edition of Seattle's The Stranger.

Couldn't agree more ;-)

Black Rebel Motorcycle Club at The Showbox

Dirty, that's the best way to describe the guitar sound opening the new Black Rebel Motorcycle Club record, Baby 81. It reminds me of the John Squire riff that opens The Stone Roses' Love Spreads. It's a sound that we haven't heard from BRMC for a while. Their last album, Howl, had a much more low key, alt-country feel than their first two records. I loved Howl, it was my album of the year for 2005, I bought copies for friends and everything, but I missed those dirty guitars and awesome rolling, rumbling bass lines. Well they're back and were in full effect at their Sunday night show at The Showbox in Seattle.

They played most of the new record and the new songs came over strong, better than on the record in many cases, especially "666 Conducer" which I thought was just brilliant live. They played just a few songs off the first two records including "Whatever Happened to my Rock and Roll (Punk Song)" which was as great as ever. Throughout the set they switched instruments a fair bit with Robert playing bass, guitar and piano and Peter playing guitar, bass and trombone - very cool.

After the end of the main set, they reappeared for an encore without Nick. I think they said he was suffering from allergies or something. It was definitely unplanned. Anyway, whatever the cause, the result was great. First Robert played "Mercy" (in the dark) and then Peter came out and played "Fault Line" from Howl. Then we had a great version of "Promise" before an all acoustic version of "Ain't No Easy Way" with Robert playing guitar standing behind the drum kit, pounding the bass drum with his foot - very cool. Then, a real surprise, Robert played Dylan's "Lonesome Death of Hattie Carol" - an excellent version too.

The impromptu acoustic encore reinforced the point that while BRMC make the most beautiful dirty garage rock, they still sound great when stripped down to the bone. Go and see them if you get the chance.

OneNote Live Sharing

I've been using OneNote 2007 quite a bit since I joined Microsoft. It's a really nice tool for taking and organising notes. I particularly like the way it integrates with Outlook so that I can right-click on an Outlook meeting and automatically create a new note with all the meeting info (participants etc.) right there. Anyway, it has a bunch of cool features but I discovered a new one yesterday that blew me away, it's called OneNote Live Sharing. Using this wizzo feature it's possible to share a note with a group of others over the network. You can either use this in broadcast mode where the group see the changes the owner is making, or you can enable it so that the group can collaborate on updating the content - it's very cool. A copy of the note gets also saved in each user's notebook. I think it synchronises offline updates too but I haven't tried that yet.

I was in a Scrum planning meeting yesterday and we used this Live Sharing feature to collaborate together to edit the plans - very agile! It's kinda like a giant whiteboard that everyone can update. If you have OneNote, give it a whirl.