Email's problems

Relative to the lifespan of most technology, email is ancient. Invented over 40 years ago, email predates the internet and the World Wide Web, instant messaging, homepages, search engines, forums, blogs, Wikipedia, eBay, Craigslist, and YouTube. Despite its age, email hasn't evolved that much since the 1960s. Electronic mail is based on the paradigm of postal mail: a system of passing messages back and forth between senders and recipients. But this pattern doesn't scale well to in-depth conversations among groups of people. Here's why.

Email propagates multiple, disconnected copies and versions of messages. You type an email message, address it, and send it. A copy of that message stays in your sent email box, and appears in your recipient's inbox. Your recipient replies and optionally includes a copy of your original message in her response. A copy stays in her sent box, and appears in your inbox. You reply to her reply, and cc: another recipient and send the next message. In three email interactions, seven copies of the same message appear in differing states for three people. It's a mess.

There's no easy way to exchange rich content like maps, videos, or images via email. Ever receive an email message from a family member with an enormous image attachment that takes forever to download? Or emailed a link to a web page that broke or became unclickable? Email attachments and URLs are not a good way to share content like photos or maps or videos with others easily. Furthermore, email software represents messages differently — some display HTML and images, others just plain text. No one's email always looks the same.

There's no easy way to reply to a subsection of an email. Jack sends Jill an email telling her all about his latest project, then asking when she'll be in town, and where she's staying when she gets there. An email message is just a flat document, so it's not easy for Jill to reply to ONLY the questions Jack asked. She could reply to his message and manually copy and paste just his questions and position her answers directly after them, but that's a lot of work that most people don't do. Often questions and individual points that need addressing via email get lost because there's no easy way to reply to a specific section of a message.

There's no easy way to privately respond to specific people within a group email. When Jack and Jill do finally meet up, Jane invites them over for dinner via email. Jack wants to reply only to Jill and ask what bottle of wine they should bring to dinner. Reply to all doesn't work, because Jane will see it, so he has to manually edit the recipient list on the private email and create yet another copy of the invitation.

While Google Wave isn't a full-on replacement for email (yet), understanding email's problems given the capabilities of the modern web is a good framework for understanding what Google Wave can do.

Wave's solution: Conversations as live documents

Rather than pass back and forth multiple copies of messages, Google Wave hosts a single copy of a conversation (a lowercase "wave") that everyone involved edits. Wave displays the latest version of that conversation to everyone in the group in real-time, even as it's changing. That means if Jack has the wave he sent Jill open on his computer, and Jill is typing her responses across the country on her computer, Jack sees the wave change keystroke by keystroke.

Google Wave treats an email conversation with several recipients and senders as a document with several editors and writers. If you can make the conversations-as-documents and documents-as-conversations leap along with Wave, the system makes 100% more sense.

In other smaller ways, Google Wave addresses the rest of the problems with email. Using Google Wave, all the participants in a conversation have the ability to reply to a specific question or statement inside a wave inline, add rich media like videos, images, maps, and polls in-wave, reply privately inline to a subset of participants, and play back earlier versions of a wave sequentially, so that you can revert to an older state of a given wave, or see how it changed over time.

This short video demonstrates how Wave offers a better alternative to email in an everyday office interaction.

Wave's pitfall: It's confusing

Right now, Google Wave's biggest pitfall is its complexity. Parody web site EasierToUnderstandThanWave.com jokes that heady topics like radiocarbon dating, neoclassical economics, and polymodal chromaticism are easier to understand than Wave. The joke is funny because the initial Wave confusion is a universal experience. The first waves you're bound to receive from your friends and co-workers, fresh on Wave, will say things like "I don't get it" and "This is weird."

There are a few good reasons for the initial confusion:

Conversation-as-document is a whole new paradigm with no existing precedent. For most computer users, editing a Microsoft Word document and instant messaging are two very different activities. Google Wave fundamentally conflates messaging and document-editing, so there's no obvious existing parallel for what you do in Wave to what you do now. It's not quite email, and it's not quite writing a Word document. Google Wave is both and neither, and that feels totally foreign, and makes Wave difficult to explain.

Conversation trees, or non-linear message threads, are chaotic. Forums, blog comments, email threads, instant messaging sessions are all linear conversations, where the most new message appears at the bottom (or top) of the list. You read them sequentially, in one direction, one after the other. Google Wave's inline reply capability turns a conversation into a tree that can grow any number of branches. When wave participants add new information to a wave on different branches at different times, the non-linear nature of a busy wave can feel overwhelming and unnatural.

Document versioning is foreign (to non-programmers). Software developers have been using file versioning tools like the one built into Google Wave for decades now. But most computer users don't version their files or use a feature like Wave's playback in any other context, so its utility isn't immediately obvious.

Wave isn't done yet, so it has gaping holes of missing functionality. Basic functionality that you'd expect from a messaging and document-editing platform is currently missing in the Wave preview, which is currently pre-beta software. The lacking features makes Wave seem less useful than doing those things "the old way."

Google released the Wave preview, accessible by invitation only, to start getting feedback from users and developers before the product is fully-baked. If you're interested in giving Wave a spin, ask a friend who's already in to give you an invite (each user gets invites to hand out to their associates), or put your name in the hat here.

What do you think of Google Wave? Let us know in the comments.

So here's the deal with Wave:  If you deal in technology, and you get this one wrong, you'll miss the boat.  And it's a big boat.  If, on the other hand, you get this one right, you have the potential to do some incredible innovation.

In a nutshell, this is the next revolutionary leap in Internet application architecture.  Maybe the first truly revolutionary leap since HTTP itself.

I've been wanting to write this post for a while, but first I wanted to read fully thru and digest the specs and available code.  I haven't done any posts about XMPP for quite a while, but you're going to start hearing a whole lot about it, and not just from me.

What is it?

At its core, Wave is an extension to the XMPP protocol.  This is the REALLY important part.  Here I'll back up for a moment for a little background on XMPP.  

XMPP is a protocol which describes communication.  It models communication between two nodes on a network.

I hear people mistakenly talking about Wave as immature or new technology.  It's not.  XMPP has been around since 1998, being developed and actively worked on for almost 12 years now.  It's been approved by the IETF since 2004.  

Although it's been mostly used for chat, that's only the tip of the iceberg when you dig into this protocol.  I'm still pretty flabbergasted that this protocol hasn't been used more than it has, and I'm excited to see somebody finally tapping into its potential.

The Server

The server (a "wave provider") is a modified Openfire XMPP server that understands the Wave protocol extensions.  Openfire has been around for a while too.

While wave providers are used for storing and server XMPP content in Google's implementation, there is a lot of potential in turning existing applications into wave providers as well.  Any existing server-based content can be used as the basis for a wave, so just about any application out there right now has the potential to extend its existing functionality by offering its contents as waves.

The Client

Ok, this is probably what you've seen videos of.  It's a wave client because it speaks wave protocol to wave providers.  

What Google has done is develop the first really full-featured XMPP client, which also uses some of their new XMPP extensions to facilitate things like character-by-character updates.  They've developed an incredibly sexy client, and I'm glad about it, because a sexy client like is what's required to sell an innovation this large to both the mass market AND the technical community.

What Google has done, exactly?

So what exactly has Google done to the XMPP protocol?

A couple of things:

• It recognizes a conversation as data, and stores the data in a persistent way that can be easily referenced over time.  Conversation storage and persistence is a huge gaping hole in enterprises right now, which you'll nod your head in agreement with if you've ever scoured a wiki for information or cleaned out your inbox because it got too big.
• It makes XMPP conversations secure and scalable.  By building in synchronization protocols, conversations can take place distributed across the network instead of at specific central servers (which is how wikis, blogs, and microblogs operate).
• It describes a way to replicate content over a large network so that it's available on a wide scale while still being fast and synchronized.  Data replication on a global (but as-needed) scale, very cool. 
• It integrates a very simple and elegant security model which operates on an as-needed basis.  I've blogged about this in more detail before here. 

These were some needed additions to XMPP, and really describe some of the peer-to-peer operations that I've been looking for for a long time.

Why is this so interesting?

XMPP.  In case you haven't noticed, I'm a big fan.

XMPP is so versatile that if it becomes widely adopted it will be to the Internet what HTTP was:  a platform for new types of applications.  And where HTTP as a platform is a server-centric model, XMPP is capable of peer-to-peer communication.

Remember what happened when everyone got HTTP clients (they're called browsers :) ?  The Internet exploded.  Well, if everyone gets a full-fledged XMPP client I think you can expect roughly the same thing to happen.

One of the most fascinating features of XMPP is the way things are addressed.  EVERYTHING is addressable over the network. You can talk directly to ANYTHING, and ANYONE.  I can't stress how big of a shift that would be from the current model.  It's HUGE.  I wrote a whole series of posts on this a few years ago, and it's just as exciting now as it was then.

Let's take a step back and think about this for a second.  I'll probably do another post just on the addressing scheme at some point because it's so key, but it's worth a brief recap.

• Right now I cannot send text directly to your instant message account (unless you're using an XMPP-based client), I have to send the message to your IM server which relays the message to you.
• I cannot send audio directly to your phone, the phone company has to route it there. 
• I cannot share a picture directly with your Facebook account, I have to sent it to Facebook first to be carried on to you.  
• I can't send a file directly to you, I have to put it on a share or email it to you. 

(Not to mention the fact that these are all disconnected, you can't combine these into a single message stream.  XMPP addresses that problem very nicely, as the wave client shows.)

XMPP removes these intermediaries from the network.  Social networks and proprietary transports no longer have an exclusive license to deliver content, the clients talk directly to one another.

Do you see the difference?  There are no longer social networks or any other type of networks required to relay the communication, we are now down to exactly 3 components:

1. Clients 
2. Storage 
3. Applications 

Of course there is always the underlying dumb pipes that transport the data, but from a functional perspective the network has been normalized out of importance.

Clients can be whatever we need them to be.  It can be the Google wave client, it can be your phone, it can be a desktop app.  These will evolve over time, but the Google client is a fantastic starting point, certainly light years ahead of anything else that's available today.

Storage becomes a utility, something you pay for as you go.  I already use this model myself for backups, I shoot them up to the Amazon cloud and pay for the amount I use.  As time goes on my communication--audio, video, pictures, text--will be stored there as well, and I'll use it in my waves as needed.  (Note that waves do NOT embed this content, they link to it and the client downloads and renders it in place.)

And the applications.   This is really exciting, because just about every application in existence will be transformed by this quantum shift in the network topology.  Applications now interact with your client and provide input to your communication stream, and output to your storage.  They will become a facet of your communication, not a completely disconnected activity.  You will communicate with apps much in the same way that you communicate with people, and they will communicate with you.

Take CRM and ERP systems for example.  Instead of customers emailing you about a sale and then sending purchase orders, it will be part of the "sale wave".  The entire sale, from start to finish, will be encapsulated in a single wave, bringing individuals in and out of the conversation as need.  The ERP and CRM platforms themselves will be participants in this conversation, recognizing the purchase order, executing the workflow, processing the order, making the order details available to manufacturing or delivery in a sub-wave, and then making the receipt available to the customer and the sales team.  Your CRM Whether you approve the purchase order from your desktop, your phone, or a point of sale device, makes no difference--they can all be directly addressed and participate in the conversation natively.

That's just one example off the top of my head, but I truly believe that every software application in existence will eventually need to be re-architected to be much less application-centric and much more communication-centric.

The Bottom Line

This is no longer pie-in-the-sky stuff like it was when I was first writing about it.

I have seen nothing else out there that rivals the XMPP/Wave protocol for the sheer richness of the conversation that's possible, not to mention the fact that it can easily turn into a bona-fide platform for next-generation applications.

The danger, then, is that you ignore this and it takes off.  You and your application will be shut out of this rich, real-time collaborative stream of communication.  You can, of course, tack integration on later, but the real benefit here is to the application that incorporates these concepts into its core architecture.  I can look at just about any application out there and think of tons of potential applications for this technology.

This is the type of revolutionary advance that is required to lift productivity and open brand new possibilities to the extent necessary to revive the economy, which is pretty exciting.  If this post sounds like I'm breathlessly waiting for this technology to take off, it's because I am, and I have been for about 3 years now.  Here's hoping Google succeeds.