Using Comet to update web application data in real time?

Recently I've been looking into a powerful but complex web application technology we're considering using in future versions of the Hub.

The problem: how do we update a browser view when a background process updates the database?

A few examples of places in the Hub where this comes up...

  • User launches server, waits for the server to get an IP, for the status icon to change color, etc.
  • User associates a server with an elastic IP
  • User attaches an EBS or detaches an EBS
  • The instance cache expires after N minutes, a background process reloads the information, which is different from the previously displayed information.

You can use "plain old" AJAX polling every N seconds (and most sites with "real-time" data do), but there are a couple of limitations:

  1. Will never be real-time
  2. It can be tricky to scale

Wouldn't it be neat if instead of polling, any changes to database state could be PUSHED on the fly to the browser without any action on the part of the user?

Well it turns out you can do exactly that via a technique called "reverse AJAX" or "Comet" that uses long waiting requests from the client to the server to make sure there is a connection that's always on and ready to accept "pushed" updates from the server.

But since you don't want to dedicate a synchronous process to every waiting poll, it's best to leverage specially designed server software to help you with that.

Currently Comet is mostly being used for chat-like applications (e.g., GTalk, Meebo, Facebook chat) and seems to add considerable complexity (even more than regular AJAX) so I'm not sure it would be a good idea to use this for something as simple as updating a server view but that depends on implementation details so I wouldn't rule it out off the bat.

Comet is just an umbrella term for this type of technique. The actual protocol used in the implementation is typically STOMP, a messaging protocol supported by RabbitMQ (and many others).

Note that there are many excellent open source components that can be leveraged to implement Comet applications:

  • Orbited: an asynchronous COMET server in Python
  • js.io: client-side STOMP client

Further reading

Comments

Chris Musty's picture

Real Software (Not free by any means - www.realsoftware.com) is a fairly good IDE for makig cross platform applications and have recently released their web edition to make desktop like apps in a browser. Its not quite at a point where I would use it commercially (some bugs etc) but they do use the push methods explained above.

The reson I bring this up is that with my playing around with their web edition for about 3 months now, the loading mouse icon never disappears and causes some real annoyance with end users. It sounds trivial but they believe they are downloading viruses or their connection is not working...

Is this a problem with Comet too? 

Chris Musty

Director

Specialised Technologies

Liraz Siri's picture

Comet is an umbrella name for a family of techniques that involve having the web browser use Javascript to make requests (e.g., http://example.com/update) in the background that are intentionally not answered by the web server until the web server wants to "push" something to the client. In other words, it's a trick used to achieve server->client style communication over the regular client->server protocols. Traditional web servers choke when you have too many open connections, hence the need for special software and techniques. But it's not a specific piece of software.

I'm not sure regarding the mouse pointer issue. I can see how leaving requests hanging indefinitely might confuse some browsers into thinking the page is still "loading". OTOH, if I'm not mistaken it should be possible to use CSS3 to override the default mouse pointers.

Liraz Siri's picture

Thanks for the link Emanuel. This might be just what we were looking for.

Pages

Add new comment