Blog Tags: 

Background task processing and deferred execution in Django

Or, Celery + RabbitMQ = Django awesomeness!

As you know, Django is synchronous, or blocking. This means each request will not be returned until all processing (e.g., of a view) is complete. It's the expected behavior and usually required in web applications, but there are times when you need tasks to run in the background (immediately, deferred, or periodically) without blocking.

Some common use cases:

  • Give the impression of a really snappy web application by finishing a request as soon as possible, even though a task is running in the background, then update the page incrementally using AJAX.
  • Executing tasks asynchronously and using retries to make sure they are completed successfully.
  • Scheduling periodic tasks.
  • Parallel execution (to some degree).
Blog Tags: 

Good automation vs bad automation

I recently eliminated a bit of code that was supposed to handle upgrading our build infrastructure from using one distribution (e.g., Ubuntu 8.04 LTS) to another (e.g., Ubuntu 10.04 LTS). That got me thinking about how to decide (and then explain) when it's a good idea to automate and when it isn't.

Blog Tags: 

Django Signals: Be lazy, let stuff happen magically

When I first learned about Django signals, it gave me the same warm fuzzy feeling I got when I started using RSS. Define what I'm interested in, sit back, relax, and let the information come to me.

You can do the same in Django, but instead of getting news type notifications, you define what stuff should happen when other stuff happens, and best of all, the so-called stuff is global throughout your project, not just within applications (supporting decoupled apps).

For example:

  • Before a blog comment is published, check it for spam.
  • After a user logs in successfully, update his twitter status.

What you can do with signals are plentiful, and up to your imagination, so lets get into it.

We don't need no stinking SSL

Why we disabled SSL and use an SSH tunnel for web site administration

Content managements systems like the one we're using for the web site (Drupal) need to provide a privileged administration interface which you usually want to access securely. Due to the insecure nature of the Internet, it's reasonable to assume your traffic may be intercepted at some point. So how do you prevent that?

Up until recently, we used SSL. You could access the web site from both:

Unfortunately, as the site grew in complexity this created a range of subtle but annoying paper-cut type problems.

Converting a virtual disk image: VDI or VMDK to an ISO you can distribute

Why would anyone in their right mind want to convert a VM into an ISO?

Good question, the answer for Conor Fox (who was the inspiration for this post - thanks Conor!) was to distribute his customized TurnKey PostgreSQL image so others could use it.

Distributing an ISO as opposed to a VM image allows it to be installed on any virtualization platform, as well as on bare metal, with the added bonus of running live.

I suppose that's a good enough reason, so lets get to it.

Converting a virtual disk image: VDI to VMDK to a raw loopback file you can mount

By default, VirtualBox creates virtual disk images in a special format called VDI, which is unique to VirtualBox. Disk images are stored in $HOME/.VirtualBox/HardDisks.

You'll need to convert VDI into another format if you want to run a VirtualBox VM on another virtualization platform, such as VMWare or KVM.

Blog Tags: 

Friends don't let friends program in shell script

Lately I've been going over a hellish patch-work of old shell scripts we wrote to automate some internal processes and I realized something: friends shouldn't let friends program in shell script.

Why?