TurnKey Linux Virtual Appliance Library

Optimizing Django: tricks for faster page loads

By reducing the file size of your CSS, JavaScript and images, as well as the number of unnecessary browser requests made to your site, load time of your applications pages can be drastically reduced, not to mention the load on your server.

Yahoo have created a list of the 35 best practices to speed up your website, a recommended read for any web developer. I wanted to summarize a few I recently implemented in a Django application.

In a nutshell:

  • Serve compressed static media files. This one is obvious, the smaller the file size, the quicker it is to download.
  • Once a browser has already downloaded the static media, it shouldn't have to download it again. This is what actually happens, as the server will respond with 304 Not Modified, which means that the cached file is still up-to-date.
  • However, the HTTP request is still sent. This is unnecessary and should be avoided by setting the HTTP Expire header to a date in the future.
  • When files are updated, you need to give them a new URL so the browser will request them. This is referred to as versioning. A common scheme is to use the modification time/date of the file in its URL, either as part of the name or in the query string. This is obviously tedious and error prone, so automation is required.
     

Django-Compress (CSS/JS compression and auto-versioning)

django-compress provides an automated system for compressing CSS and JavaScript files, with built in support for auto-versioning. I was pleasantly surprised by the ease of integration, and was up and running within a couple of minutes.

Installation

Firstly, django-compress depends on csstidy, so lets install it.

apt-get update
apt-get install csstidy

Now, grab the latest tarball from github.

wget http://github.com/pelme/django-compress/tarball/master

Unpack and add it to your site (or the Python path).

tar -zxf pelme-django-compress-*.tar.gz
cp -a pelme-django-compress-*/compress /path/to/django/apps/compress

Update settings.py to enabled django-compress, auto-updates, and versioning.

# settings.py

COMPRESS = True
COMPRESS_AUTO = True
COMPRESS_VERSION = True
...

INSTALLED_APPS = (
    ...
    'compress',
)

Configuration

Configure which CSS and JavaScript files to compress and auto-version.

# settings.py

COMPRESS_CSS = {
    'screen': {
        'source_filenames': ('css/style.css', 'css/foo.css', 'css/bar.css'),
        'output_filename': 'compress/screen.?.css',
        'extra_context': {
            'media': 'screen,projection',
        },
    },
}

COMPRESS_JS = {
    'all': {
        'source_filenames': ('js/jquery-1.2.3.js', 'js/jquery-preload.js')
        'output_filename': 'compress/all.?.js',
    },
}

Note: The '?' will be substituted with the epoch time (I.e., the version).
 

I hate hardcoding, it's just too error prone and not scalable, so I used this primitive helper function to auto-generate the required configuration.

# utils.py

import os

# generate config for django-compression
# alpha-numeric ordering for customization as required
def compress_cfg(media_root, dir, output):
    files = os.listdir(os.path.join(media_root, dir))
    files.sort()
    return {'source_filenames': tuple(os.path.join(dir, f) for f in files),
            'output_filename': output}

Using the above helper function, the hardcoded files can be replaced with the following for auto-generation (after MEDIA_ROOT is defined).

# settings.py

from utils import compress_cfg
...

COMPRESS_CSS = {'screen': compress_cfg(MEDIA_ROOT, 'css', 'compress/screen.?.css')}
COMPRESS_JS  = {'all': compress_cfg(MEDIA_ROOT, 'js', 'compress/all.?.js')}

Usage

Now you can update your templates to use the compressed auto-versioned files, for example:


{% load compressed %}
<html>
    <head>
        {% compressed_css 'screen' %}
        {% compressed_js 'all' %}
    </head>
    ...


Image optimization and versioning

With regards to image optimization, take a look at Liraz's post on PNG vs JPG: 6 simple lessons you can learn from our mistakes.

It would be great if django-compress supported image versioning, but it currently doesn't.

I found this nice snippet which provides a template tag for asset versioning, such as images, but it only solves half the problem, the other half being images specified in CSS.

If you need to go the image versioning route, a more complete solution is django-static, which also does CSS/JS compression, though I prefer django-compress.

Currently, I have not implemented image versioning. Its just not worth the complexity as I don't have too many images, and have no plans to change them, often, hopefully. The Expires Header is good enough for now. (Pre-mature optimization is the root of all evil).
 

HTTP Expire header

As discussed above, without an Expires header, the browser will request media files on every page load, and will receive a 304 Not Modified response if the cached media files are up-to-date.

Setting a Far Future Expire is possible, and recommended, now that you have versioned CSS and JavaScript files.

In the following examples I have set images to expire 2 hours after they are accessed, but you can tweak this to your specific use case. CSS and JavaScript (basically) never expire.

Depending on your media server, add your custom configuration, enable the expire(s) module, and reload the webserver.

Apache2

ExpiresActive On

ExpiresByType image/jpg "access plus 2 hours"
ExpiresByType image/png "access plus 2 hours"
...

ExpiresByType text/css "access plus 10 years"
ExpiresByType application/x-javascript "access plus 10 years"

Lighttpd

server.modules = (
    "mod_expire",
    ...

$HTTP["url"] =~ "\.(jpg|png|gif|ico)$" {
    expire.url = ("" => "access plus 2 hours")
}

$HTTP["url"] =~ "\.(css|js)$" {
    expire.url = ("" => "access plus 10 years")
}

 

Lastly, don't forget to enable compression (gzip) on your media server for an extra load-time performance gain.

You can get future posts delivered by email or good old-fashioned RSS.
TurnKey also has a presence on Google+, Twitter and Facebook.

Comments

Image versioning

If you want simple image versioning I recommend using relativnes of CSS. I do it by versioning path to static media and making rewrite to remove version from path: For example:

 

/static/23445/css/style.css -> /static/css/style.css

and in css use only relatives URIs like:

background: url(../images/bg.png);

With that images will be taken from versioned path. I often use revision number for version, but it can be whatever you want, just change it before deploying.

See my post for some discussion and alternatives: http://hauru.eu/2009/10/18/idea-about-expires-header/

Watch out for Vary: * header

Thanks for the tips. django-compress looks handy. Just to clarify, it is writing its output where the webserver serves it directly, right? (As opposed to Django serving it, which would be slower.) It sounds like the case from the docs, but I wanted to be sure.

Also, I will throw out a warning. I recently set up an Apache server to send the Expires: header for cached content, and ran into a problem I hadn't seen anywhere else I'd done that. In that Apache config there was an old setting for mod_deflate that included the response header "Vary: *" which disabled browser caching because any request header could invalidate the cache.

Getting rid of that Vary: header allowed the Expires: header do its job right.

The excellent http://www.webpagetest.org/ makes it easy to spot this and other problems in IE 7 and IE 8, browsers which many of us otherwise wouldn't pay enough attention to. :)

Alon Swartz's picture

Yes, you're correct

Yes, you're correct. django-compress outputs to the filesystem, and updates the template tags accordingly. The web server will then serve the files, not Django.

Thanks for the Vary header tip, I'll look out for it.

I would strongly recommend django-static instead

I genuinely recommend that for anybody who want's the absolutely most optimization instead install django-static and read  The awesomest way possible to serve your static stuff in Django with Nginx

It will be much simpler to configure and you get a more optimal compression of your CSS and your Javascript with the absolutely ideal HTTP headers. As the author of this package I might be biased but there is no doubt it will give you the best efficiency you can get with today's tools. 

Alon Swartz's picture

The thing I really like...

The thing I really like about django-static is that its a more encompassing solution than django-compress, with built-in support for images (as well as those in CSS which is really useful).

In the future when/if our requirements change I will surely take a closer look at using django-static. For now, django-compress is doing its job.

A couple of questions though:

  • What do you mean by "absolutely ideal HTTP headers"?
  • Do you have benchmarks on django-static vs. django-compress compression?

Agree

I agree with Peter here. Django-static is a much better choice and thanks for the links (good info).

Awesome Peter

I'm looking at django-static now, it's looking like I can use this to optimize serving images from my application. The app contains lots and lots of images and I intend to keep it that way.

Yet another alternative to django-compress

For awhile I used django-compress, but as you stated I hated hardcoding my static files in my settings.py.  I ended up switching to django-compressor (http://github.com/carljm/django_compressor) which takes all files listed between a block and compresses them into one file stripping unnecessary white space.  So you would use:

{% load compress %}

{% compress css %}

<link rel="stylesheet" href="/static/css/file1.css" media="screen" type="text/css" />

<link rel="stylesheet" href="/static/css/file2.css" media="screen" type="text/css" />

{% endcompress %}

How is this optimizing Django

The first rule of Django is that you serve static content directly from the webserver, Django should never touch it. There are already better minification tools out there (packer). So, again in what way is this Django based, these rules apply to every web site under every possible framework.

Let Apache do it for you!

Instead of doing this through Django (presumably through Python, and perhaps some C), use the goodness of mod_pagespeed (C/C++) from Google to do all your compression/minify/etc. My Django apps load as much as twice as fast, just by installing it and configuring. 

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)