Python optimization principles and methodology


The basic methodology for optimization:

  1. Discover where you program is spending its time (hotspots vs coolspots)

    A good way to get an overview is to use the Python profiler. The Python profile will usually be included in Python's standard library:

4 simple software optimization tips

1) Always be experimenting!

Trying to squeeze out more performance out of your program? Don't be afraid to experiment!

In practice what that means is you setup small, simple throwaway experiments to establish how things work when you're not absolutely sure you fully understand something such (e.g., how many times a second a certain function can be invoked, how the profiler measures blocking IO or the time it takes a sub-program to complete).

Transcend the Drupal documentation, use the source Luke!

During the first few months of my Drupal experience I looked for answers to any issues that came up first in the official documentation, then on Google. It's a big Drupal world out there so more often then not I would find someone had come across exactly the same issue before and I could just parrot the solution without necessarily understanding why it worked.

Blog Tags: 

Pythonic attribute magic (property, customized attribute access)

Many languages encourage programmers to use a getter/setter pattern.

Like this:

Tips for the Object Oriented Programming novice

The following is written for programmers who don't really understand object oriented programming yet. They probably understand the language semantics, but don't really understand how to use them correctly.

If you find yourself misusing object oriented semantics that probably means you don't have the skills to develop good software. This is a problem because bad software is much harder to develop and even harder to maintain.

Unix buffering delays output to stdout, ruins your day

Let's say you have the following program:

import time
while True:
    print 'hello world'

chmod +x ./

If you run this program from a terminal, it will print hello world every second.

But redirect the output to a file and something different happens:

./ > output &
tail -f output

You won't see any output! (At least not for a long while)

The same is true if you redirect's through a unix pipe which you can do on the shell:

Two simple ways a script can detect if it's online

I didn't want installation of one of TurnKey's components to freeze up too long if it was installed offline so I looked for a nice way to detect that without bringing in additional dependencies.

Two methods stood out...

  1. ping Google DNS servers: one second constant delay, regardless if you're online or off:

Blog Tags: 

configuring libresolv to timeout DNS queries to unreachable nameservers more quickly when offline

When I go offline I pull my network cable. This causes DNS queries to take forever to resolve. Actually 40 seconds but it feels like forever.

Blog Tags: 

StdTrap: a magical Pythonic mechanism for intercepting console output

As a programmer I believe less is more. Good code is small, simple and elegant and many times favorable to larger, noisier code that does the same. It's not just about aesthetics either. Making code small and beautiful makes it easier to read, and easier to understand. Which is guaranteed to make it work better. Trust me on this.

How Foodsoft inspired me to rewrite the TKLDev documentation

A couple of weeks ago I was corresponding with @wvengen, a free software developer from the Netherlands who has been using TKLDev to package Foodsoft into a ready-to-use Foodsoft in a box solution that can be easily deployed to manage your own non-profit Food Coop.


Subscribe to development (Blog)