TurnKey Linux Virtual Appliance Library

Getting started with Python and Lisp

A few weeks ago I talked with a friend studying computer science who I discovered had never experienced the joy of programming with a high level language. Not only that but he didn't have the first clue what he was missing. I feared without my immediate intervention another perfectly good mind would be wasted in programming hell. At his university they were using Java for nearly everything so he had somehow gotten the terribly mistaken idea that it didn't really matter what programming language one used. I carefully explained that:

Parallelize - a simple yet powerful high-level interface to multiprocessing

When I was developing Cloudtask, I discovered none of the interfaces in the Python multiprocessing module were powerful enough for my needs so I had to roll my own. The result is the generically useful multiprocessing_utils module in turnkey-pylib which from my totally subjective perspective provides a far superior interface to parallelization than the built-in multiprocessing interfaces.

Python optimization principles and methodology

Methodology

The basic methodology for optimization:

  1. Discover where you program is spending its time (hotspots vs coolspots)

    A good way to get an overview is to use the Python profiler. The Python profile will usually be included in Python's standard library:

4 simple software optimization tips

1) Always be experimenting!

Trying to squeeze out more performance out of your program? Don't be afraid to experiment!

In practice what that means is you setup small, simple throwaway experiments to establish how things work when you're not absolutely sure you fully understand something such (e.g., how many times a second a certain function can be invoked, how the profiler measures blocking IO or the time it takes a sub-program to complete).

Pythonic attribute magic (property, customized attribute access)

Many languages encourage programmers to use a getter/setter pattern.

Like this:

StdTrap: a magical Pythonic mechanism for intercepting console output

As a programmer I believe less is more. Good code is small, simple and elegant and many times favorable to larger, noisier code that does the same. It's not just about aesthetics either. Making code small and beautiful makes it easier to read, and easier to understand. Which is guaranteed to make it work better. Trust me on this.

Python iterators considered harmful

I just tracked down a nasty bug in my code to a gotcha with Python iterators.

Consider the following code...

Python property gotcha

If you like using a single getter/setter function for your properties, watch out if using None for the default. If you do that you won't be able to set your property to None!

Example code and workaround...

My code refactoring algorithm

You're looking at a block of Python code. It's not immediately obvious what it does. It's sort of a mess and you realize it needs to be refactored. But how? What mental algorithm do you use?

Neat trick: invoking a Python debugger at an arbitrary point in your program

Do you find yourself occasionally wishing you could freeze a misbehaving program at an arbitrary point in time and then examining what was going on interactively?

That's exactly what the debugger is for, but sometimes it's just too much of a bother to run your program inside it, you have to set breakpoints, etc.

Well there's a really simple alternative: call the debugger from an arbitrary point in your program, like this...