TurnKey Linux Virtual Appliance Library

pyproject-pub: A simple Python project template

I hate repeating myself. It's boring. Life is too short. Like any self respecting hacker I will go out of my way to avoid it, even when I suspect it would cost me more to automate something away than to just do it by hand.

On the other hand, doing stuff I've done before by hand is no fun, while writing scripts is fun. Even when it does take longer, time is relative, or so Einstein said.

Getting started with Python and Lisp

A few weeks ago I talked with a friend studying computer science who I discovered had never experienced the joy of programming with a high level language. Not only that but he didn't have the first clue what he was missing. I feared without my immediate intervention another perfectly good mind would be wasted in programming hell. At his university they were using Java for nearly everything so he had somehow gotten the terribly mistaken idea that it didn't really matter what programming language one used. I carefully explained that:

Parallelize - a simple yet powerful high-level interface to multiprocessing

When I was developing Cloudtask, I discovered none of the interfaces in the Python multiprocessing module were powerful enough for my needs so I had to roll my own. The result is the generically useful multiprocessing_utils module in turnkey-pylib which from my totally subjective perspective provides a far superior interface to parallelization than the built-in multiprocessing interfaces.

Python optimization principles and methodology

Methodology

The basic methodology for optimization:

  1. Discover where you program is spending its time (hotspots vs coolspots)

    A good way to get an overview is to use the Python profiler. The Python profile will usually be included in Python's standard library:

4 simple software optimization tips

1) Always be experimenting!

Trying to squeeze out more performance out of your program? Don't be afraid to experiment!

In practice what that means is you setup small, simple throwaway experiments to establish how things work when you're not absolutely sure you fully understand something such (e.g., how many times a second a certain function can be invoked, how the profiler measures blocking IO or the time it takes a sub-program to complete).

Pythonic attribute magic (property, customized attribute access)

Many languages encourage programmers to use a getter/setter pattern.

Like this:

StdTrap: a magical Pythonic mechanism for intercepting console output

As a programmer I believe less is more. Good code is small, simple and elegant and many times favorable to larger, noisier code that does the same. It's not just about aesthetics either. Making code small and beautiful makes it easier to read, and easier to understand. Which is guaranteed to make it work better. Trust me on this.

Python iterators considered harmful

I just tracked down a nasty bug in my code to a gotcha with Python iterators.

Consider the following code...

Python property gotcha

If you like using a single getter/setter function for your properties, watch out if using None for the default. If you do that you won't be able to set your property to None!

Example code and workaround...

My code refactoring algorithm

You're looking at a block of Python code. It's not immediately obvious what it does. It's sort of a mess and you realize it needs to be refactored. But how? What mental algorithm do you use?