TurnKey Linux Virtual Appliance Library

Python performance tests reaffirms "premature optimization is the root of all evil"

"Premature optimization is the root of all evil"

Today while programming I found myself thinking about the performance impact of various basic Python operations such as method invocation, the "in" operator, etc.

This is a mistake because you really shouldn't be thinking about this stuff while programming. The performance impact of doing things one way vs another way is usually so small that you couldn't measure it.

Besides, often your program is IO bound rather than CPU bound which means even if you magically optimized CPU time to 0 you probably wouldn't notice it that much anyway.

The correct way to go about optimization is to ignore it altogether and just focus on optimizing for readability. Optimizing for performance is the last thing you should do. Literally. When you're done programming and you discover you really do need to squeeze out more performance out of your program you can run it through a profiler, identify the hotspots where your program is spending most of its time and then go optimize those for better IO or CPU performance.

But... I was still curious about Python's performance so I ran some tests. Just to put my mind at ease. And because it's pretty neat to measure things that are literally happening millions of times a second.

Some fun facts (my computer, one thread):

  • In general, high level Pythonic code is faster than equivalent lower level Python code.

  • Python's null-op, "pass" can run 50M times a second

  • function calls can happen 6.5M times a second

  • instances can be created 3.5M times a second

  • If making comparisons, set is faster than array when you have over 5 elements:

    # execution time increases linearly with size of array
    # runs at 40M comparisons a second
    # if len(array) == 1000000 this will run 40 times a second
    val in array
    
    # execution time constant
    # runs 8M times a second
    val in set
    
  • set(iterable) is 4X slower than list(iterable) but still really fast

    In 1 second == create a set with 130,000 elements. In 1 second == create a list with 446,000 elements.

Bottom line: thinking about writing well optimized Python code is usually a waste of time and energy.

Appendix

Here's a little code snippets I use to time stuff:

import os
import time

def bench(f, howmany):
    start = time.time()
    for i in xrange(howmany):
        f()
    end = time.time()

    elapsed = end - start

    print "%d runs in %.4f seconds (%.2f per/sec)" % (howmany, elapsed, howmany / elapsed)

The timeit module from the standard library can can come in useful as well:

import timeit

# by default timeit runs a million times
timeit.Timer(stmt="pass").timeit()
timeit.Timer(stmt="pass").timeit()

# how many times a second
1 / timeit.Timer("pass").timeit(1000) * 1000

You can get future posts delivered by email or good old-fashioned RSS.
TurnKey also has a presence on Google+, Twitter and Facebook.

Comments

Somewhat disagree

I agree that premature optimization is bad, but at the same time you should know which common Python constructs are fast or slow so that *by default* your lower level code tends to be fast. Often idiomatic Python tends to be fastest (or fast enough), because Python programmers like writing things certain way, it appears in lots of cases, and has been optimized in the language implementation. The point here is that once you have learned what is fast and you write that way by default, you don't need to think about it and waste time thinking about it (or waste time in the future profiling it wondering if it might be slow).

Liraz Siri's picture

Readability never gets old

I think it's better to focus on readability rather than adapt your style to squeeze out more performance our of a specific implementation of Python. Readability never gets old, while the underlying implementation details may change in a way that breaks your assumptions on what is fast and what isn't.

If you need to optimize at some point

My experience is that first I never code with optimisation in mind.

Readability and Code evolution is more important.

After if I need perf when arrays are involved I switch some parts to numpy or in last ressort cython.

Using cython, you will quickly understood that trying to optimize python high level code is loosing your time ;)

Also you can use python hotshot that is really good analyzing where you loose time!

Easier ways to squeeze out better performance

I would like to start with a disclaimer that it all depends on the circumstances but what I have been generally observing about extracting better performance is that, tweaking the code (for better perf.) is fast becoming obsolete.  Nowadays, we could do it in much simpler ways like jazzing up the hardware - like adding more CPUs and RAM.  So, I largely concur with article.

The only area where I feel that tweaking the code is unavoidable, is while dealing with parallel programming.

Nitpicking about set

Just some little nitpicking: I don't think you can say anything about set()'s performance compared to list() because it depends in large part to how fast the hash function is.

I haven't tried, but my guess is that if you try to create a set of very very long strings, it will be much slower (but for lists, it will take the same time... normally)

Liraz Siri's picture

True, the circumstances always matter

It's not unusual for rules of thumb to break down when you venture too far from the well beaten path. Circumstances matter.

Great article!

Great article on the perils of pre-mature optimizations!

It's always a bad thing. Write your application first.

Optimize it later.

 

--JamesMills (prologic)

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)