Python: Increasing performance of your code: For vs Map

Gaurav Yadav

8 years ago

In this performance of programming language article we are going to talk about some small tips which will help you in Increasing performance of your python code.

Lookup for function is costly.

Lets say we have to write a loop which will run till the length and calls upper on each string.

alist = ['a','b','c']
for item in alist:
    len(item)

This can be written in faster way as below:

alist = ['a','b','c']
fn = len
for item in alist:
    fn(item)

Here we are not asking for function lookup every time instead keep the function in local scope and used it. Thus it increased our performance

Here is what I wrote for performance testing and what it returned.

import datetime
alist = [str(x)for x in range(10000000)]
a = datetime.datetime.now()
for item in alist:
    len(item)
b = datetime.datetime.now()
print (b-a).total_seconds()
a = datetime.datetime.now()
fn = len
for item in alist:
    fn(item)
b = datetime.datetime.now()
print (b-a).total_seconds()

What we get from here is as below.

Thus there is a difference of almost 1 sec. Thus proved.

Use map where possible.

Now we can further decrease the time by using map function. Since map function is direct implementation in C code. It runs really fast.

Here how it is done

map(len, alist)

Now the time taken for the three functions are like this

Again there is decrease in time taken.

Can we further decrease the time, lets try approach 2 and three together i.e. use local functions.

fn = len

map(len, alist)

Yes this must run fastest and it does. Here are the results.

If you want to learn data structures in python you can read the below books.

So we saw what all is happening here.

Here is the final code if you want to test it yourself.

import datetime
alist = [str(x)for x in range(10000000)]
a = datetime.datetime.now()
for item in alist:
    len(item)
b = datetime.datetime.now()
print (b-a).total_seconds()
a = datetime.datetime.now()
fn = len
for item in alist:
    fn(item)
b = datetime.datetime.now()
print (b-a).total_seconds()
a = datetime.datetime.now()
map(len, alist)
b = datetime.datetime.now()
print (b-a).total_seconds()

a = datetime.datetime.now()
fn = len
map(fn, alist)
b = datetime.datetime.now()
print (b-a).total_seconds()

Update:

Follow the below code for python 3

import datetime 
alist = [str(x) for x in range(10000000)]: 
print("\nStandard loop.") 
a = datetime.datetime.now() 
result = [] 
for item in alist: 
    result.append(len(item)) 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 
print("\nStandard loop with function name in local namespace.") 
a = datetime.datetime.now() 
result = [] 
fn = len 
for item in alist:
    result.append(fn(item))
b = datetime.datetime.now()
print((b-a).total_seconds())
print("\nUsing map.")
a = datetime.datetime.now()
result = list(map(len, alist))
b = datetime.datetime.now()
print((b-a).total_seconds())
print("\nUsing map with function name in local namespace.")
a = datetime.datetime.now() 
fn = len 
result = list(map(fn, alist)) 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 
print("\nList comprehension.") 
a = datetime.datetime.now() 
result = [len(i) for i in alist] 
b = datetime.datetime.now() 
print((b-a).total_seconds()) 
print("\nList comprehension with name in local namespace.") 
a = datetime.datetime.now() 
fn = len 
result = [fn(i) for i in alist] 
b = datetime.datetime.now() 
print((b-a).total_seconds())

Result for the following is as below:

Standard loop. 

1.728273 



Standard loop with function name in local namespace. 

1.706101 



Using map. 

0.539602 



Using map with function name in local namespace. 

0.536706



List comprehension. 

0.668053



List comprehension with name in local namespace. 

0.609126