Why to use process instead of threads in python?

Yup you may be thinking why so? These are not my words but comes from the experts of python. They use to say this because of a component called as GIL or global interpreter lock. Let’s see Why to use process instead of threads in python?

Why to use process instead of threads in python?

Since python is an interpreted language it has no idea of the program before it runs. Since we know that working with threads may cause problems like race conditions, dead locks can exist. You want your program to be thread safe. Thread safe means that every time you run your program the output is consistent weather one thread is running it or multiple threads are running it. To ensure this in parallelism the GIL or Global  Interpreter Lock ensures that this does not happen. Due to this GIL in python only one thread can execute at any time.

Then why processes are safe?

We are aware that thread share a common memory space thus making it vulnerable for unintended access and modification of memory. While in process the memory space are different thus it is ensured that in no other way the shared memory is going to be changed by any other program.

Lets discuss more..

You can implement multi threading in python but then you will realise than your multi threaded program is running slower than it was running in single thread. Its because of the fact that you have to save the state of the program at every atomic level if you want your program to be thread safe and since it is running still on single thread thus adding to the complexity of the program. Thus making your program take more time. 

Its not that this problem is not attempted to be solved. It is tried by many in the past and the patches are rejected due to one or the other reason. One of such was in 1999 by Greg Stein, the multi threaded solution was good but when the same program was running in single thread it was taking 40% more time than it was supposed to take in normal case. So the patch was rejected.

There were major changes in GIL in python 3.2 but I haven’t gone through them for now. I will post a blog once I read them and understand what changes were done.

Till then use process in python if you want multi threading so badly go with java or go for now.

Read more about python articles

Python: Increasing performance of your code: For vs Map