Why python is single threaded and GIL(Global Interpreter Lock)

Gaurav Yadav

3 years ago

GIL is a mutex lock in python which allows only one thread to execute at a time. This is why python is a single-threaded application. So if your program is single threaded python will perform as equal to any other language. But when it comes to multithreading and executing threads in parallel, it is not possible in python

Why GIL?

Python uses reference counting for memory management. What it does is keeps track of all the variables that are created and whether they are being used in the program now or not. If there is no reference for those variables. The python garbage collector kicks in and removes that variable from the heap.

Now for this reference counting to work properly it needs to be safe from race conditions. That is two threads try to change the reference count. If that happens that can lead to a memory leak. This reference count can be kept safe by locking all the data structures that are shared across the threads.

Now if we lock these data structures. It means multiple locks will exist which can lead to problems like a deadlock. Also, it will decrease performance as acquisition and release of locks will happen quite frequently.

GIL is a single lock that says that execution of any python byte code requires the interpreter lock. This prevents the above problem but makes the python single-threaded.

DevOps Interview Questions: Important Python questions.

Why only GIL as a solution?

Python has been around since the days when operating systems did not have a concept of threads. Python was designed to be easy to use in order to make development quicker and more and more developers started using it.

A lot of extensions were being written for the existing C libraries whose features were needed in Python. To prevent inconsistent changes, these C extensions required thread-safe memory management which the GIL provided.

The GIL is simple to implement and was easily added to Python. It provides a performance increase to single-threaded programs as only one lock needs to be managed.

C libraries that were not thread-safe became easier to integrate. And these C extensions became one of the reasons why Python was readily adopted by different communities.

Impacts?

There are two kinds of multithreaded programs.

I/O bound program

Programs that wait for input/output from the user, file, database, network, etc. These sometimes have to wait a lot.

CPU bound program

These programs are the one which does high computational tasks.

We will see how python works in both these cases.

In the case of CPU-bound programs, the multithreading will not make any difference as only one thread will execute. In fact, you can see performance degrading for these cases.

In the case of the IO-bound process, the multithreading will increase performance as the IO-bound processes mostly wait for the input or output to be provided to them and they are in a waiting state. Acquiring and releasing locks for this kind of process doesn’t cause problems to cause when one thread may get the output or input the others may still be waiting for the event.

Workaround?

If you want to make use of all your core in python programs instead of going for multithreading go for multiprocessing. Each process gets its own python interpreter this the GIL is applied on only that process and hence you can do a lot of work in parallel.