Python - Multithreading

Modern operating systems are driven by powerful processors. They are known to provide multitasking capabilities. Powerful processors are likely to remain idle for long intervals of time because of comparatively lower speeds of I/O operations. Operating systems today are designed to perform multiple tasks concurrently, so as to keep processor engaged. We often see a big document being printed in the background, while another program running simultaneously. In real sense, processor performs one task at a time, but it switches between tasks so fast that multiple tasks appear to be handled simultaneously.

Thread can be considered as flow of execution of a process. The term 'multithreading' refers to occurrence of more than one simultaneous paths of execution of instructions in a single process. As any program is executed, a primary thread is said to have started running. During the run, another line of parallel execution may be initiated. It is called secondary thread. Such a program in which contains more than one flows of control is called 'multithreaded program'.

Process based multitasking involves huge overheads in terms of resources. Multithreading on the other hand is comparatively lightweight as multiple threads in a single process share same memory space. Of course a careful maneuvering of threads needs to be done to avoid 'race condition' – a situation where more than one threads trying to update data at same location.

Thread life cycle

Primary or main thread starts as soon as the program is run, and it terminates as the program ends. A secondary thread may be initiated during the execution of primary thread. Upon creation, it starts 'running' a stipulated function. As the function is over, the secondary thread merges with the primary thread.

While a thread is in running state, it may be halted, either for a specified interval or till a condition occurs, or till some other thread completes its operation. Halted thread resumes either after a specified interval or conditionally.

Earlier versions of Python contained _thread module which offered a bit primitive threading mechanism. Newer 'threading' module provides higher-level threading interfaces on top of the lower level _thread module.

As mentioned earlier, main thread is always present representing the control flow of program. Additional thread is created by declaring object of Thread class which is defined in 'threading' module.

Thread(name,target)

Although 'name' parameter is none by default, a unique name to each thread is internally assigned. You can specify it exclusively if required.

Each new thread should invoke start() method, which will in turn call run() method. Instead of run() method, if any other function is to be called by start(), it should be set as target parameter.

Following methods are defined in Thread class:

start() :

This method starts the thread’s activity. It must be called once a thread object is created. It automatically invokes object’s run() method  in a separate thread. If called more than once, a RuntimeError will be raised.

run() :

This method represents the thread’s activity. It may be overriden in a subclass. Instead of standard run() method, the object invokes the function passed to the it's constructor as the target argument.

join() :

This method blocks the calling thread until the thread whose join() method is called terminates. The termination may be either normal, because of an unhandled exception – or until the optional timeout occurs. It can be called many times. The join() raises a RuntimeError if an attempt is made to join the current thread. Attempt to join() a thread before it has been started also raises the same exception.

name :

This is a property object for getter and setter methods of thread's name attribute.

getName() :

This returns name attribute of thread.

setName() :

This method assigns a string as name attribute of thread object.

is_alive() :

This method returns whether the thread is alive. It returns True just before calling run() method and until just after the run() method terminates.

Following program illustrates basic functionality of multithreading mechanism:

from threading import Thread

import time

class myThread(Thread):

       def __init__(self, name):
               Thread.__init__(self)
               self.name=name
               self.start()
       def run(self):
               for i in range(1,6):
                       time.sleep(5)
                       print (self.name, i)
t1=myThread('Thread1')
for i in range(1,6):
       print ('main', i)
       time.sleep(5)

Here, the main thread runs a for loop after creating a new thread. In each iteration there is a call to sleep() function allowing progress of main thread to pause, paving way for secondary thread to run.

The other thread is an object of a subclass of Thread class in threading module. On creating, the start() method is called upon, which in turn invokes run() method.

The run() method, overridden in myThread subclass also has a for loop which takes one iteration and pauses. By that time main thread's pause time is over. So the main and secondary threads take turns to complete one iteration of loop at a time. The output is as below:

main 1
Thread1 1
main 2
Thread1 2
main 3
Thread1 3
main 4
Thread1 4
main 5
Thread1 5

Thread synchronization

In a multithreaded program, there may be a situation more than one threads try to gain access to same resource, unpredictable result may arise.  For example, if multiple threads try to perform read/write operation on same file, it may cause data inconsistency because one of the threads may change its contents before earlier thread's file closing operation hasn't been finished. Hence such a concurrent handling of shared resources such as file or function need to be synchronized in such a way that only one thread can access the resource at a given point in time.

Lock object

A lock object provides a synchronization mechanism. It is not owned by a particular thread when locked. The Lock object is always in one of two states, “locked” or “unlocked”. When created itt is in the unlocked state. It has two basic methods, acquire() and release().  

acquire() :

When the state is unlocked, this method changes the state to locked and returns immediately. The method takes an optional blocking argument. If set to False, it means do not block. If a call with blocking set to True would block, return False immediately; otherwise, set the lock to locked and return True.

The return value of this method is True if the lock is acquired successfully, False if not.

release() :

When the state is locked, this method in another thread changes it to unlocked. The release() method should only be called in the locked state. If an attempt is made to release an unlocked lock, a RuntimeError will be raised.

When the lock is locked, reset it to unlocked, and return. If any other threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed. There is no return value of this method.

In following example, the locking mechanism is at work.

import threading, time
class myThread(threading.Thread):
       def __init__(self,name):
               threading.Thread.__init__(self)
               self.name=name
       def run(self):
               lock.acquire()
               syncmethod(self.name)
               lock.release()
def syncmethod(threadName):
       print (threadName,"has acquired lock and is running'syncmethod'")
       counter=10
       while counter:
               time.sleep(1)
               print ('..', end='')
               counter=counter-1
       print('\nlock released for', threadName)
lock=threading.Lock()
threads=[]
t1=myThread('Thread1')
t2=myThread('Thread2')
t1.start()
threads.append(t1)
t2.start()
threads.append(t2)
for t in threads:
       t.join()
print ("end of main thread")

The program initializes two thread objects. As one of them acquires control of the lock to invoke 'syncmethod', other is kept waiting for former to release the lock. Here is the output:

Thread1 has acquired lock and is running'syncmethod'
....................
lock released for Thread1
Thread2 has acquired lock and is running'syncmethod'
....................
lock released for Thread2
end of main thread

Semaphore Object

This synchronization is mechanism invented by Edsger W. Dijkstra. Python implements semaphore by an internal counter which is decremented by each acquire() call and incremented by each release() call. The counter can never go below zero; when acquire() finds that it is zero, it blocks, waiting until some other thread calls release().

The threading module also defines following static functions and constants:

active_count() :

It returns the number of Thread objects currently alive.

current_thread() :

It return the current Thread object, corresponding to the caller’s thread of control.

enumerate() :

It returns a list of all Thread objects currently alive including the main thread. It excludes terminated threads and threads that have not yet been started.

main_thread() :

it returns the main Thread object. In normal conditions, the main thread is the thread from which the Python interpreter was started.

settrace(func) :

Set a trace function for all threads started from the threading module. The func will be passed to sys.settrace() for each thread, before its run() method is called.

setprofile(func) :

Set a profile function for all threads started from the threading module. The func will be passed to sys.setprofile() for each thread, before its run() method is called.

stack_size([size]) :

It returns the thread stack size used when creating new threads.

This module also defines the following constant:

TIMEOUT_MAX :

The maximum value allowed for the timeout parameter of blocking functions like Lock.acquire(),