Python Parallel Processing using MultiThreading !!!

 An Introduction to Python Threading:

Folks, you can go through the multiprocessing in Python in the below blog.

Python Multiprocessing !!!

A thread is an entity within a process that can be scheduled for execution. Also, it is the smallest unit of processing that can be performed in an OS (Operating System). In simple words, a thread is a sequence of such instructions within a program that can be executed independently of other code. For simplicity, you can assume that a thread is simply a subset of a process!

If you want your application to make better use of the computational resources of multi-core machines, you are advised to use multiprocessing or concurrent.futures.ProcessPoolExecutor. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.

                                            Dia. Multiple threads in a process.

Here, we will discuss and execute the parallel processing using the help of multi-threading package.

import threading, os, time

my_list = [1,2,3,4,5,6]

def my_func(num):
  print(f"Inside function for {num}:")
  print("Process ID :",os.getpid())
  print("sleeping for five seconds:\n")
  time.sleep(5)
  print(f"Completed task for {num}:\n")

thread_list = []
if __name__ =="__main__":
  for element in my_list:
    thread_list.append(threading.Thread(target = my_func, args = (element,)))
  for thrd in thread_list:
    thrd.start()
  
  for thrd in thread_list:
    thrd.join()
  
  print("Done")

Code Interpretation:

We imported the required multithreading package. The required function my_func() is defined with sleep and print statements. Also, we can print the process ID of each thread created by the processor.

In the second snippet, we have defined the list "thread_list" to accumulate the threads. In the loop, we created separate threads by "threading.Thread" method and appended each thread object to the list "thread_list".

After adding threads to the list, we trigger each thread parallelly by thread.start() method. To stop the execution of the current program until a thread is complete, we use the join() method.

Output of above code:

Interpretation of the output:

Once we triggered the threads in the loop, all threads started simultaneously. Each thread has the same parent process id as 2463. After a sleep of 5 seconds, threads completed their executions, and the program was completed.

Points to remember:

1. You can adjust the threads invocation as per the available cores keeping in mind the daemons of OS, and also the available memory space, as multiple threads will run in memory space of only once process. You can check available cores with os.cpu_count() function.