Python Multi Threading – Networking Lab

I am by no means a Python expert and when researching how to get my own multi threading working I see there are several ways depending on the task that you are performing.
To summarise what I had read over and over, multi processing, spawns multiple processes for your CPU and is best for CPU bound tasks. Multi threading is for I/O bound tasks where there is a lot of waiting.

There seems to be more than these two basic statements to it, I understand that multi processing can have a thread pool. However I’m only dealing with something basic for a lab here.

What I needed was two specific points;

The first was to have only have a set amount of threads running at once. I’ll get onto why later.
The second was to have the values returned when each thread was executed.

First point, a set amount of threads. I needed this as the base multi threading I have used before has spawned as many threads as devices and then opened up SSH connections to them. While this is ok in a lab environment. If I have 100 devices, then I’m spawning 100 threads. I want to have a little more control.

Second point, in my TACACS script I’m getting output from the devices which is written to a file and used later in the script. Not being able to return a value means I could only login to the device quicker than the standard linear method. But no use to me.

This first multi threads script I am painting walls, two at a time. The output is the thread number (always 0 and 1), and the wall painted. Nothing is returned. Once the print statement telling me which wall is being painted has been output, that’s it.
This takes care of my first requirement


import queue
import threading
import time

# https://medium.datadriveninvestor.com/the-most-simple-explanation-of-threads-and-queues-in-python-cbc206025dd1

def paintwall(q, thread_no):
    while True:
        task = q.get()
        time.sleep(2)
        print(f'Thread #{thread_no} is painting wall #{task} in the queue.')
        q.task_done()
        

q = queue.Queue()
num_walls = ["front", "side1", "side2", "back", "outside", "big", "small"]
finished_walls = []

for i in range(2):
    worker = threading.Thread(target=paintwall, args=(q, i,), daemon=True)
    worker.start()


for wall in num_walls:
    q.put(wall)

q.join()

import queue

import threading

import time

# https://medium.datadriveninvestor.com/the-most-simple-explanation-of-threads-and-queues-in-python-cbc206025dd1

def paintwall(q, thread_no):

while True:

task = q.get()

time.sleep(2)

print(f'Thread #{thread_no} is painting wall #{task} in the queue.')

q.task_done()

q = queue.Queue()

num_walls = ["front", "side1", "side2", "back", "outside", "big", "small"]

finished_walls = []

for i in range(2):

worker = threading.Thread(target=paintwall, args=(q, i,), daemon=True)

worker.start()

for wall in num_walls:

q.put(wall)

q.join()

The next version of this is to have an output in a list so we can pick up later. For this I am passing in a list called return_list.
This takes care of the second point.


import queue
from threading import Thread
import time


def paintwall(q, thread_no, return_list):
    while True:
        task = q.get()
        print(f'Thread #{thread_no} is painting wall {task} in the queue.')
        output = (f'Wall {task} has been painted.')
        time.sleep(0.2)
        return_list.append(output)
        q.task_done()


q = queue.Queue()
return_list = []
num_walls = ["front", "side1", "side2", "back", "outside", "big", "small"]

for i in range(2):
    worker = Thread(target=paintwall, args=(q, i, return_list,), daemon=True)
    worker.start()

for wall in num_walls:
    q.put(wall)


q.join()

print("\n")
for returned in return_list:
    print(f"This is output: {returned}")

import queue

from threading import Thread

import time

def paintwall(q, thread_no, return_list):

while True:

task = q.get()

print(f'Thread #{thread_no} is painting wall {task} in the queue.')

output = (f'Wall {task} has been painted.')

time.sleep(0.2)

return_list.append(output)

q.task_done()

q = queue.Queue()

return_list = []

num_walls = ["front", "side1", "side2", "back", "outside", "big", "small"]

for i in range(2):

worker = Thread(target=paintwall, args=(q, i, return_list,), daemon=True)

worker.start()

for wall in num_walls:

q.put(wall)

q.join()

print("\n")

for returned in return_list:

print(f"This is output: {returned}")

Further Examples


import threading
import time


class CPUPainter:
   
    def __init__(self, i):
        # self.paintwall()
        t = threading.Thread(target=self.paintwall, args=(i,))
        t.start()

    def paintwall(self, i):
        time.sleep(2)
        finished = i
        return finished


for i in range(10):
    returned = CPUPainter(i)
    # print(returned)

import threading

import time

class CPUPainter:

def __init__(self, i):

# self.paintwall()

t = threading.Thread(target=self.paintwall, args=(i,))

t.start()

def paintwall(self, i):

time.sleep(2)

finished = i

return finished

for i in range(10):

returned = CPUPainter(i)

# print(returned)

Further Examples

Leave a Comment Cancel Reply