Python3 Multithreading

Multithreading is similar to executing multiple different programs simultaneously. The advantages of multithreading are as follows:

• Threads can be used to move time-consuming tasks in a program to the background for processing.
• The user interface can be more attractive. For example, when a user clicks a button to trigger the processing of certain events, a progress bar can be displayed to show the progress of the processing.
• The running speed of the program may be accelerated.
• Threads are particularly useful in implementing tasks that involve waiting, such as user input, file read/write operations, and network data sending/receiving. In such cases, we can release some precious resources like memory usage.

Each independent thread has an entry point for program execution, a sequential execution sequence, and an exit point for the program. However, threads cannot execute independently; they must exist within an application, which provides control for multiple threads to execute.

Each thread has its own set of CPU registers, called the thread's context, which reflects the state of the CPU registers when the thread was last run.

The instruction pointer and stack pointer registers are the two most important registers in the thread context. Threads always run in the context of a process, and these addresses are used to identify memory within the address space of the process that owns the thread.

• Threads can be preempted (interrupted).
• While other threads are running, a thread can be temporarily suspended (also called sleeping) — this is thread yielding.

Threads can be categorized as:

• Kernel Threads: Created and destroyed by the operating system kernel.
• User Threads: Threads implemented in user programs without kernel support.

The two commonly used modules for Python3 threading are:

• _thread
• threading (recommended)

The thread module has been deprecated. Users can use the threading module instead. Therefore, in Python3, the "thread" module can no longer be used. For compatibility, Python3 renamed thread to "_thread".

Getting Started with Python Threads

There are two ways to use threads in Python: using a function or wrapping a thread object in a class.

Functional approach: Call the start_new_thread() function in the _thread module to create a new thread. The syntax is as follows:

_thread.start_new_thread(function, args[, kwargs])

Parameter description:

• function - The thread function.
• args - Arguments passed to the thread function; it must be a tuple.
• kwargs - Optional arguments.

Example

#!/usr/bin/python3

import _thread
import time

# Define a function for the thread
def print_time(threadName, delay):
    count = 0
    while count < 5:
        time.sleep(delay)
        count += 1
        print("%s: %s" % (threadName, time.ctime(time.time())))

# Create two threads
try:
    _thread.start_new_thread(print_time, ("Thread-1", 2,))
    _thread.start_new_thread(print_time, ("Thread-2", 4,))
except:
    print("Error: unable to start thread")

while 1:
    pass

Executing the above program produces the following output:

Thread-1: Wed Jan 5 17:38:08 2022
Thread-2: Wed Jan 5 17:38:10 2022
Thread-1: Wed Jan 5 17:38:10 2022
Thread-1: Wed Jan 5 17:38:12 2022
Thread-2: Wed Jan 5 17:38:14 2022
Thread-1: Wed Jan 5 17:38:14 2022
Thread-1: Wed Jan 5 17:38:16 2022
Thread-2: Wed Jan 5 17:38:18 2022
Thread-2: Wed Jan 5 17:38:22 2022
Thread-2: Wed Jan 5 17:38:26 2022

After executing the above program, you can press ctrl-c to exit.

---

Thread Module

Python3 provides support for threads through two standard libraries: _thread and threading.

_thread provides low-level, primitive threads and a simple lock. Compared to the threading module, its functionality is quite limited.

In addition to containing all the methods in the _thread module, the threading module also provides the following methods:

• threading.current_thread(): Returns the current thread variable.
• threading.enumerate(): Returns a list of currently running threads. "Running" means the thread has started but has not yet terminated, excluding threads before startup and after termination.
• threading.active_count(): Returns the number of running threads, which is the same result as len(threading.enumerate()).
• threading.Thread(target, args=(), kwargs={}, daemon=None):
  • Creates an instance of the Thread class.
  • target: The target function the thread will execute.
  • args: Arguments for the target function, passed as a tuple.
  • kwargs: Keyword arguments for the target function, passed as a dictionary.
  • daemon: Specifies whether the thread is a daemon thread.

The threading.Thread class provides the following methods and attributes:

1. __init__(self, group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None):
   • Initializes the Thread object.
   • group: Thread group, currently unused, reserved for future expansion.
   • target: The target function the thread will execute.
   • name: The name of the thread.
   • args: Arguments for the target function, passed as a tuple.
   • kwargs: Keyword arguments for the target function, passed as a dictionary.
   • daemon: Specifies whether the thread is a daemon thread.
2. start(self):
   • Starts the thread. This will call the thread's run() method.
3. run(self):
   • The code to be executed by the thread is defined within this method.
4. join(self, timeout=None):
   • Waits for the thread to terminate. By default, join() will block until the calling thread terminates. If the timeout parameter is specified, it will wait for at most timeout seconds.
5. is_alive(self):
   • Returns whether the thread is running. If the thread has started and has not yet terminated, it returns True; otherwise, it returns False.
6. getName(self):
   • Returns the name of the thread.
7. setName(self, name):
   • Sets the name of the thread.
8. ident attribute:
   • The unique identifier of the thread.
9. daemon attribute:
   • The daemon flag of the thread, indicating whether it is a daemon thread.
10. isDaemon() method:
    • Checks if the thread is a daemon thread.

A simple thread example:

Example

import threading
import time

def print_numbers():
    for i in range(5):
        time.sleep(1)
        print(i)

# Create thread
thread = threading.Thread(target=print_numbers)

# Start thread
thread.start()

# Wait for thread to finish
thread.join()

The output is:

0
1
2
3
4

---

Creating Threads Using the threading Module

We can create a new subclass by directly inheriting from threading.Thread, instantiate it, and then call the start() method to start a new thread, which in turn calls the thread's run() method:

Example

#!/usr/bin/python3

import threading
import time

exitFlag = 0

class myThread(threading.Thread):
    def __init__(self, threadID, name, delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay

    def run(self):
        print("Starting thread: " + self.name)
        print_time(self.name, self.delay, 5)
        print("Exiting thread: " + self.name)

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)
        print("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new threads
thread1.start()
thread2.start()

thread1.join()
thread2.join()

print("Exiting main thread")

The execution result of the above program is as follows:

Starting thread: Thread-1
Starting thread: Thread-2
Thread-1: Wed Jan 5 17:34:54 2022
Thread-2: Wed Jan 5 17:34:55 2022
Thread-1: Wed Jan 5 17:34:55 2022
Thread-1: Wed Jan 5 17:34:56 2022
Thread-2: Wed Jan 5 17:34:57 2022
Thread-1: Wed Jan 5 17:34:57 2022
Thread-1: Wed Jan 5 17:34:58 2022
Exiting thread: Thread-1
Thread-2: Wed Jan 5 17:34:59 2022
Thread-2: Wed Jan 5 17:35:01 2022
Thread-2: Wed Jan 5 17:35:03 2022
Exiting thread: Thread-2
Exiting main thread

---

Thread Synchronization

If multiple threads modify the same data, unpredictable results may occur. To ensure data correctness, synchronization of multiple threads is required.

Using the Lock and Rlock objects of the Thread class can achieve simple thread synchronization. Both objects have acquire and release methods. For data that only allows one thread to operate at a time, its operations can be placed between the acquire and release methods. As follows:

The advantage of multithreading is that multiple tasks can be run simultaneously (or at least it feels that way). However, when threads need to share data, data inconsistency issues may arise.

Consider this scenario: a list where all elements are 0. The "set" thread changes all elements from back to front to 1, while the "print" thread is responsible for reading and printing the list from front to back.

Then, it's possible that when the "set" thread starts modifying, the "print" thread comes to print the list, resulting in an output that is half 0 and half 1. This is data inconsistency. To avoid this situation, the concept of a lock is introduced.

A lock has two states — locked and unlocked. Whenever a thread, like "set", wants to access shared data, it must first acquire the lock. If another thread, like "print", has already acquired the lock, then the "set" thread is paused, which is synchronous blocking. After the "print" thread finishes accessing and releases the lock, the "set" thread is allowed to continue.

After this processing, when printing the list, it will either output all 0s or all 1s, avoiding the awkward situation of half 0 and half 1.

Example

#!/usr/bin/python3

import threading
import time

class myThread(threading.Thread):
    def __init__(self, threadID, name, delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay

    def run(self):
        print("Starting thread: " + self.name)
        # Acquire lock for thread synchronization
        threadLock.acquire()
        print_time(self.name, self.delay, 3)
        # Release lock to allow next thread
        threadLock.release()

def print_time(threadName, delay, counter):
    while counter:
        time.sleep(delay)
        print("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

threadLock = threading.Lock()
threads = []

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new threads
thread1.start()
thread2.start()

# Add threads to thread list
threads.append(thread1)
threads.append(thread2)

# Wait for all threads to complete
for t in threads:
    t.join()

print("Exiting main thread")

Executing the above program produces the following output:

Starting thread: Thread-1
Starting thread: Thread-2
Thread-1: Wed Jan 5 17:36:50 2022
Thread-1: Wed Jan 5 17:36:51 2022
Thread-1: Wed Jan 5 17:36:52 2022
Thread-2: Wed Jan 5 17:36:54 2022
Thread-2: Wed Jan 5 17:36:56 2022
Thread-2: Wed Jan 5 17:36:58 2022
Exiting main thread

---

Thread Priority Queue (Queue)

Python's Queue module provides synchronized, thread-safe queue classes, including FIFO (First-In-First-Out) queue Queue, LIFO (Last-In-First-Out) queue LifoQueue, and priority queue PriorityQueue.

These queues all implement lock primitives and can be used directly in multithreading. Queues can be used to achieve synchronization between threads.

Common methods in the Queue module:

• Queue.qsize() returns the size of the queue.
• Queue.empty() returns True if the queue is empty, otherwise False.
• Queue.full() returns True if the queue is full, otherwise False.
• Queue.full corresponds to the maxsize size.
• Queue.get([block[, timeout]]) gets an item from the queue, with timeout being the waiting time.
• Queue.get_nowait() is equivalent to Queue.get(False).
• Queue.put(item) puts an item into the queue, with timeout being the waiting time.
• Queue.put_nowait(item) is equivalent to Queue.put(item, False).
• Queue.task_done() After completing a task, the Queue.task_done() function sends a signal to the queue that the task has been completed.
• Queue.join() Actually means to wait until the queue is empty before performing other operations.

Example

#!/usr/bin/python3

import queue
import threading
import time

exitFlag = 0

class myThread(threading.Thread):
    def __init__(self, threadID, name, q):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q

    def run(self):
        print("Starting thread: " + self.name)
        process_data(self.name, self.q)
        print("Exiting thread: " + self.name)

def process_data(threadName, q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            data = q.get()
            queueLock.release()
            print("%s processing %s" % (threadName, data))
        else:
            queueLock.release()
        time.sleep(1)

threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]

queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads = []
threadID = 1

# Create new threads
for tName in threadList:
    thread = myThread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1

# Fill the queue
queueLock.acquire()
for word in nameList:
    workQueue.put(word)
queueLock.release()

# Wait for queue to empty
while not workQueue.empty():
    pass

# Notify threads it's time to exit
exitFlag = 1

# Wait for all threads to complete
for t in threads:
    t.join()

print("Exiting main thread")

The execution result of the above program is:

Starting thread: Thread-1
Starting thread: Thread-2
Starting thread: Thread-3
Thread-3 processing One
Thread-1 processing Two
Thread-2 processing Three
Thread-3 processing Four
Thread-1 processing Five
Exiting thread: Thread-3
Exiting thread: Thread-2
Exiting thread: Thread-1
Exiting main thread