Python3 Multithreading

Python3.x Python3 Multithreading

Multithreading is similar to executing multiple different programs simultaneously. The advantages of multithreading are as follows:

• Using threads can move time-consuming tasks in a program to the background for processing.
• The user interface can be more attractive. For example, when a user clicks a button to trigger the processing of certain events, a progress bar can be popped up to display the progress of the processing.
• The running speed of the program may be accelerated.
• Threads are useful in implementing some waiting tasks, such as user input, file read/write, and network data sending/receiving. In this case, we can release some precious resources such as memory usage, etc.

Each independent thread has an entry point for program execution, a sequential execution sequence, and an exit point for the program. However, threads cannot execute independently; they must exist within an application, and the application provides control for multiple threads to execute.

Each thread has its own set of CPU registers, called the thread's context. This context reflects the state of the CPU registers when the thread last ran.

The instruction pointer and stack pointer registers are the two most important registers in the thread context. Threads always run in the context of the process, and these addresses are used to mark memory in the address space of the process that owns the thread.

• Threads can be preempted (interrupted).
• While other threads are running, a thread can be temporarily suspended (also called sleeping) -- this is thread yielding.

Threads can be divided into:

• Kernel threads: Created and destroyed by the operating system kernel.
• User threads: Threads implemented in user programs without kernel support.

The two commonly used modules in Python3 threading are:

• _thread
• threading (recommended)

The thread module has been deprecated. Users can use the threading module instead. Therefore, the "thread" module can no longer be used in Python3. For compatibility, Python3 renamed thread to "_thread".

Getting Started with Python Threads

There are two ways to use threads in Python: using a function or using a class to wrap the thread object.

Functional style: Call the start_new_thread() function in the _thread module to create a new thread. The syntax is as follows:

_thread.start_new_thread ( function, args[, kwargs] )

Parameter description:

• function - The thread function.
• args - Arguments passed to the thread function, must be a tuple type.
• kwargs - Optional arguments.

Example

#!/usr/bin/python3

import _thread
import time

# Define a function for the thread
def print_time( threadName, delay):
    count = 0
    while count < 5:
        time.sleep(delay)
        count += 1
        print ("%s: %s" % ( threadName, time.ctime(time.time())))

# Create two threads
try:
    _thread.start_new_thread( print_time, ("Thread-1", 2, ) )
    _thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
    print ("Error: unable to start thread")

while 1:
    pass

Executing the above program produces the following output:

Thread-1: Wed Jan 5 17:38:08 2022
Thread-2: Wed Jan 5 17:38:10 2022
Thread-1: Wed Jan 5 17:38:10 2022
Thread-1: Wed Jan 5 17:38:12 2022
Thread-2: Wed Jan 5 17:38:14 2022
Thread-1: Wed Jan 5 17:38:14 2022
Thread-1: Wed Jan 5 17:38:16 2022
Thread-2: Wed Jan 5 17:38:18 2022
Thread-2: Wed Jan 5 17:38:22 2022
Thread-2: Wed Jan 5 17:38:26 2022

After executing the above program, you can press ctrl-c to exit.

---

Thread Module

Python3 provides support for threads through two standard libraries: _thread and threading.

_thread provides low-level, primitive threads and a simple lock. Its functionality is relatively limited compared to the threading module.

In addition to containing all the methods in the _thread module, the threading module also provides other methods:

• threading.current_thread(): Returns the current thread variable.
• threading.enumerate(): Returns a list of running threads. Running means after the thread starts and before it ends, not including threads before starting and after terminating.
• threading.active_count(): Returns the number of running threads, which has the same result as len(threading.enumerate()).
• threading.Thread(target, args=(), kwargs={}, daemon=None):
  • Creates an instance of the Thread class.
  • target: The target function that the thread will execute.
  • args: Arguments for the target function, passed as a tuple.
  • kwargs: Keyword arguments for the target function, passed as a dictionary.
  • daemon: Specifies whether the thread is a daemon thread.

The threading.Thread class provides the following methods and attributes:

1. __init__(self, group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None):
   • Initializes the Thread object.
   • group: Thread group, not used currently, reserved for future expansion.
   • target: The target function that the thread will execute.
   • name: The name of the thread.
   • args: Arguments for the target function, passed as a tuple.
   • kwargs: Keyword arguments for the target function, passed as a dictionary.
   • daemon: Specifies whether the thread is a daemon thread.
2. start(self):
   • Starts the thread. It will call the thread's run() method.
3. run(self):
   • The thread defines the code to be executed in this method.
4. join(self, timeout=None):
   • Waits for the thread to terminate. By default, join() will block until the calling thread terminates. If the timeout parameter is specified, it will wait for at most timeout seconds.
5. is_alive(self):
   • Returns whether the thread is running. If the thread has started and has not terminated, it returns True, otherwise it returns False.
6. getName(self):
   • Returns the name of the thread.
7. setName(self, name):
   • Sets the name of the thread.
8. ident attribute:
   • The unique identifier of the thread.
9. daemon attribute:
   • The daemon flag of the thread, used to indicate whether it is a daemon thread.
10. isDaemon() method:
    • Returns whether the thread is a daemon thread.

A simple thread example:

Example

import threading
import time

def print_numbers():
    for i in range(5):
        time.sleep(1)
        print(i)

# Create thread
thread = threading.Thread(target=print_numbers)

# Start thread
thread.start()

# Wait for thread to finish
thread.join()

The output is:

0
1
2
3
4

---

Creating Threads Using the threading Module

We can create a new subclass by directly inheriting from threading.Thread, instantiate it, and call the start() method to start a new thread, which calls the thread's run() method:

Example

#!/usr/bin/python3

import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay

    def run(self):
        print ("Starting thread：" + self.name)
        print_time(self.name, self.delay, 5)
        print ("Exiting thread：" + self.name)

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)
        print ("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new threads
thread1.start()
thread2.start()

thread1.join()
thread2.join()

print ("Exiting main thread")

The execution result of the above program is as follows:

Starting thread：Thread-1
Starting thread：Thread-2
Thread-1: Wed Jan 5 17:34:54 2022
Thread-2: Wed Jan 5 17:34:55 2022
Thread-1: Wed Jan 5 17:34:55 2022
Thread-1: Wed Jan 5 17:34:56 2022
Thread-2: Wed Jan 5 17:34:57 2022
Thread-1: Wed Jan 5 17:34:57 2022
Thread-1: Wed Jan 5 17:34:58 2022
Exiting thread：Thread-1
Thread-2: Wed Jan 5 17:34:59 2022
Thread-2: Wed Jan 5 17:35:01 2022
Thread-2: Wed Jan 5 17:35:03 2022
Exiting thread：Thread-2
Exiting main thread

---

Thread Synchronization

If multiple threads modify the same data, unpredictable results may occur. To ensure data correctness, synchronization of multiple threads is required.

Using the Lock and Rlock objects of the Thread class can achieve simple thread synchronization. Both objects have acquire and release methods. For data that only allows one thread to operate at a time, its operation can be placed between the acquire and release methods. As follows:

The advantage of multithreading is that multiple tasks can run simultaneously (at least it feels that way). However, when threads need to share data, there may be a problem of data inconsistency.

Consider this situation: a list where all elements are 0, thread "set" changes all elements to 1 from back to front, and thread "print" is responsible for reading the list from front to back and printing it.

Then, when thread "set" starts to change, thread "print" may come to print the list, and the output becomes half 0 and half 1, which is data inconsistency. To avoid this situation, the concept of a lock is introduced.

A lock has two states -- locked and unlocked. Whenever a thread like "set" wants to access shared data, it must first acquire the lock; if another thread like "print" has already acquired the lock, then thread "set" is paused, that is, synchronously blocked; after thread "print" finishes accessing and releases the lock, thread "set" is allowed to continue.

After such processing, when printing the list, it will either output all 0 or all 1, and there will be no awkward situation of half 0 and half 1.

Example

#!/usr/bin/python3

import threading
import time

class myThread (threading.Thread):
    def __init__(self, threadID, name, delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay

    def run(self):
        print ("Starting thread： " + self.name)
        # Get the lock for thread synchronization
        threadLock.acquire()
        print_time(self.name, self.delay, 3)
        # Release the lock to start the next thread
        threadLock.release()

def print_time(threadName, delay, counter):
    while counter:
        time.sleep(delay)
        print ("%s: %s" % (threadName, time.ctime(time.time())))
        counter -= 1

threadLock = threading.Lock()
threads = []

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new threads
thread1.start()
thread2.start()

# Add threads to thread list
threads.append(thread1)
threads.append(thread2)

# Wait for all threads to complete
for t in threads:
    t.join()

print ("Exiting main thread")

Executing the above program, the output is:

Starting thread： Thread-1
Starting thread： Thread-2
Thread-1: Wed Jan 5 17:36:50 2022
Thread-1: Wed Jan 5 17:36:51 2022
Thread-1: Wed Jan 5 17:36:52 2022
Thread-2: Wed Jan 5 17:36:54 2022
Thread-2: Wed Jan 5 17:36:56 2022
Thread-2: Wed Jan 5 17:36:58 2022
Exiting main thread

---

Thread Priority Queue (Queue)

Python's Queue module provides synchronized, thread-safe queue classes, including FIFO (First-In-First-Out) queue Queue, LIFO (Last-In-First-Out) queue LifoQueue, and priority queue PriorityQueue.

These queues all implement lock primitives and can be used directly in multithreading. Queues can be used to achieve synchronization between threads.

Common methods in the Queue module:

• Queue.qsize() Returns the size of the queue
• Queue.empty() Returns True if the queue is empty, otherwise False
• Queue.full() Returns True if the queue is full, otherwise False
• Queue.full corresponds to maxsize
• Queue.get([block[, timeout]]) Gets the queue, timeout waiting time
• Queue.get_nowait() Equivalent to Queue.get(False)
• Queue.put(item) Writes to the queue, timeout waiting time
• Queue.put_nowait(item) Equivalent to Queue.put(item, False)
• Queue.task_done() After completing a task, the Queue.task_done() function sends a signal to the queue that the task is completed
• Queue.join() Actually means waiting until the queue is empty before performing other operations

Example

#!/usr/bin/python3

import queue
import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, q):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q

    def run(self):
        print ("Starting thread：" + self.name)
        process_data(self.name, self.q)
        print ("Exiting thread：" + self.name)

def process_data(threadName, q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            data = q.get()
            queueLock.release()
            print ("%s processing %s" % (threadName, data))
        else:
            queueLock.release()
        time.sleep(1)

threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads = []
threadID = 1

# Create new threads
for tName in threadList:
    thread = myThread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1

# Fill the queue
queueLock.acquire()
for word in nameList:
    workQueue.put(word)
queueLock.release()

# Wait for queue to empty
while not workQueue.empty():
    pass

# Notify threads it's time to exit
exitFlag = 1

# Wait for all threads to complete
for t in threads:
    t.join()

print ("Exiting main thread")

The execution result of the above program:

Starting thread：Thread-1
Starting thread：Thread-2
Starting thread：Thread-3
Thread-3 processing One
Thread-1 processing Two
Thread-2 processing Three
Thread-3 processing Four
Thread-1 processing Five
Exiting thread：Thread-3
Exiting thread：Thread-2
Exiting thread：Thread-1
Exiting main thread