Python Queue

A queue in Python is a collection that follows the First-In-First-Out (FIFO) principle, meaning that the first element added to the queue will be the first one to be removed. Queues are commonly used in scenarios such as task scheduling, breadth-first search algorithms, and managing resources.

Key Features of Queues:

1. FIFO Structure: Elements are processed in the order they were added.

2. Mutable: You can add or remove elements dynamically.

3. Thread-Safe: Queues can be safely used in multi-threaded applications.

Example of a Python Queue: You can use the `queue.Queue` class from the `queue` module in Python. Here’s a basic example: 
import queue

# Create a queue
q = queue.Queue()

# Adding elements
q.put("task1")
q.put("task2")
q.put("task3")


Diagram Representation Here's a simple representation of the queue: 
Front
  |
  v
+-------+
| "task1"|
+-------+
| "task2"|
+-------+
| "task3"|
+-------+
Back
Breakdown of the Diagram

- Front: The front of the queue where elements are removed.

- Back: The back of the queue where elements are added.

- Elements: Each task is represented in a box, showing the order of insertion.

Step 1: Importing the Queue Module

The queue module is part of Python’s standard library, so there’s no need for additional installation. 
import queue

Step 2: Creating Different Types of Queues

The queue module provides several types of queues:
1. FIFO Queue (First-In-First-Out) - Basic queue where the first item added is the first to be removed.

2. LIFO Queue (Last-In-First-Out) - Stack-like queue where the last item added is the first to be removed.

3. Priority Queue - Items are sorted based on priority, not order of addition. 
# Creating different types of queues
fifo_queue = queue.Queue()       # FIFO
lifo_queue = queue.LifoQueue()    # LIFO
priority_queue = queue.PriorityQueue()  # Priority Queue

Step 3: Basic Operations (put, get, qsize, empty)

The put and get methods are the main operations for adding and removing items. Additionally, the qsize and empty methods help check the queue’s status. 
# Adding elements to FIFO Queue
fifo_queue.put("Task 1")
fifo_queue.put("Task 2")
fifo_queue.put("Task 3")
print("FIFO Queue Size:", fifo_queue.qsize())  # Get current size

# Retrieving elements from FIFO Queue
print(fifo_queue.get())  # Task 1
print(fifo_queue.get())  # Task 2
print("FIFO Queue is empty:", fifo_queue.empty())  # False

Output:

FIFO Queue Size: 3
Task 1
Task 2
FIFO Queue is empty: False
Explanation:
- `put(item)`: Adds an item to the queue.
- `get()`: Retrieves (and removes) the next item from the queue.
- `qsize()`: Returns the approximate size of the queue.
- `empty()`: Checks if the queue is empty.

Step 4: Working with LIFO (Stack) Queue

A LifoQueue works as a stack, where the last item added is the first one retrieved. 
# Adding elements to LIFO Queue
lifo_queue.put("Task A")
lifo_queue.put("Task B")
lifo_queue.put("Task C")

# Retrieving elements from LIFO Queue
print(lifo_queue.get())  # Task C
print(lifo_queue.get())  # Task B

Output:

Task C
Task B
Explanation:
- `LifoQueue()`: Creates a Last-In-First-Out (LIFO) queue.
- The `get` method retrieves the most recently added item, simulating stack behavior.

Step 5: Using Priority Queue for Task Prioritization

The PriorityQueue class allows items to be retrieved in order of priority, with lower values indicating higher priority. 
# Adding tasks with priorities to PriorityQueue
priority_queue.put((1, "High priority task"))
priority_queue.put((3, "Low priority task"))
priority_queue.put((2, "Medium priority task"))

# Retrieving tasks by priority
print(priority_queue.get())  # (1, 'High priority task')
print(priority_queue.get())  # (2, 'Medium priority task')

Output:

(1, 'High priority task')
(2, 'Medium priority task')
Explanation:
- `PriorityQueue()`: Stores items in priority order.
- Items are added as tuples (priority, item), where lower priority values come first when using get.

Step 6: Thread-Safe Queues

Queues in the queue module are thread-safe, meaning multiple threads can safely interact with them. This makes queue useful for multithreaded applications where shared data needs to be managed.

Example of Using a Queue in a Multithreaded Environment

This example demonstrates a producer-consumer scenario, where multiple producer threads add items to the queue, and consumer threads retrieve them. 
import threading
import queue
import time

# Create a FIFO queue
task_queue = queue.Queue()

# Producer thread function
def producer():
    for i in range(5):
        task_queue.put(f"Task {i}")
        print(f"Produced Task {i}")
        time.sleep(1)

# Consumer thread function
def consumer():
    while not task_queue.empty():
        task = task_queue.get()
        print(f"Consumed {task}")
        task_queue.task_done()
        time.sleep(1.5)

# Start producer and consumer threads
producer_thread = threading.Thread(target=producer)
consumer_thread = threading.Thread(target=consumer)

producer_thread.start()
time.sleep(0.5)  # Delay to allow some production before consumption
consumer_thread.start()

# Wait for both threads to finish
producer_thread.join()
consumer_thread.join()

print("All tasks processed.")

Output:

Produced Task 0
Produced Task 1
Consumed Task 0
Produced Task 2
Consumed Task 1
Produced Task 3
Consumed Task 2
Produced Task 4
Consumed Task 3
Consumed Task 4
All tasks processed.
Explanation:
- Producer Thread: Adds items to the queue every second.
- Consumer Thread: Retrieves items from the queue at a slower rate, simulating consumption time.
- `task_done()`: Indicates that a specific task is finished, which is useful when combining with join.

Step 7: Queue Timeout for Blocking Operations

The put and get methods can be set to block with a timeout if the queue is full or empty, respectively. This can be very useful in managing resources and avoiding indefinite blocking. 
# Adding items with a timeout
try:
    fifo_queue.put("Task 4", timeout=2)  # Waits up to 2 seconds to put item
    print("Task 4 added")
except queue.Full:
    print("Queue is full")

# Retrieving items with a timeout
try:
    print(fifo_queue.get(timeout=2))  # Waits up to 2 seconds to retrieve item
except queue.Empty:
    print("Queue is empty")

Output:

Task 4 added
Queue is empty
Explanation:
- `put(item, timeout=x)`: Waits for a specified amount of time before raising a `queue.Full` exception if the queue is full.
- `get(timeout=x)`: Waits for a specified amount of time before raising a `queue.Empty` exception if the queue is empty.

Step 8: Task Completion Tracking with task_done() and join()

To synchronize task completion, use task_done to signal task completion, and join to wait until all tasks are done. 
# Adding tasks to the queue
for i in range(3):
    fifo_queue.put(f"Task {i}")

# Define a worker that processes tasks
def worker():
    while not fifo_queue.empty():
        task = fifo_queue.get()
        print(f"Processing {task}")
        fifo_queue.task_done()

# Start worker threads
thread1 = threading.Thread(target=worker)
thread2 = threading.Thread(target=worker)

thread1.start()
thread2.start()

# Wait until all tasks are completed
fifo_queue.join()
print("All tasks processed.")

Output:

Processing Task 0
Processing Task 1
Processing Task 2
All tasks processed.
Explanation:
- `task_done()`: Used by the worker to signal that the task is done.
- `join()`: Blocks until all items in the queue are marked as done.

Conclusion

Queues are a fundamental data structure in Python, ideal for managing tasks and resources in a controlled manner. They are widely used in various applications, particularly where order of processing is critical!

Previous: Python Set | Next: Python namedtuple

<
>