Asynchronous programming Python: Asyncio, Aiohttp

What is Asynchronous Programming in Python

Asynchronous programming is a method of writing code where tasks are executed concurrently or by waiting for events without blocking the main execution thread. In modern programming, efficient use of resources and application runtime plays a key role.

Asynchronous programming allows you to:

Speed up task execution
Improve program responsiveness
Use system resources efficiently

Main Advantages of Asynchronicity

Areas of Application

Asynchronous programming is particularly useful for I/O operations:

File operations
Network requests (HTTP API)
Database interaction
Web socket processing
Parsing large amounts of data

Differences from Multithreading

Unlike multithreading, asynchronicity does not require creating additional threads or processes. This reduces the load on the system and avoids problems with blocking and synchronizing data between threads.

How Asynchronicity Works in Python

Python offers built-in support for asynchronous programming through the asyncio module, which was officially added to the standard library starting with version 3.4.

Basic Concepts

async def — defines an asynchronous function (coroutine)
await — indicates that you need to wait for the coroutine to complete
asyncio — a standard module for working with asynchronous tasks
event loop — an event processing cycle that manages the execution of coroutines

Basic Example of an Asynchronous Function

import asyncio

async def greet():
    print("Hello!")
    await asyncio.sleep(1)
    print("1 second has passed")

asyncio.run(greet())

In this example, await asyncio.sleep(1) simulates waiting for a resource-intensive operation, such as a network request.

Creating and Using Asynchronous Functions

Defining an Asynchronous Function

To create an asynchronous function, use the async keyword before the function definition:

async def fetch_data():
    print("Requesting data...")
    await asyncio.sleep(2)
    print("Data received")

Calling an Asynchronous Function

An asynchronous function is called via asyncio.run():

asyncio.run(fetch_data())

Parallel Execution of Multiple Tasks

Using `asyncio.gather()`

To run multiple coroutines in parallel, use asyncio.gather():

async def task(name, delay):
    print(f"Starting task {name}")
    await asyncio.sleep(delay)
    print(f"Task {name} completed")

async def main():
    await asyncio.gather(
        task("A", 2),
        task("B", 1),
        task("C", 3)
    )

asyncio.run(main())

Alternative Ways to Launch Tasks

In addition to asyncio.gather(), you can use:

asyncio.create_task() — to create individual tasks
asyncio.wait() — to wait for tasks to complete with additional settings
asyncio.as_completed() — to get results as they become available

Asynchronous Iterators and Generators

Creating an Asynchronous Iterator

To process data streams asynchronously, use asynchronous iterators:

class AsyncCounter:
    def __init__(self, max_count):
        self.max_count = max_count
        self.current = 0
    
    def __aiter__(self):
        return self
    
    async def __anext__(self):
        if self.current >= self.max_count:
            raise StopAsyncIteration
        self.current += 1
        await asyncio.sleep(1)
        return self.current

async def main():
    async for number in AsyncCounter(3):
        print(number)

asyncio.run(main())

Asynchronous Generators

Asynchronous generators allow you to create simpler asynchronous iterators:

async def async_generator():
    for i in range(3):
        await asyncio.sleep(1)
        yield i

async def main():
    async for value in async_generator():
        print(value)

asyncio.run(main())

Working with HTTP Requests via aiohttp

Installing the Library

pip install aiohttp

Performing an Asynchronous HTTP Request

import aiohttp
import asyncio

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

async def main():
    html = await fetch('https://www.python.org')
    print(html[:500])  # Output the first 500 characters

asyncio.run(main())

Multiple HTTP Requests

async def fetch_multiple_urls(urls):
    async with aiohttp.ClientSession() as session:
        tasks = []
        for url in urls:
            task = asyncio.create_task(fetch_url(session, url))
            tasks.append(task)
        
        results = await asyncio.gather(*tasks)
        return results

async def fetch_url(session, url):
    async with session.get(url) as response:
        return await response.text()

Error Handling in Asynchronous Code

Using try-except Blocks

async def safe_fetch(url):
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as response:
                return await response.text()
    except aiohttp.ClientError as e:
        print(f"Network error: {e}")
        return None
    except Exception as e:
        print(f"Unexpected error: {e}")
        return None

Handling Timeouts

async def fetch_with_timeout(url, timeout=5):
    try:
        timeout_obj = aiohttp.ClientTimeout(total=timeout)
        async with aiohttp.ClientSession(timeout=timeout_obj) as session:
            async with session.get(url) as response:
                return await response.text()
    except asyncio.TimeoutError:
        print(f"Timeout when requesting {url}")
        return None

When NOT to Use Asynchronous Programming

Inappropriate Scenarios

When performing heavy calculations (it is better to use multiprocessing)
If your project is not related to processing network or file operations
When it is easier to implement synchronous logic without the need for complex task management
When working with CPU-intensive tasks

Alternative Approaches

For CPU-intensive tasks, it is better to use:

The multiprocessing module for parallelizing computations
Libraries like concurrent.futures to manage a pool of threads
Specialized solutions for high-performance computing

Additional Tools and Libraries

Popular Asynchronous Libraries

aiohttp — for HTTP requests
aiomysql and aiopg — for asynchronous work with databases
FastAPI — an asynchronous web framework
aiofiles — for asynchronous work with files
asyncpg — for working with PostgreSQL

Integration with Synchronous Code

To execute synchronous code in an asynchronous function, use run_in_executor:

loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, sync_function)

Asynchronous Methods in Classes

Defining Asynchronous Methods

class AsyncExample:
    async def async_method(self):
        await asyncio.sleep(1)
        print("Asynchronous class method executed")
    
    async def fetch_data(self, url):
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as response:
                return await response.text()

Using Context Managers

class AsyncContextManager:
    async def __aenter__(self):
        print("Entering context")
        return self
    
    async def __aexit__(self, exc_type, exc_val, exc_tb):
        print("Exiting context")

async def main():
    async with AsyncContextManager() as manager:
        await asyncio.sleep(1)
        print("Working in context")

Conclusion

Asynchronous programming in Python is a powerful tool for improving application performance, especially when working with I/O operations. Understanding the basics of async, await, and working with the asyncio module allows you to create more efficient and responsive code.