Python异步IO实战:5个高效并发编程技巧
1. 基础协程实现
使用async/await创建简单协程:
import asyncio
async def hello_world():
print("Hello")
await asyncio.sleep(1)
print("World")
# 运行协程
asyncio.run(hello_world())
输出结果:
Hello
(等待1秒后)
World
2. 并发任务处理
使用gather并行执行多个任务:
async def fetch_data(url):
print(f"开始获取 {url}")
await asyncio.sleep(2) # 模拟网络请求
return f"{url} 的数据"
async def main():
tasks = [
fetch_data("https://api/1"),
fetch_data("https://api/2")
]
results = await asyncio.gather(*tasks)
print(results)
asyncio.run(main())
3. 异步上下文管理器
使用async with管理异步资源:
class AsyncConnection:
async def __aenter__(self):
print("建立连接")
return self
async def __aexit__(self, *args):
print("关闭连接")
async def use_connection():
async with AsyncConnection() as conn:
print("使用连接中...")
asyncio.run(use_connection())
4. 异步生成器
使用async for处理异步数据流:
async def async_counter(max):
for i in range(max):
yield i
await asyncio.sleep(0.5)
async def main():
async for num in async_counter(3):
print(f"收到: {num}")
asyncio.run(main())
方法 | 多线程 | 异步IO |
---|---|---|
性能 | 上下文切换开销大 | 单线程高效 |
复杂度 | 锁/同步问题 | 无竞争条件 |
适用场景 | CPU密集型 | IO密集型 |
5. 实战:异步Web爬虫
高效并发爬取网页内容:
import aiohttp
async def fetch_page(url):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.text()
async def crawl(urls):
tasks = [fetch_page(url) for url in urls]
return await asyncio.gather(*tasks)
urls = ["https://example.com/1", "https://example.com/2"]
pages = asyncio.run(crawl(urls))
Python异步IO为高性能网络编程提供了优雅的解决方案,特别适合Web爬虫、微服务、实时应用等IO密集型场景,能大幅提升程序的吞吐量和响应速度。