我有一个简单的脚本,可以创建用于加载不同页面的异步任务.第一个请求因TimeoutError失败,导致下一个查询失败.但是第二个超时时间更长,应该通过.

有可能让其他查询不会失败吗？

import aiohttp
import asyncio
import logging


logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(name)s %(levelname)s %(message)s')


async def main():
    asyncio.ensure_future(
        load_page('https://www.netflix.com:5000'))

    asyncio.ensure_future(
        load_page('http://bash.im/', 10))

    asyncio.ensure_future(
        load_page('https://myshows.me/'))

    asyncio.ensure_future(
        load_page('http://www.lostfilm.tv/'))


async def load_page(url, timeout=3):
    try:
        async with session.get(url, timeout=timeout) as response:
            text = await response.text()
            print(len(text))

    except Exception:
        logging.warning(type(e))


if __name__ == '__main__':
    loop = asyncio.get_event_loop()

    conn = aiohttp.TCPConnector(limit=1)
    session = aiohttp.ClientSession(connector=conn, loop=loop)

    asyncio.ensure_future(main())
    loop.run_forever()

日志：

2017-06-26 13:57:37,869 asyncio DEBUG Using selector: EpollSelector
2017-06-26 13:57:41,780 root WARNING <class 'concurrent.futures._base.TimeoutError'>
2017-06-26 13:57:41,780 root WARNING <class 'concurrent.futures._base.TimeoutError'>
2017-06-26 13:57:41,780 root WARNING <class 'concurrent.futures._base.TimeoutError'>
2017-06-26 13:57:48,780 root WARNING <class 'concurrent.futures._base.TimeoutError'>

解决方法:

是的,这是可能的.我重写了你的代码并使用了一些这个概念来填充你的请求：

>我使用itertools.starmap来传递多个参数并创建传递给load_page函数的每个参数的列表.
>我使用asyncio.gather来将任务包装在一起,并使用了将return_exceptions标志更改为true以确保不会引发异常.
>在def main之前更改了异步.它现在返回收集的任务.
>我最后关闭了循环.

代码：

import aiohttp
import asyncio
import logging
import itertools

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(name)s %(levelname)s %(message)s')

def main(session):
    args = [
        ('https://www.netflix.com:5000', session,),
        ('http://bash.im/', session, 10),
        ('https://myshows.me/', session,),
        ('http://www.lostfilm.tv/', session,),
    ]
    tasks = itertools.starmap(load_page, args)
    futures = map(asyncio.ensure_future, tasks)

    return asyncio.gather(*futures, return_exceptions=True)


async def load_page(url, session, timeout=3):
    try:
        async with session.get(url, timeout=timeout) as response:
            text = await response.text()
            print(len(text))
    except Exception:
        logging.warning(type(e))


if __name__ == '__main__':
    loop = asyncio.get_event_loop()

    conn = aiohttp.TCPConnector(limit=1)
    session = aiohttp.ClientSession(connector=conn, loop=loop)
    loop.run_until_complete(main(session))
    loop.close()

欲了解更多信息：asyncio.gather.

欲了解更多信息,请访问：itertools.starmap.

请享用！