python - Aiohttp, Asyncio: RuntimeError: Event loop is closed -


i have 2 scripts, scraper.py , db_control.py. in scraper.py have this:

... def scrap(category, field, pages, search, use_proxy, proxy_file):     ...     loop = asyncio.get_event_loop()      to_do = [ get_pages(url, params, conngen) url in urls ]     wait_coro = asyncio.wait(to_do)     res, _ = loop.run_until_complete(wait_coro)     ...     loop.close()      return [ x.result() x in res ]  ... 

and in db_control.py:

from scraper import scrap ... while new < 15:     data = scrap(category, field, pages, search, use_proxy, proxy_file)     ... ... 

theoretically, scrapper should started unknown-times until enough of data have been obtained. when new not imidiatelly > 15 error occurs:

  file "/usr/lib/python3.4/asyncio/base_events.py", line 293, in run_until_complete self._check_closed()   file "/usr/lib/python3.4/asyncio/base_events.py", line 265, in _check_closed raise runtimeerror('event loop closed') runtimeerror: event loop closed 

but scripts works fine if run scrap() once. guess there problem recreating loop = asyncio.get_event_loop(), have tried this nothing changed. how can fix this? of course snippets of code, if think problem can elsewhere, full code available here.

methods run_until_complete, run_forever, run_in_executor, create_task, call_at explicitly check loop , throw exception if it's closed.

quote docs - baseevenloop.close:

this idempotent , irreversible


unless have some(good) reasons, might omit close line:

def scrap(category, field, pages, search, use_proxy, proxy_file):     #...     loop = asyncio.get_event_loop()      to_do = [ get_pages(url, params, conngen) url in urls ]     wait_coro = asyncio.wait(to_do)     res, _ = loop.run_until_complete(wait_coro)     #...     # loop.close()     return [ x.result() x in res ] 

if want have each time brand new loop, have t create manually , set default:

def scrap(category, field, pages, search, use_proxy, proxy_file):     #...     loop = asyncio.new_event_loop()     asyncio.set_event_loop(loop)         to_do = [ get_pages(url, params, conngen) url in urls ]     wait_coro = asyncio.wait(to_do)     res, _ = loop.run_until_complete(wait_coro)     #...     return [ x.result() x in res ] 

Comments

Popular posts from this blog

sequelize.js - Sequelize group by with association includes id -

android - Robolectric "INTERNET permission is required" -

java - Android raising EPERM (Operation not permitted) when attempting to send UDP packet after network connection -