Using async libraries with PyTest

[amit-traiana]

Hello everyone!

I am experiencing inconsistent behaviour when using async/sync fixtures. Our tests using pyppeteer (Python wrapper around Google's Puppeteer library). Almost all pyppeteer methods are async. A common code will look like so:

from pyppeteer import launch

browser = await launch()
page = await browser.newPage()
await page.goto('http://example.com')
await page.screenshot({'path': 'example.png'})
await browser.close()

So in order to be able to run async tests, I'm using pytest-asyncio (and there actually an integration bug between pytest-asyncio and pyppeteer. A bug was opened on the pytest-asyncio that explains the issue). The above share a common code that should get executed before every test:

browser = await launch()
page = await browser.newPage()

and at the end of each test:

await browser.close()

So obviously, a fixture is needed here.

@pytest.fixture()
async def page():
    browser = await launch()
    page = await browser.newPage()
    yield page
    await browser.close()

The above will work properly if using a normal fixture (like seen in the sftp client example on the official documentation), and while it's a bit confusing yield returns a value and not an iterator - there no problem with using the fixture. But unlike yield in a non-async fixture, the yield here will not return the value of page, but instead will return an async-generator. So the following test will fail:

@pytest.mark.asyncio
async def test_some_test(page)
    await page.goto('http://example.com')

with the following error: AttributeError: 'async_generator' object has no attribute 'goto'. The only way around it is to manually iterate over async generator so the test will look like so:

@pytest.mark.asyncio
async def test_some_test(page)
    async for x in page:
        await page.goto('http://example.com')

It that the right way of doing it? because it seems a bit cumbersome and contradict how yield works on non-async methods.

Thanks!

[RonnyPfannschmidt]

please try with pytest < 3.7

[amit-traiana]

Looking good on 3.6.4!
Is that a regression? or something was changed in how async work on 3.7?

[RonnyPfannschmidt]

#3755 and similar where triggerd by code that was intended to trigger a deprecation

[amit-traiana]

So if I understand correctly pytest-asyncio trigger a code that is being deprecated on pytest? If that's the case, I'm guessing I should open an issue on pytest-asyncio. Can you perhaps share more information on what code was deprecated? maybe It's a small fix I can PR.

[RonnyPfannschmidt]

@amit-traiana no , pytest introducing the deprecation triggers the issue (as we wrap a async fixture with a generator

actually i just realized we dont handle async functions at all that way

so bascially the more recent pytest release wraps stuff in a way thats unfortunate

@nicoddemus ping on that just to cross-check if my understanding is correct

[amit-traiana]

Thanks for clearing it out @RonnyPfannschmidt . In that case, I think I'll revert back to 3.6.4 for now until @nicoddemus can comment on what can be done about it.

Thank you!

[nicoddemus]

@amit-traiana sorry for the delay. This issue has been fixed by #3780 and was released in 3.7.2.

This is also a duplicate of #3774, although the description is completely different. 👍

[amit-traiana]

Hello there @nicoddemus - No problem at all, I was happy with 3.6.4, and now I even more happy to switch to 3.7.2 :-)

Thanks you!

[nikan1996]

import pytest
from pyppeteer import launch


@pytest.mark.asyncio
async def test_some_test():
    browser = await launch()
    page = await browser.newPage()
    await page.goto('asd.com')  # wrong url
    await browser.close()

It is ok in pyppeteer==0.0.21 but hang forever in pyppeteer==0.0.20 when I use pytest-asyncio，
@amit-traiana