diff --git a/docs/source/history.rst b/docs/source/history.rst index a27ad57..2f8fce7 100644 --- a/docs/source/history.rst +++ b/docs/source/history.rst @@ -5,6 +5,26 @@ Release history .. towncrier release notes start +pytest-trio 0.5.0 (????-??-??) +------------------------------ + +This is a major release, including a rewrite of large portions of the +internals. We believe it should be backwards compatible with existing +projects, but major new features include: + +* "trio mode": no more writing ``@pytest.mark.trio`` everywhere! +* it's now safe to use nurseries inside fixtures (`#55 + `__) +* new ``@trio_fixture`` decorator to explicitly mark a fixture as a + trio fixture +* a number of easy-to-make mistakes are now caught and raise + informative errors +* the :data:`nursery` fixture is now 87% more magical + +For more details, see the manual. Oh right, speaking of which: we +finally have a manual! You should read it. + + pytest-trio 0.4.2 (2018-06-29) ------------------------------ diff --git a/docs/source/index.rst b/docs/source/index.rst index 4682c05..9af200c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -15,16 +15,22 @@ and async I/O in Python. Features include: * Async tests without the boilerplate: just write ``async def test_whatever(): ...``. -* Useful fixtures included: use ``autojump_clock`` for easy testing of - code with timeouts. +* Useful fixtures included: use :data:`autojump_clock` for easy + testing of code with timeouts, or :data:`nursery` to easily set up + background tasks. * Write your own async fixtures: set up an async database connection or start a server inside a fixture, and then use it in your tests. + If you have multiple async fixtures, pytest-trio will even do + setup/teardown concurrently whenever possible. * Integration with the fabulous `Hypothesis `__ library, so your async tests can use property-based testing: just use ``@given`` like you're used to. +* Supports testing projects that use Trio exclusively, and also + projects that support multiple async libraries. + Vital statistics ================ diff --git a/docs/source/quickstart.rst b/docs/source/quickstart.rst index 04b3cc1..1c0682d 100644 --- a/docs/source/quickstart.rst +++ b/docs/source/quickstart.rst @@ -172,6 +172,13 @@ regular pytest fixture:: # Teardown code, executed after the test is done await connection.execute("ROLLBACK") +If you need to support Python 3.5, which doesn't allow ``yield`` +inside an ``async def`` function, then you can define async fixtures +using the `async_generator +`__ +library – just make sure to put the ``@pytest.fixture`` *above* the +``@async_generator``. + .. _server-fixture-example: @@ -191,10 +198,11 @@ arbitrary data, and then send it back out again:: # Usage: await trio.serve_tcp(echo_server_handler, ...) Since this is such complicated and sophisticated code, we want to -write lots of tests to make sure it's working correctly. To test it, -we want to start a background task running the echo server itself, and -then we'll connect to it, send it test data, and see how it responds. -Here's a first attempt:: +write lots of tests to make sure it's working correctly. Our basic +strategy for each test will be to start the echo server running in a +background task, and then in the test body we'll connect to our +server, send it some test data, and see how it responds. Here's a +first attempt:: # Let's cross our fingers and hope no-one else is using this port... PORT = 14923 @@ -208,12 +216,12 @@ Here's a first attempt:: ) # Connect to the server. - client = await trio.open_tcp_stream("127.0.0.1", PORT) + echo_client = await trio.open_tcp_stream("127.0.0.1", PORT) # Send some test data, and check that it gets echoed back - async with client: - for test_byte in [b"a", b"c", b"c"]: - await client.send_all(test_byte) - assert await client.receive_some(1) == test_byte + async with echo_client: + for test_byte in [b"a", b"b", b"c"]: + await echo_client.send_all(test_byte) + assert await echo_client.receive_some(1) == test_byte This will mostly work, but it has a few problems. The first one is that there's a race condition: We spawn a background task to call @@ -232,7 +240,7 @@ but basically the idea is that both ``nursery.start_soon(...)`` and ``start`` waits for the new task to finish getting itself set up. This requires some cooperation from the background task: it has to notify ``nursery.start`` when it's ready. Fortunately, :func:`trio.serve_tcp` -already knows how to cooperated with ``nursery.start``, so we can +already knows how to cooperate with ``nursery.start``, so we can write:: # Let's cross our fingers and hope no-one else is using this port... @@ -248,11 +256,11 @@ write:: ) # Connect to the server - client = await trio.open_tcp_stream("127.0.0.1", PORT) - async with client: - for test_byte in [b"a", b"c", b"c"]: - await client.send_all(test_byte) - assert await client.receive_some(1) == test_byte + echo_client = await trio.open_tcp_stream("127.0.0.1", PORT) + async with echo_client: + for test_byte in [b"a", b"b", b"c"]: + await echo_client.send_all(test_byte) + assert await echo_client.receive_some(1) == test_byte That solves our race condition. Next issue: hardcoding the port number like this is a bad idea, because port numbers are a machine-wide @@ -294,11 +302,11 @@ Putting it all together:: # There might be multiple listeners (example: IPv4 and # IPv6), but we don't care which one we connect to, so we # just use the first. - client = await open_stream_to_socket_listener(listeners[0]) - async with client: - for test_byte in [b"a", b"c", b"c"]: - await client.send_all(test_byte) - assert await client.receive_some(1) == test_byte + echo_client = await open_stream_to_socket_listener(listeners[0]) + async with echo_client: + for test_byte in [b"a", b"b", b"c"]: + await echo_client.send_all(test_byte) + assert await echo_client.receive_some(1) == test_byte Okay, this is getting closer... but if we try to run it, we'll find that it just hangs instead of completing. What's going on? @@ -311,54 +319,54 @@ our test is finished:: # Don't copy this -- it finally works, but we can still do better! async def test_attempt_4(): async with trio.open_nursery() as nursery: - # Start server running in the background - # AND wait for it to finish starting up before continuing - # AND find out where it's actually listening - listeners = await nursery.start( - partial(trio.serve_tcp, echo_server_handler, port=0) - ) - - # Connect to the server. - # There might be multiple listeners (example: IPv4 and - # IPv6), but we don't care which one we connect to, so we - # just use the first. - client = await open_stream_to_socket_listener(listeners[0]) - async with client: - for test_byte in [b"a", b"c", b"c"]: - await client.send_all(test_byte) - assert await client.receive_some(1) == test_byte - - # Shut down the server now that we're done testing it - nursery.cancel_scope.cancel() + try: + listeners = await nursery.start( + partial(trio.serve_tcp, echo_server_handler, port=0) + ) + echo_client = await open_stream_to_socket_listener(listeners[0]) + async with echo_client: + for test_byte in [b"a", b"b", b"c"]: + await echo_client.send_all(test_byte) + assert await echo_client.receive_some(1) == test_byte + finally: + # Shut down the server now that we're done testing it + nursery.cancel_scope.cancel() Okay, finally this test works correctly. But that's a lot of -boilerplate. Remember, we need to write lots of tests for this server, -and we don't want to have to copy-paste all that stuff into every -test. Let's factor it out into a fixture. +boilerplate. Can we slim it down? We can! -Probably our first attempt will look something like:: +First, pytest-trio provides a magical fixture that takes care of the +nursery management boilerplate: just request the :data:`nursery` fixture, +and you get a nursery object that's already set up, and will be +automatically cancelled when you're done:: - # DON'T DO THIS, IT DOESN'T WORK - @pytest.fixture - async def echo_server_connection(): - async with trio.open_nursery() as nursery: - await nursery.start(...) - yield ... - nursery.cancel_scope.cancel() + # Don't copy this -- it finally works, but we can *still* do better! + async def test_attempt_5(nursery): + listeners = await nursery.start( + partial(trio.serve_tcp, echo_server_handler, port=0) + ) + echo_echo_client = await open_stream_to_socket_listener(listeners[0]) + async with echo_client: + for test_byte in [b"a", b"b", b"c"]: + await echo_echo_client.send_all(test_byte) + assert await echo_echo_client.receive_some(1) == test_byte -Unfortunately, this doesn't work. **You cannot make a fixture that -opens a nursery or cancel scope, and then yields from inside the -nursery or cancel scope block.** Sorry 🙁. We're `working on it -`__. +And finally, remember, we need to write lots of tests, and we don't +want to have to copy-paste the server setup code every time. Let's +factor it out into a fixture:: -Instead, pytest-trio provides a built-in fixture called -``test_nursery``. This is a nursery that's created before each test, -and then automatically cancelled after the test finishes. Which is -exactly what we need – in fact it's even simpler than our first try, -because now we don't need to worry about cancelling the nursery -ourselves. + @pytest.fixture + async def echo_server_connection(nursery): + listeners = await nursery.start( + partial(trio.serve_tcp, echo_server_handler, port=0) + ) + echo_client = await open_stream_to_socket_listener(listeners[0]) + async with echo_client: + yield echo_client -So here's our complete, final version:: +And now in tests, all we have to do is request the ``echo_client`` +fixture, and we get a background server and a client stream connected +to it. So here's our complete, final version:: # Final version -- copy this! from functools import partial @@ -376,18 +384,18 @@ So here's our complete, final version:: # The fixture: @pytest.fixture - async def echo_server_connection(test_nursery): - listeners = await test_nursery.start( + async def echo_client(nursery): + listeners = await nursery.start( partial(trio.serve_tcp, echo_server_handler, port=0) ) - client = await open_stream_to_socket_listener(listeners[0]) - async with client: - yield client + echo_client = await open_stream_to_socket_listener(listeners[0]) + async with echo_client: + yield echo_client # A test using the fixture: - async def test_final(echo_server_connection): - for test_byte in [b"a", b"c", b"c"]: - await echo_server_connection.send_all(test_byte) - assert await echo_server_connection.receive_some(1) == test_byte + async def test_final(echo_client): + for test_byte in [b"a", b"b", b"c"]: + await echo_client.send_all(test_byte) + assert await echo_client.receive_some(1) == test_byte No race conditions, no hangs, simple, clean, and reusable. diff --git a/docs/source/reference.rst b/docs/source/reference.rst index 1021093..8ddd954 100644 --- a/docs/source/reference.rst +++ b/docs/source/reference.rst @@ -106,6 +106,42 @@ but Trio fixtures **must be test scoped**. Class, module, and session scope are not supported. +Concurrent setup/teardown +------------------------- + +If your test uses multiple fixtures, then for speed, pytest-trio will +try to run their setup and teardown code concurrently whenever this is +possible while respecting the fixture dependencies. + +Here's an example, where a test depends on ``fix_b`` and ``fix_c``, +and these both depend on ``fix_a``:: + + @trio_fixture + def fix_a(): + ... + + @trio_fixture + def fix_b(fix_a): + ... + + @trio_fixture + def fix_c(fix_a): + ... + + @pytest.mark.trio + async def test_example(fix_b, fix_c): + ... + +When running ``test_example``, pytest-trio will perform the following +sequence of actions: + +1. Set up ``fix_a`` +2. Set up ``fix_b`` and ``fix_c``, concurrently. +3. Run the test. +4. Tear down ``fix_b`` and ``fix_c``, concurrently. +5. Tear down ``fix_a``. + + Built-in fixtures ----------------- @@ -140,16 +176,34 @@ available. For example, you can call mock_clock.jump(10) assert trio.current_time() == 10 -.. data:: test_nursery - - A nursery created and managed by pytest-trio itself. When - pytest-trio runs a test, it performs these steps in this order: - - 1. Open the ``test_nursery`` - 2. Set up all Trio fixtures. - 3. Run the test. - 4. Tear down all Trio fixtures. - 5. Cancel the ``test_nursery``. +.. data:: nursery + + A nursery created and managed by pytest-trio itself, which + surrounds the test/fixture that requested it, and is automatically + cancelled after the test/fixture completes. Basically, these are + equivalent:: + + # Boring way + async def test_with_background_task(): + async with trio.open_nursery() as nursery: + try: + ... + finally: + nursery.cancel_scope.cancel() + + # Fancy way + async def test_with_background_task(nursery): + ... + + For a fixture, the cancellation always happens after the fixture + completes its teardown phase. (Or if it doesn't have a teardown + phase, then the cancellation happens after the teardown phase + *would* have happened.) + + This fixture is even more magical than most pytest fixtures, + because if it gets requested several times within the same test, + then it creates multiple nurseries, one for each fixture/test that + requested it. See :ref:`server-fixture-example` for an example of how this can be used.