Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When running tests, @pytest.mark.trio is not recognized #103

Open
PureTryOut opened this issue Aug 26, 2020 · 5 comments
Open

When running tests, @pytest.mark.trio is not recognized #103

PureTryOut opened this issue Aug 26, 2020 · 5 comments

Comments

@PureTryOut
Copy link

I'm running the tests as I described here (PYTHONPATH="." pytest --pyargs pytest_trio), but it seems it doesn't recognize the marker correctly:

====================================================== ERRORS =======================================================
_____________________________ ERROR collecting pytest_trio/_tests/test_clock_fixture.py _____________________________
pytest_trio/_tests/test_clock_fixture.py:5: in <module>
    @pytest.mark.trio
/usr/lib/python3.8/site-packages/_pytest/mark/structures.py:513: in __getattr__
    warnings.warn(
E   pytest.PytestUnknownMarkWarning: Unknown pytest.mark.trio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
______________________________ ERROR collecting pytest_trio/_tests/test_contextvars.py ______________________________
pytest_trio/_tests/test_contextvars.py:35: in <module>
    ???
/usr/lib/python3.8/site-packages/_pytest/mark/structures.py:513: in __getattr__
    warnings.warn(
E   pytest.PytestUnknownMarkWarning: Unknown pytest.mark.trio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
_____________________________ ERROR collecting pytest_trio/_tests/test_fixture_names.py _____________________________
pytest_trio/_tests/test_fixture_names.py:11: in <module>
    ???
/usr/lib/python3.8/site-packages/_pytest/mark/structures.py:513: in __getattr__
    warnings.warn(
E   pytest.PytestUnknownMarkWarning: Unknown pytest.mark.trio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
____________________________ ERROR collecting pytest_trio/_tests/test_fixture_nursery.py ____________________________
pytest_trio/_tests/test_fixture_nursery.py:17: in <module>
    ???
/usr/lib/python3.8/site-packages/_pytest/mark/structures.py:513: in __getattr__
    warnings.warn(
E   pytest.PytestUnknownMarkWarning: Unknown pytest.mark.trio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
________________________ ERROR collecting pytest_trio/_tests/test_hypothesis_interaction.py _________________________
pytest_trio/_tests/test_hypothesis_interaction.py:19: in <module>
    @pytest.mark.trio
/usr/lib/python3.8/site-packages/_pytest/mark/structures.py:513: in __getattr__
    warnings.warn(
E   pytest.PytestUnknownMarkWarning: Unknown pytest.mark.trio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
_____________________________ ERROR collecting pytest_trio/_tests/test_sync_fixture.py ______________________________
pytest_trio/_tests/test_sync_fixture.py:9: in <module>
    ???
/usr/lib/python3.8/site-packages/_pytest/mark/structures.py:513: in __getattr__
    warnings.warn(
E   pytest.PytestUnknownMarkWarning: Unknown pytest.mark.trio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html

This is the latest git master in an attempt to get the Alpine Linux package working again now pytest has been updated to 6.0.x
@altendky
Copy link
Member

This is certainly not all locked down as I'm no Docker expert... but there's a Dockerfile and output below that seem to work.

tl;dr: PYTHONPATH="." PYTEST_PLUGINS="pytest_trio.plugin" pytest --pyargs pytest_trio

Wall of text 
FROM alpine:edge

RUN apk update --update-cache
RUN apk add git py3-pytest py3-pytest-cov py3-trio py3-hypothesis

RUN git clone https://github.com/python-trio/pytest-trio
WORKDIR pytest-trio
RUN git checkout 57f458dc59468b9cf88deb6c21791c1ee0a15673
RUN PYTHONPATH="." PYTEST_PLUGINS="pytest_trio.plugin" pytest --pyargs pytest_trio
$ sudo docker build --no-cache --tag hmm .
Sending build context to Docker daemon  22.64MB
Step 1/7 : FROM alpine:edge
 ---> 3c791e92a856
Step 2/7 : RUN apk update --update-cache
 ---> Running in 3bde62a5bbfb
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
v20200626-4194-gb70c911952 [https://dl-cdn.alpinelinux.org/alpine/edge/main]
v20200626-4193-g859945b9f9 [https://dl-cdn.alpinelinux.org/alpine/edge/community]
OK: 13134 distinct packages available
Removing intermediate container 3bde62a5bbfb
 ---> 89c11dd0902c
Step 3/7 : RUN apk add git py3-pytest py3-pytest-cov py3-trio py3-hypothesis
 ---> Running in 9d12026a0096
(1/46) Installing ca-certificates (20191127-r4)
(2/46) Installing brotli-libs (1.0.9-r0)
(3/46) Installing nghttp2-libs (1.41.0-r0)
(4/46) Installing libcurl (7.72.0-r0)
(5/46) Installing expat (2.2.9-r1)
(6/46) Installing pcre2 (10.35-r0)
(7/46) Installing git (2.28.0-r1)
(8/46) Installing libbz2 (1.0.8-r1)
(9/46) Installing libffi (3.3-r2)
(10/46) Installing gdbm (1.13-r1)
(11/46) Installing xz-libs (5.2.5-r0)
(12/46) Installing ncurses-terminfo-base (6.2_p20200906-r1)
(13/46) Installing ncurses-libs (6.2_p20200906-r1)
(14/46) Installing readline (8.0.4-r0)
(15/46) Installing sqlite-libs (3.33.0-r0)
(16/46) Installing python3 (3.8.5-r0)
(17/46) Installing py3-attrs (20.2.0-r0)
(18/46) Installing py3-sortedcontainers (2.2.2-r0)
(19/46) Installing py3-hypothesis (5.33.2-r0)
(20/46) Installing py3-atomicwrites (1.4.0-r0)
(21/46) Installing py3-more-itertools (8.4.0-r0)
(22/46) Installing py3-iniconfig (1.0.1-r0)
(23/46) Installing py3-pluggy (0.13.1-r0)
(24/46) Installing py3-py (1.9.0-r0)
(25/46) Installing py3-six (1.15.0-r0)
(26/46) Installing py3-wcwidth (0.2.5-r0)
(27/46) Installing py3-ordered-set (4.0.2-r0)
(28/46) Installing py3-appdirs (1.4.4-r1)
(29/46) Installing py3-parsing (2.4.7-r1)
(30/46) Installing py3-packaging (20.4-r0)
(31/46) Installing py3-setuptools (50.3.0-r0)
(32/46) Installing py3-toml (0.10.1-r0)
(33/46) Installing py3-pytest (6.0.1-r1)
(34/46) Installing py3-coverage (5.2.1-r0)
(35/46) Installing py3-pytest-cov (2.10.1-r0)
(36/46) Installing py3-curio (1.4-r0)
(37/46) Installing py3-sniffio (1.1.0-r0)
(38/46) Installing py3-async_generator (1.10-r1)
(39/46) Installing py3-outcome (1.0.1-r0)
(40/46) Installing py3-idna (2.10-r0)
(41/46) Installing py3-cparser (2.20-r0)
(42/46) Installing py3-cffi (1.14.2-r0)
(43/46) Installing py3-asn1crypto (1.4.0-r0)
(44/46) Installing py3-cryptography (2.9.2-r0)
(45/46) Installing py3-openssl (19.1.0-r0)
(46/46) Installing py3-trio (0.16.0-r0)
Executing busybox-1.31.1-r21.trigger
Executing ca-certificates-20191127-r4.trigger
OK: 93 MiB in 60 packages
Removing intermediate container 9d12026a0096
 ---> 85559c8bc339
Step 4/7 : RUN git clone https://github.com/python-trio/pytest-trio
 ---> Running in 7d25546e620c
Cloning into 'pytest-trio'...
Removing intermediate container 7d25546e620c
 ---> aa977755f8fe
Step 5/7 : WORKDIR pytest-trio
 ---> Running in 511a661a18e5
Removing intermediate container 511a661a18e5
 ---> b0fc2a7b4f66
Step 6/7 : RUN git checkout 57f458dc59468b9cf88deb6c21791c1ee0a15673
 ---> Running in 3c3d5dc640cf
Note: switching to '57f458dc59468b9cf88deb6c21791c1ee0a15673'.

You are in 'detached HEAD' state. You can look around, make experimental                                                                                             
changes and commit them, and you can discard any commits you make in this                                                                                            
state without impacting any branches by switching back to a branch.                                                                                                  

If you want to create a new branch to retain commits you create, you may                                                                                             
do so (now or later) by using -c with the switch command. Example:                                                                                                   

  git switch -c <new-branch-name>                                                                                                                                    

Or undo this operation with:                                                                                                                                         

  git switch -                                                                                                                                                       

Turn off this advice by setting config variable advice.detachedHead to false                                                                                         

HEAD is now at 57f458d Merge pull request #106 from altendky/correct_example_assertion                                                                               
Removing intermediate container 3c3d5dc640cf
 ---> 6304af436d8b
Step 7/7 : RUN PYTHONPATH="." PYTEST_PLUGINS="pytest_trio.plugin" pytest --pyargs pytest_trio
 ---> Running in 1e5b1193849d
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/pytest-trio/.hypothesis/examples')
rootdir: /pytest-trio, configfile: pytest.ini
plugins: hypothesis-5.33.2, cov-2.10.1
collecting ... collected 53 items

pytest_trio/_tests/test_async_fixture.py::test_single_async_fixture PASSED [  1%]
pytest_trio/_tests/test_async_fixture.py::test_async_fixture_recomputed_for_each_test PASSED [  3%]
pytest_trio/_tests/test_async_fixture.py::test_nested_async_fixture PASSED [  5%]
pytest_trio/_tests/test_async_fixture.py::test_async_within_sync_fixture PASSED [  7%]
pytest_trio/_tests/test_async_fixture.py::test_raise_in_async_fixture_cause_pytest_error XFAIL [  9%]
pytest_trio/_tests/test_async_yield_fixture.py::test_single_async_yield_fixture[Python>=36] PASSED [ 11%]
pytest_trio/_tests/test_async_yield_fixture.py::test_single_async_yield_fixture[async_generator] PASSED [ 13%]
pytest_trio/_tests/test_async_yield_fixture.py::test_nested_async_yield_fixture[Python>=36] PASSED [ 15%]
pytest_trio/_tests/test_async_yield_fixture.py::test_nested_async_yield_fixture[async_generator] PASSED [ 16%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_within_sync_fixture[Python>=36] PASSED [ 18%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_within_sync_fixture[async_generator] PASSED [ 20%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_within_sync_yield_fixture[Python>=36] PASSED [ 22%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_within_sync_yield_fixture[async_generator] PASSED [ 24%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_with_multiple_yields[Python>=36] PASSED [ 26%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_with_multiple_yields[async_generator] PASSED [ 28%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_with_nursery[Python>=36] PASSED [ 30%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_with_nursery[async_generator] PASSED [ 32%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_crashed_teardown_allow_other_teardowns[Python>=36] PASSED [ 33%]
pytest_trio/_tests/test_async_yield_fixture.py::test_async_yield_fixture_crashed_teardown_allow_other_teardowns[async_generator] PASSED [ 35%]
pytest_trio/_tests/test_basic.py::test_async_test_is_executed PASSED     [ 37%]
pytest_trio/_tests/test_basic.py::test_async_test_as_class_method PASSED [ 39%]
pytest_trio/_tests/test_basic.py::test_sync_function_with_trio_mark XFAIL [ 41%]
pytest_trio/_tests/test_clock_fixture.py::test_sleep_with_autojump_clock PASSED [ 43%]
pytest_trio/_tests/test_contextvars.py::test_contextvars PASSED          [ 45%]
pytest_trio/_tests/test_fixture_mistakes.py::test_trio_fixture_with_non_trio_test PASSED [ 47%]
pytest_trio/_tests/test_fixture_mistakes.py::test_trio_fixture_with_wrong_scope_without_trio_mode PASSED [ 49%]
pytest_trio/_tests/test_fixture_mistakes.py::test_trio_fixture_with_wrong_scope_in_trio_mode[enable_trio_mode_via_pytest_ini] PASSED [ 50%]
pytest_trio/_tests/test_fixture_mistakes.py::test_trio_fixture_with_wrong_scope_in_trio_mode[enable_trio_mode_via_conftest_py] PASSED [ 52%]
pytest_trio/_tests/test_fixture_mistakes.py::test_async_fixture_with_sync_test_in_trio_mode[enable_trio_mode_via_pytest_ini] PASSED [ 54%]
pytest_trio/_tests/test_fixture_mistakes.py::test_async_fixture_with_sync_test_in_trio_mode[enable_trio_mode_via_conftest_py] PASSED [ 56%]
pytest_trio/_tests/test_fixture_mistakes.py::test_fixture_cancels_test_but_doesnt_raise[enable_trio_mode_via_pytest_ini] PASSED [ 58%]
pytest_trio/_tests/test_fixture_mistakes.py::test_fixture_cancels_test_but_doesnt_raise[enable_trio_mode_via_conftest_py] PASSED [ 60%]
pytest_trio/_tests/test_fixture_names.py::test_fixture_names PASSED      [ 62%]
pytest_trio/_tests/test_fixture_nursery.py::test_try PASSED              [ 64%]
pytest_trio/_tests/test_fixture_ordering.py::test_fixture_basic_ordering PASSED [ 66%]
pytest_trio/_tests/test_fixture_ordering.py::test_nursery_fixture_teardown_ordering PASSED [ 67%]
pytest_trio/_tests/test_fixture_ordering.py::test_error_collection PASSED [ 69%]
pytest_trio/_tests/test_fixture_ordering.py::test_background_crash_cancellation_propagation[nursery fixture] PASSED [ 71%]
pytest_trio/_tests/test_fixture_ordering.py::test_background_crash_cancellation_propagation[manual nursery] PASSED [ 73%]
pytest_trio/_tests/test_fixture_ordering.py::test_complex_cancel_interaction_regression PASSED [ 75%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_mark_inner PASSED [ 77%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_mark_outer PASSED [ 79%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_mark_and_parametrize[1] PASSED [ 81%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_mark_and_parametrize[2] PASSED [ 83%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_the_trio_scheduler_is_deterministic_under_hypothesis PASSED [ 84%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_the_trio_scheduler_is_not_deterministic <- ../usr/lib/python3.8/site-packages/trio/tests/test_scheduler_determinism.py PASSED [ 86%]
pytest_trio/_tests/test_hypothesis_interaction.py::test_the_trio_scheduler_is_deterministic_if_seeded <- ../usr/lib/python3.8/site-packages/trio/tests/test_scheduler_determinism.py PASSED [ 88%]
pytest_trio/_tests/test_sync_fixture.py::test_single_sync_fixture PASSED [ 90%]
pytest_trio/_tests/test_sync_fixture.py::test_single_yield_fixture PASSED [ 92%]
pytest_trio/_tests/test_sync_fixture.py::test_single_yield_fixture_with_async_deps PASSED [ 94%]
pytest_trio/_tests/test_sync_fixture.py::test_sync_yield_fixture_crashed_teardown_allow_other_teardowns PASSED [ 96%]
pytest_trio/_tests/test_trio_mode.py::test_trio_mode[enable_trio_mode_via_pytest_ini] PASSED [ 98%]
pytest_trio/_tests/test_trio_mode.py::test_trio_mode[enable_trio_mode_via_conftest_py] PASSED [100%]

----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name                                                Stmts   Miss Branch BrPart    Cover
---------------------------------------------------------------------------------------
pytest_trio/__init__.py                                 3      3      0      0     0.0%
pytest_trio/_tests/__init__.py                          0      0      0      0   100.0%
pytest_trio/_tests/conftest.py                          1      0      0      0   100.0%
pytest_trio/_tests/helpers.py                           6      0      0      0   100.0%
pytest_trio/_tests/test_async_fixture.py               22      0      0      0   100.0%
pytest_trio/_tests/test_async_yield_fixture.py         41      0      4      0   100.0%
pytest_trio/_tests/test_basic.py                       14      0      0      0   100.0%
pytest_trio/_tests/test_clock_fixture.py               10      0      2      0   100.0%
pytest_trio/_tests/test_contextvars.py                 26      0      0      0   100.0%
pytest_trio/_tests/test_fixture_mistakes.py            34      0      0      0   100.0%
pytest_trio/_tests/test_fixture_names.py               12      0      2      0   100.0%
pytest_trio/_tests/test_fixture_nursery.py             15      0      0      0   100.0%
pytest_trio/_tests/test_fixture_ordering.py            29      0      2      0   100.0%
pytest_trio/_tests/test_hypothesis_interaction.py      34      0      0      0   100.0%
pytest_trio/_tests/test_sync_fixture.py                20      0      0      0   100.0%
pytest_trio/_tests/test_trio_mode.py                    9      0      0      0   100.0%
pytest_trio/_version.py                                 1      1      0      0     0.0%
pytest_trio/enable_trio_mode.py                         6      0      0      0   100.0%
pytest_trio/plugin.py                                 206     47     73      0    83.2%
---------------------------------------------------------------------------------------
TOTAL                                                 489     51     83      0    91.1%


=========================== short test summary info ============================
XFAIL pytest_trio/_tests/test_async_fixture.py::test_raise_in_async_fixture_cause_pytest_error
  Not implemented yet
XFAIL pytest_trio/_tests/test_basic.py::test_sync_function_with_trio_mark
  Raises pytest internal error so far...
======================== 51 passed, 2 xfailed in 5.13s =========================
Removing intermediate container 1e5b1193849d
 ---> 9c3ffc03ec8a
Successfully built 9c3ffc03ec8a
Successfully tagged hmm:latest

@altendky altendky mentioned this issue Sep 11, 2020
Open
@PureTryOut
Copy link
Author

Well this has nothing to do with Docker really, Alpine is used for more things 😉

I gave your command a shot and it failed, although differently:

ValueError: Plugin already registered: pytest_trio.plugin=<module 'pytest_trio.plugin' from '/home/bart/Documents/Git/alpine/aports/community/py3-pytest-trio/src/pytest-trio-0.6.0/pytest_trio/plugin.py'>

Strange that it works on the Dockerfile, I don't really do anything different...

@altendky
Copy link
Member

Sorry for piling on with the ignorance there... and actually I recreated the same issue outside in my Ubuntu install.

Could you share the full command line and output anyways? Also include python -m pip freeze for the Python that the pytest is running from. And type python and head $(which pytest). Adjust all as needed so as to have them using the same Python. I'm looking to make sure pytest-trio isn't somehow installed now since the error sounds like it is and the PYTEST_PLUGINS is now redundant with the entry point plugin specification.

Or... maybe just seeing more stuff will help my light bulb turn on.

@hhartzer
Copy link

From what I can tell, pytest now refuses to run async tests without a marker, so it's not possible to use pytest-trio at the moment, with a recent pytest.

Or might I be mistaken about that?

@altendky
Copy link
Member

pytest has long had a feature where it complains about not-handled async tests. before they added that what happened was that it would execute the async call as a sync call, get a coroutine back, and consider the test passed since there was no exception. so you could easily get silent passes without running tests.

afaik, the documented mechanisms would generally be expected to work. more details would certainly be needed to debug the problem. given that we aren't clearly debugging the same situation, i would suggest a separate ticket be filed with all the details provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@altendky @PureTryOut @hhartzer