Warning about `asyncio_mode` even when unused in tests.

[cladmi]

When pytest-asyncio is installed, it would warn about its configuration even if nothings makes use of it in the test suite.
Meaning all tests on my machine would need to have the asyncio_mode configuration, even if not using it, to not have a warning reported.
(in a non virtualenv environment, I agree).

It would be nice to only warn if it's actually used in a test.

With a clean directory with only a bare test, and pytest-asyncio installed in the user directory:

python3 -m pip freeze | grep asyncio
pytest-asyncio==0.18.1

# test_lala.py 
def test_lala():
    pass

pytest
========================================= test session starts ==========================================
platform linux -- Python 3.8.10, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/gha/empty
plugins: html-3.1.1, cov-2.12.1, asyncio-0.18.1, requests-mock-1.9.3, metadata-1.11.0
asyncio: mode=legacy
collected 1 item                                                                                       

test_lala.py .                                                                                   [100%]

=========================================== warnings summary ===========================================
../.local/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191
  /home/gha/.local/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
    config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
===================================== 1 passed, 1 warning in 0.01s =====================================

[seifertm]

I understand that the warning may be confusing if pytest-asyncio is installed, but you're not making use of it in your project.

May I ask what prevents you from removing pytest-asyncio from your environment? Are you using a global pytest-asyncio installation or are you installing it in a virtualenv?

[cladmi]

Hi seifertm,

I am not using virtual environment for this, just relying on the same dependencies for different tests.
So everything installed as --user or globally. Things are in sync with the required versions across projects.

pytest-asyncio would be used to unit-tests some of the tool, but then the integration tests using this tools would not use it.
No "problem" with using the same dependencies versions as it's the same software, but the integration tests would not use pytest-asyncio and still have the warning.

My current solution was to take a version <0.17 on my machine.
Would be better to use tox for this, but then means documentation/colleagues update, so was not done yet.
So no blocker for me, just thought it make sense to report it.

Having this warning means pytest-asyncio should really only ever be installed in virtualenv or single app container.
Otherwise anytime you execute pytest outside one, it would show a warning.

It would become problematic when installed from system dependencies.
Debian did not update yet, but Arch already picked it https://archlinux.org/packages/community/any/python-pytest-asyncio/ .

[asvetlov]

The lazy asyncio mode detection is possible.
The problem is that: if there is an installed third-party pytest plugin that defines async fixtures (e.g. pytest-aiohttp) -- it implicitly requests asyncio mode detection; you'll see the 'legacy' warning again.
Thus, the usefulness of the proposed change is not high, IMHO.

[graingert]

distributed uses -p no:asyncio due to a catch 22 where --strict-config prevents setting asyncio_mode=strict

https://github.com/dask/distributed/blob/8d21040d22c98981c9d1c3ddcd99fd9e673d8f4f/setup.cfg#L51

[graingert]

@mgorny
re:

Hmm, I wonder if patching pytest-asyncio to default to auto would break stuff…

patching pytest-asyncio to auto would probably break stuff, "legacy" mode has been deprecated for a while now so it would be interesting to know how many gentoo tests would fail where pytest-asyncio to default to "strict":

patch here:

pytest-asyncio/pytest_asyncio/plugin.py

Lines 95 to 99 in 796ef05

	parser.addini(
	"asyncio_mode",
	help="default value for --asyncio-mode",
	default="strict",
	)

[mgorny]

I'm sorry this took so long but I've tested the packages that depend on pytest-asyncio in Gentoo today and the only one failing with this change is psycopg.

Perhaps we could add a USE flag that users could use to enable the new strict default for testing. CC @arthurzam, @thesamesam

Full list of packages tested

dev-python/aiofiles
dev-python/aiohttp-socks
dev-python/aioredis
dev-python/aiorpcX
dev-python/asgiref
dev-python/async-lru
dev-python/async-timeout
dev-python/autobahn
dev-python/backoff
dev-python/dbus-next
dev-python/denonavr
dev-python/elasticsearch-py
dev-python/fakeredis
dev-python/fsspec
dev-python/google-api-core
dev-python/httpcore
dev-python/httpx
dev-python/ipyparallel
dev-python/jeepney
dev-python/jikanpy
dev-python/jupyter_client
dev-python/outcome
dev-python/psycopg
dev-python/pytest-aiohttp
dev-python/pytest-httpx
dev-python/pytest-mock
dev-python/python-socks
dev-python/python-utils
dev-python/redis-py
dev-python/Rx
dev-python/simpervisor
dev-python/tekore
dev-python/thriftpy2
dev-python/tqdm
dev-python/uvicorn
dev-python/watchgod
dev-python/zeep
dev-python/zeroconf
media-sound/denonavr-cli
net-analyzer/wapiti
net-irc/znc-palaver
net-proxy/mitmproxy

[graingert]

@mgorny it looks like psycopg upstream has a fix already https://github.com/psycopg/psycopg/blob/13f15c3a5b472911ef6e17d645c2c54aee8bc675/pyproject.toml#L6

[mgorny]

Thanks for doing this!