Skip to content

pytest importlib doesn't work with pickle #7859

Closed
@yoav-orca

Description

@yoav-orca

I'm transitioning from pytest regular imports to --import-mode=importlib imports due to namespace collisions in tests. And I'm getting the following crash, no plugins installed, pytest 6.1.1 CPython 3.8.6, the issue reproduces on Mac & Linux.

Create a file named test_xx.py:

import pickle

def do_action():
    pass

def test_x():
    pickle.dumps(do_action)
$ pytest -v --import-mode=importlib test_xx.py
==================================================== test session starts ====================================================
platform darwin -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /Users/yoav/.local/share/virtualenvs/qwer-HyU7-4uu/bin/python
cachedir: .pytest_cache
rootdir: /private/tmp/qwer
collected 1 item

test_xx.py::test_x FAILED                                                                                             [100%]

========================================================= FAILURES ==========================================================
__________________________________________________________ test_x ___________________________________________________________

    def test_x():
>       pickle.dumps(do_action)
E       _pickle.PicklingError: Can't pickle <function do_action at 0x10f8aa3a0>: import of module 'test_xx' failed

test_xx.py:7: PicklingError
================================================== short test summary info ==================================================
FAILED test_xx.py::test_x - _pickle.PicklingError: Can't pickle <function do_action at 0x10f8aa3a0>: import of module 'tes...
===================================================== 1 failed in 0.05s =====================================================

Running the same code without pytest or with pytest without importlib work as expected.

pip list:

Package    Version
---------- -------
attrs      20.2.0
iniconfig  1.0.1
packaging  20.4
pip        20.1.1
pluggy     0.13.1
py         1.9.0
pyparsing  2.4.7
pytest     6.1.1
setuptools 50.3.0
six        1.15.0
toml       0.10.1
wheel      0.35.1

I encountered this issue because we have tests using multiprocessing.Pool, and they fail on pickling the target function.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugproblem that needs to be addressed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions