Move the tests directory outside of the code directory?

[Armavica]

Hi,
With the reorganization of the tests complete (#5777), a last question remains: where to place the tests directory. It currently resides inside the pymc code directory:

pymc/
├── benchmarks/
├── docs/
├── pymc/
│   ├── __init__.py
│   ├── model.py
│   ├── ...
│   └── tests/
└── scripts/

but one could argue that it would make more sense to put it at the same level as benchmarks/, docs/ and pymc/:

pymc/
├── benchmarks/
├── docs/
├── pymc/
│   ├── __init__.py
│   ├── model.py
│   └── ...
├── scripts/
└── tests/

This is what is recommended by the official Python packaging guide, by the Pytest good practices guide, the Poetry package management tool as well as by other guides.
However, I have to note that numpy, scipy, and pandas all have tests inside the code directory.

Besides the arguments advanced by the Pytest good practices guide, I can add that I often grep the source code directory to see where or how a function or class is used or defined. With the tests directory where it is currently, a lot of noise is often returned from the tests.

Cost of the transition: probably limited to some imports to change in the test files to reflect the move.

What is your opinion on this?

[ricardoV94]

Sounds fine by me if nobody has objections.

Some people do seem to like to run all tests when they download a package but I think those are a very small minority.

[Armavica]

Some people do seem to like to run all tests

Would this be problematic with the tests directory outside of the code directory?

With aesara which follows this structure for example, launching pytest in the main directory does seem to run all the tests.

[ricardoV94]

I think the tests are no longer included in package installation in those cases.