I have a python library with the following repository structure:
repobase |- mylibrary | |- __init__.py |- tests |- test_mylibrary.py Up to now, running the tests could simply be done by calling py.test in the repobase directory. The import mylibrary in test_mylibrary.py then used the local code in repobase/mylibrary.
Now, I've extended the library to use compiled code. Therefore the source code at repobase/mylibrary is not functional on its own. I have to do a setup.py build. This creates repobase/build/lib.linux-x86_64-2.7/mylibrary.
Is there a reasonable way to make py.test use this directory for importing mylibrary? Given these constraints:
I do not want to include any
sys.path/ import magic in test_mylibrary.py because this may break tests in other envrionments.I don't want to give up the possibility to run
py.testfrom repobase. Therefore modifying the PYTHONPATH does not help because.will still be first insys.path. And thus repobase/mylibrary would be favored over repobase/build/lib.linux-x86_64-2.7/mylibrary.
If not, what's the standard way for testing python libraries, that need building?
2 Answers
Answers 1
I think your problem is simply that py.test is not copying the built shared object to the root of your repository.
I just tried running the UT straight from the Python wiki on testing C extensions using py.test as follows:
python setup.py build py.test test/examp_unittest.py This failed with AssertionError: No module named examp.
However, when I follow the wiki to the letter (and run python setup.py test instead), I note that it copies the .so to the root directory (note the last line before it starts running the test):
running test running egg_info writing examp.egg-info/PKG-INFO writing top-level names to examp.egg-info/top_level.txt writing dependency_links to examp.egg-info/dependency_links.txt reading manifest file 'examp.egg-info/SOURCES.txt' writing manifest file 'examp.egg-info/SOURCES.txt' running build_ext copying build/lib.linux-x86_64-2.6/examp.so -> runTest (test.examp_unittest.DeviceTest) ... ok ---------------------------------------------------------------------- Ran 1 test in 0.001s OK Having run that on my system, I can now run py.test quite happily on the same code base - as shown below.
============================= test session starts ============================== platform linux2 -- Python 2.7.3, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /tmp/sotest, inifile: collected 1 items test/examp_unittest.py . =========================== 1 passed in 0.01 seconds =========================== The solution is therefore to copy your shared object to the root of your repository.
To make sure I have run the whole thing from scratch, just building the extension, copying the shared object and then running py.test. This all works as expected.
Answers 2
From the discussion in chat, it sounds as if the C implementation only provides a subset of the functionality of the Python implementation.
A common solution is to split the module such that the parts which require optimized implementations exist in a separate module.
Consider a more concrete example of a library which needs convert between different image formats.
Let's assume your layout looks like this...
repobase |- image | |- __init__.py | |- pyJPEG.py |- build | |- lib.linux-x86_64-2.7 | |- cJPEG.so |- tests |- test_image.py ...your PYTHONPATH includes /path/to/repobase:/path/to/repobase/build/lib.linux-x86_64-2.7, your cJPEG.so exports symbols jpeg_decompress and jpeg_compress, and your files look like this...
image/__init__.py
# Load the C implementation if we have it, otherwise fall back to # a pure Python implementation try: from cJPEG import jpeg_decompress, jpeg_compress except ImportError: from pyJPEG import jpeg_decompress, jpeg_compress def load_image(filename): data = open(filename, 'rb').read() if filename.endswidth('.jpg'): return jpeg_decompress(data) else: raise NotImplementedError def save_image(data, filename, filetype='JPEG'): if filetype == 'JPEG': data = jpeg_compress(data) else: raise NotImplementedError open(filename, 'wb').write(data) image/pyJPEG.py
def jpeg_decompress(data): # A pure Python implementation of a JPEG decoder def jpeg_compress(data): # A pure Python implementation of a JPEG encoder With this sort of layout, the test suite need not care whether the library is built or not - you can use the same suite in both cases, and the presence (or absence) of cJPEG.so will determine which version is tested.
0 comments:
Post a Comment