Thursday, October 19, 2017

How to mock os.listdir to pretend files and directories in Python?

Leave a Comment

I have a proprietary repository format and I'm trying to develop a Python module to process these repositories. Repo format goes as:

/home/X/        |        + alpha/        |        + beta/        |        + project.conf 

Here, X is a project. alpha and beta are folders inside this project and they represent groups in this project. A group is a container in this repo and what it represents is really not relevant for this question. The repo X also has files in its root level; project.conf is an example of such a file.

I have a class called Project that abstracts projects such as X. The Project class has a method load() that builds an in-memory representation.

class Project(object):      def load(self):         for entry in os.listdir(self.root):             path = os.path.join(self.root, entry)             if os.path.isdir(path):                 group = Group(path)                 self.groups.append(group)                 group.load()             else:                 # process files ... 

To unit test the load() method by mocking the file system, I have:

import unittest from unittest import mock import Project  class TestRepo(unittest.TestCase):      def test_load_project(self):         project = Project("X")          with mock.patch('os.listdir') as mocked_listdir:             mocked_listdir.return_value = ['alpha', 'beta', 'project.conf']             project.load()             self.assertEqual(len(project.groups), 2) 

This does mock os.listdir successfully. But I can't trick Python to treat mocked_listdir.return_value as consisting of files and directories.

How do I mock either os.listdir or os.path.isdir, in the same test, such that the test will see alpha and beta as directories and project.conf as a file?

3 Answers

Answers 1

You could use pyfakefs, it is a very handy lib to test operates on a fake file system.

if you use pytest, it has a plugin, all file system functions already got patched, you just need to use its fs fixture:

import os  def test_foo(fs):     fs.CreateFile('/home/x/alpha/1')     fs.CreateFile('/home/x/beta/2')     fs.CreateFile('/home/x/p.conf')         assert os.listdir('/home/x/')) == ['alpha', 'beta', 'p.conf'] 

or if you prefer unittest:

import os import unittest  from pyfakefs import fake_filesystem_unittest  class TestRepo(fake_filesystem_unittest.TestCase):      def setUp(self):         self.setUpPyfakefs()      def test_foo(self):         os.makedirs('/home/x/alpha')         os.makedirs('/home/x/beta')         with open('/home/x/p.conf', 'w') as f:             f.write('foo')         self.assertEqual(os.listdir('/home/x/'), ['alpha', 'beta', 'p.conf'])   if __name__ == "__main__":     unittest.main() 

Answers 2

I managed to achieve the desired behavior by passing an iterable to the side_effect attribute of the mocked isdir object.

import unittest from unittest import mock import Project  class TestRepo(unittest.TestCase):    def test_load_project(self):       project = Project("X")        with mock.patch('os.listdir') as mocked_listdir:         with mock.patch('os.path.isdir') as mocked_isdir:           mocked_listdir.return_value = ['alpha', 'beta', 'project.conf']           mocked_isdir.side_effect = [True, True, False]           project.load()           self.assertEqual(len(project.groups), 2) 

The key is the mocked_isdir.side_effect = [True, True, False] line. The boolean values in the iterable should match the order of directory and file entries passed to the mocked_listdir.return_value attribute.

Answers 3

It will depend, of course, on exactly which os functions you use, but it looks like mock.patch.multiple on os is just what you need. (Note that you may not need to patch path; many of its functions are lexical-only and do not care about the actual filesytem.)

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment