Saturday, August 25, 2018

How can I use Sphinx with subpackages without duplicating everything?

Leave a Comment

I have the following package structure as a minimal example (for convenience, all is uploaded here):

. ├── sphinx │   ├── build │   ├── Makefile │   └── source │       ├── conf.py │       ├── index.rst │       └── train.rst └── train     ├── __init__.py     └── train.py 

When writing Python packages, one must specifiy the __all__ constant in the __init__.py of any package in order for Sphinx to be able to map a reference such as train.DatasetMeta to train.train.DatasetMeta or similar. However, sphinx-apidoc generates the following sections for these packages:

train package =============  Submodules ----------  train.train module ------------------  .. automodule:: train.train     :members:     :undoc-members:     :show-inheritance:   Module contents ---------------  .. automodule:: train     :members:     :undoc-members:     :show-inheritance: 

Which duplicates the entire documentation as it contains .. automodule:: module.file as well as .. automodule:: module, which refer to the same thing. Removing either of these sections results in undefined reference warnings (turned into errors when using -n to SPHINXOPTS).

sphinx_test/train/train.py:docstring of train.DatasetMeta:1:py:class reference target not found: train.train.DatasetMeta 

How can I solve this?

train/train.py

from collections import namedtuple   class DatasetMeta(namedtuple('DatasetMeta', ['dataset', 'num_classes', 'shape'])):     @property     def size(self):         '''int: Number of examples in the dataset'''         return self.shape[0] 

train/__init__.py

from .train import *  __all__ = ['DatasetMeta'] 

sphinx/source/conf.py

import os import sys sys.path.insert(0, os.path.abspath('.')) sys.path.insert(0, os.path.abspath('../../'))   project = 'test' copyright = '' author = ''  version = '' release = '0'  extensions = [     'sphinx.ext.autodoc', ]  source_suffix = '.rst' master_doc = 'index' 

I just cannot figure out what the logic is here.

1 Answers

Answers 1

One thing we can do to make the situation simpler is a minor rename:

class DatasetMeta(namedtuple('DatasetMetaBase', ['dataset', 'num_classes', 'shape'])): 

which should make it obvious that the missing reference is train.train.DatasetMetaBase when you remove the train.train block from the rst file generated by sphinx-apidoc. The documentation for train.DatasetMeta and train.train.DatasetMeta is going to refer to train.train.DatasetMetaBase; I don't know way to hack around that without patching autodoc or adding your own directives.

From here, I see a few options:

(1) Move DatasetMetaBase to a different module that is not imported in __init__.py. For example

from .abstract import DatasetMetaBase class DatasetMeta(DatasetMetaBase): 

That way the autodoc for DatasetMeta refers to train.abstract.DatasetMetaBase, which should be a unique ref in your case.

(2) Create a separate rst file (say, hidden.rst) that renders the docs for train.train.DatasetMetaBase, but hidden from the main rst.

# hidden.rst .. autodata:: train.train.DatasetMetaBase 

That should be enough to add train.train.DatasetMetaBase to sphinx and resolve the class reference target not found warning.

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment