I have found a number of posts about setting up Sphinx and autodoc for python projects, notably this excellent post. However, despite all these resources I still spent a couple of hours running into edge cases with ReadTheDocs that were not covered in these other resources. So, to save you all (and my future self) some time I am writing all this down in one post.

Our example project directory

For this post I will create an example project directory called sphinx-ex

The directory structure of this project is as follows:

sphinx-ex
      |
      /sphinxex
            |
            /core
               |
                mymodule.py
      /tests
      README.md

We have a top-level project directory sphinxex, our python package directory sphinxex, and a directory to hold our modules core, and another directory holding our tests tests.

Install sphinx

You can install sphinx using pip

pip install sphinx

Setting up the Sphinx scaffolding

Create a docs directory

mkdir docs

Create the sphinx scaffolding

We can use sphinx-quickstart to create some basic sphinx scaffolding to use for our project. We need to run it inside our docs directory.

cd docs
sphinx-quickstart

Sphinx will prompt you with a number of interactive inputs. In general you can accept the defaults of no for this simple project with ONE EXCEPTION. We want to enable to autodoc extension to be able to use sphinx-apidoc to create documentation from our docstrings.

autodoc: automatically insert docstrings from modules (y/n) [n]: y

Our docs directory now looks like the following:

jarales:docs jmills$ ls
Makefile	_static		conf.py
_build		_templates	index.rst

Examine sphinx-quickstart artifacts

sphinx-quickstart has created a number of directories and files, and I will briefly discuss two important ones. The _build directory contains stubs used by sphinx-autodoc to create the api documentation, and The conf.py contains configuration options. For this simple case we can leave the stubs in the _build directory alone, but there are a few options we need to change in the conf.py file.

Run sphinx-apidoc to autodoc our code Now that we have our scaffolding in place we will run the sphinx-apidoc utility to create stubs for autodoc. sphinx-apidoc takes two option arguments for our purposes, the first is the output directory, we will set it to source for this case, and the second is the package directory sphinxex in this case. We need to run this command in our docs directory.

sphinx-apidoc -o source/ ../sphinxex

Configuring our scaffolding and creating the html documentation

Here will will fill out the scaffolding and create our html documentation locally. In the next section we will go over the last steps needed to make the documentation autogenerate on ReadTheDocs.

Edit the conf.py file The first thing we need to do is uncomment lines 15-17 and add an additional line.

conf.py

import os
import sys
sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath('../'))

As noted at the start of this post, we have a few peculiarities about our project that we need to deal with. First is that we are using google-style docstrings. We need to add an extension to allow sphinx to parse these strings. We do this by adding the ‘sphinx.ext.napoleon’ to the list of extensions on line ~79 and editing some options below.

extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.napoleon'
]
napoleon_google_docstring = True
napoleon_use_param = False
napoleon_use_ivar = True

However, we have another issue with our docstrings, we are using type hints in the definitions. For example,

def foo(bar: str):
    print(bar)

This issue needs to be addressed on the ReadTheDocs side, which I will cover later.

An optional edit is to set the default theme in sphinx to that of ReadTheDocs. We do this by editing the html_theme option on line ~79.

html_theme = 'sphinx_rtd_theme'

Generate our documentation locally We are now ready to generate our documentation locally. We will do this locally first to make sure sphinx is configured properly and then we will make the necessary changes to make it work with the ReadTheDocs service.

make html

You should now have new directories and files in your docs/_build directory. The html directory contains the documentation html files.

cd _build/html
ls
_sources		index.html		search.html
_static			objects.inv		searchindex.js
genindex.html		py-modindex.html	source

Dealing with ReadTheDocs issues

As noted at the top of this blog, we have two main issues with our package that affect ReadTheDocs. The first is that we import a module only available on conda-forge. The second is that we use type hints in our definitions. There are easy workarounds for these if you know what to do.

conda-forge packages

To use conda-forge packages we need to tell ReadTheDocs to use a conda environment for the build. We also need to specify that environment.

Create a readthedocs.yml file ReadTheDocs allows builds to be configured using a readthedocs.yml file. For this simple project the following yml file will suffice.

build:
    image: latest
conda:
  file: doc/environment.yml
python:
   version: 3.6
   setup_py_install: true

We will go through this line-by-line:

image: latest - To use pytohn 3.6 we need to specify the build image as latest. Note Python 3.6 is required by sphinx and ReadTheDocs to use in-line type hints.

file: doc/environment.yml - We specify our conda environment file to use for the conda build.

version: 3.6 - We specify our python version as 3.6

setup_py_install: true - run setup.py install

Create our conda environment file Because we are using conda for our build, we need to specify the conda environment to ReadTheDocs.

For this simple project we only have one dependencies on conda-forge, f90nml.

name: sphinxex-docs
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.6
  - f90nml

First thing to note here is that we specify conda-forge as our primary channel for conda. Next we specify our dependencies as python 3.6 and f90nml. Note We have to specify the python version in the conda environment file as well as our readthedocs.yml file.

Thats it!