Packages

Path

When we import a module, Python looks for it in the Python path. This is a series of directories where Python tries to find our files.

We can see what import path Python is using by checking sys.path:

>>> import sys
>>> sys.path
['', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-x86_64-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']

If we remove a directory from sys.path Python will no longer look in it when we try to import. For example, if we remove the empty string from sys.path Python will not look in the current directory when importing:

>>> sys.path.remove('')
>>> import whereami
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'whereami'

What is a Package?

Notice that when we ask the datetime for its file, we get a datetime.py file, but when we ask collections for its file, we get an __init__.py file in a collections directory:

>>> import datetime
>>> datetime.__file__
'/usr/lib/python3.3/datetime.py'
>>> import collections
>>> collections.__file__
'/usr/lib/python3.5/collections/__init__.py'
>>> collections.__path__
['/usr/lib/python3.5/collections']

The collections module is actually a package. A package is a directory that Python treats like it’s a module. Packages can contain other modules. For example, collections contains an abc module:

>>> import collections.abc
>>> collections.abc.__file__
'/usr/lib/python3.5/collections/abc.py'

Making Packages

We can make our own packages by making a directory with an __init__.py file in it. The directory name will be used as the name of the package and the __init__.py file will be used whenever that package name is imported.

Modules and sub-packages of our package must live inside this package directory. This directory must be in our Python path (in the current directory or elsewhere), just like every other module file we want to use.

Let’s make a food_utils package. You can copy-paste this code to create the package structure:

from pathlib import Path
Path("food_utils").mkdir(exist_ok=True)
Path("food_utils/__init__.py").write_text("")
Path("food_utils/fruits.py").write_text(
    'watermelon = "WATERMELON"\n'
    'apple = "APPLE"\n'
)
Path("food_utils/non_fruits.py").write_text(
    'tomato = "TOMATO"\n'
    'cucumber = "CUCUMBER"\n'
)

This will create the following directory structure:

food_utils/
├── __init__.py
├── fruits.py
└── non_fruits.py

The fruits.py file contains:

watermelon = "WATERMELON"
apple = "APPLE"

The non_fruits.py file contains:

tomato = "TOMATO"
cucumber = "CUCUMBER"

And the __init__.py file is empty for now.

Now we can import this package:

>>> import food_utils
>>> food_utils
<module 'food_utils' from '/home/trey/food_utils/__init__.py'>

Notice that when we import food_utils, Python references the __init__.py file.

We can access the submodules using dot notation:

>>> import food_utils.fruits
>>> food_utils.fruits.watermelon
'WATERMELON'
>>> import food_utils.non_fruits
>>> food_utils.non_fruits.tomato
'TOMATO'

Packages are useful for splitting our code into a logical hierarchy. Splitting our code into separate files helps us organize our code into logical groups. Using packages allows us to not only group our code, but make a hierarchy for it.

Don’t get carried away with sub-packages though. Remember “flat is better than nested” from the Zen of Python!

Relative Imports

Let’s say we want to make a food_utils/helpers.py file that imports from fruits:

from fruits import watermelon

If we try to import this new food_utils.helpers module we’ll see an exception:

>>> import food_utils.helpers
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "food_utils/helpers.py", line 1, in <module>
    from fruits import watermelon
ImportError: No module named 'fruits'

In order to fix this we can use an absolute import:

from food_utils.fruits import watermelon

Alternatively we can use a relative import to note that we’re importing a module within the same package:

from .fruits import watermelon