Tutorial 7: Creating and Using Your Own Packages

Learning Objectives

Write and use your own modules and packages.

1. Introduction: From Modules to Packages – The Next Level of Organization

In Tutorial 6, you learned how to create individual modules (.py files) to organize your code. But what happens when your project grows to dozens or even hundreds of modules? Throwing them all into one directory becomes chaotic. You need a hierarchical structure to group related modules together.

Packages are the solution. They allow you to organize modules into directories, creating a logical, hierarchical namespace. Think of a package as a folder containing related modules – like having separate folders for "utilities," "database," and "web" in your project. Packages are the standard way to structure large Python projects and libraries.

In this tutorial, you will learn how to create your own packages, leverage __init__.py for powerful initialization and API design, understand the nuances of relative versus absolute imports, and finally, learn how to distribute your packages or install third-party packages using pip.

2. Recap: Module vs. Package

Before diving in, let's reinforce the distinction:

Feature	Module	Package
File System	A single `.py` file.	A directory (folder).
Required File	Only the `.py` file itself.	Must contain an `__init__.py` file (for regular packages).
Purpose	Groups related functions, classes, and variables for a specific task.	Groups related modules and sub-packages for a broader domain.
Namespace Example	`import math` (where `math.py` is the file).	`import xml.etree` (where `xml` is a package, `etree` is a sub-package).
Analogy	A single book (e.g., "Math Handbook").	A bookshelf or library section containing many books.

3. Creating a Package – Step by Step

Step 1: Create the Directory

Create a new directory for your package. The name of this directory will be the name of your package. Use snake_case (e.g., my_package, data_utils).

Step 2: Add `init.py`

Inside the directory, create an empty file named __init__.py. This file is required for Python to recognize the directory as a package. It can be empty, but it serves as the package's "entry point."

Step 3: Add Your Modules

Add your .py module files inside the directory.

Example Structure

Let's create a package called shapes:

shapes/
├── __init__.py          # Marks this as a package
├── circle.py            # Module for circle calculations
└── rectangle.py         # Module for rectangle calculations

Step 4: Write the Module Code

shapes/circle.py:

# shapes/circle.py
import math

def area(radius):
    """Calculate the area of a circle."""
    return math.pi * radius ** 2

def circumference(radius):
    """Calculate the circumference of a circle."""
    return 2 * math.pi * radius

shapes/rectangle.py:

# shapes/rectangle.py
def area(length, width):
    """Calculate the area of a rectangle."""
    return length * width

def perimeter(length, width):
    """Calculate the perimeter of a rectangle."""
    return 2 * (length + width)

Step 5: Import and Use the Package

Now, from a script in the parent directory of shapes/, you can import and use the package:

# main.py (in the same directory as the 'shapes' folder)
import shapes.circle
import shapes.rectangle

print(shapes.circle.area(5))           # 78.53981633974483
print(shapes.rectangle.area(4, 6))     # 24

Or, using from:

from shapes import circle, rectangle
print(circle.area(5))
print(rectangle.area(4, 6))

4. The Power of `init.py` – Beyond Just Marking a Package

The __init__.py file is executed automatically when the package or any module inside it is imported. It is not just a marker; it's a powerful tool for package design.

4.1. Package Initialization

You can place initialization code in __init__.py that runs once when the package is first imported. This is useful for:

Setting up package-level configuration (e.g., loading environment variables).
Connecting to a database or external service.
Initializing logging.
Checking system dependencies.

Example:

# shapes/__init__.py
print("Initializing the shapes package...")

# Package-level configuration
DEBUG = True
VERSION = "1.0.0"

# Logging setup (if needed)
import logging
logging.basicConfig(level=logging.INFO)

4.2. Simplifying the Public API

One of the most powerful uses of __init__.py is to pre-import key functions from submodules into the package namespace. This allows users to call functions directly from the package level, without needing to know the internal module structure.

Before (without pre-imports):

import shapes.circle
import shapes.rectangle

area_circle = shapes.circle.area(5)
area_rect = shapes.rectangle.area(4, 6)

After (with pre-imports in __init__.py):

# shapes/__init__.py
from .circle import area as circle_area
from .rectangle import area as rect_area

Now the user can write:

import shapes
print(shapes.circle_area(5))    # Clean and intuitive!
print(shapes.rect_area(4, 6))

Note: The . before circle means "from the same package/current directory." This is a relative import (discussed in detail later).

4.3. Controlling Wildcard Imports with `all`

When a user writes from shapes import *, Python checks the __all__ variable in __init__.py to determine which modules to import. If __all__ is not defined, from shapes import * imports nothing (only the package itself).

Example:

# shapes/__init__.py
__all__ = ['circle', 'rectangle']  # Only these modules will be imported with *

Now, from shapes import * imports the circle and rectangle modules. This gives you fine-grained control over your package's public interface.

4.4. Combining All Three Techniques

A well-designed __init__.py often combines all three techniques:

# shapes/__init__.py

# 1. Documentation
"""
shapes – A package for calculating properties of geometric shapes.

This package provides modules for circles, rectangles, and more.
"""

# 2. Initialization
print("Initializing shapes package...")
VERSION = "1.0.0"

# 3. Simplify API (pre-import key functions)
from .circle import area as circle_area, circumference as circle_circ
from .rectangle import area as rect_area, perimeter as rect_perimeter

# 4. Control wildcard imports
__all__ = ['circle_area', 'circle_circ', 'rect_area', 'rect_perimeter']

Now the user can use the package elegantly:

import shapes
print(shapes.circle_area(5))       # 78.53981633974483
print(shapes.rect_area(4, 6))      # 24
print(shapes.__version__)          # "1.0.0"

5. Importing from a Package

There are several ways to import from a package:

5.1. Importing the Entire Module

import shapes.circle
import shapes.rectangle

shapes.circle.area(5)
shapes.rectangle.area(4, 6)

5.2. Importing Specific Modules

from shapes import circle, rectangle
circle.area(5)
rectangle.area(4, 6)

5.3. Importing Specific Functions (if exposed via `init.py`)

from shapes import circle_area, rect_area
circle_area(5)
rect_area(4, 6)

5.4. Importing Everything (Using `*`)

from shapes import *   # Only what's in __all__ is imported
circle.area(5)         # Works if 'circle' is in __all__

5.5. Importing Nested Sub-Packages

If you have nested packages (e.g., shapes/3d/sphere.py):

from shapes.3d import sphere
sphere.volume(5)

6. Relative vs. Absolute Imports

When importing modules within the same package, you have two options: absolute imports and relative imports.

6.1. Absolute Imports

An absolute import specifies the full path from the top-level package.

Example:

# Inside shapes/circle.py
from shapes.rectangle import perimeter   # Absolute import

Pros: Explicit, clear, works from anywhere. Cons: Verbose, breaks if the package is renamed.

6.2. Relative Imports

A relative import uses dots (.) to refer to the current and parent packages.

Syntax	Meaning
`.`	The current package.
`..`	The parent package.
`...`	The grandparent package, etc.

Example:

# Inside shapes/circle.py
from .rectangle import perimeter   # Relative import (from the same package)

Pros: Concise, portable (no need to repeat package names). Cons: Cannot be used in scripts run directly (only inside packages).

Important: Relative imports only work inside packages where the __name__ is not "__main__". If you try to run circle.py directly (python circle.py), a relative import will raise an ImportError because the interpreter doesn't know the package context.

6.3. Which One Should You Use?

Use absolute imports in most cases – they are explicit and clearer, especially for larger projects.
Use relative imports for convenience within a package, especially to shorten long paths.

Best Practice: Many Python projects use absolute imports exclusively for clarity and to avoid confusion.

7. Installing External Packages with `pip`

While you are learning to create your own packages, you will also frequently need to use packages created by others. The standard tool for installing third-party Python packages is pip.

7.1. Basic `pip` Commands

Command	Description
`pip install package_name`	Installs the latest version of a package.
`pip install package_name==1.2.3`	Installs a specific version.
`pip install package_name>=1.2.0`	Installs a minimum version.
`pip uninstall package_name`	Removes a package.
`pip list`	Lists all installed packages.
`pip show package_name`	Shows detailed information about a package.
`pip freeze > requirements.txt`	Saves a list of all installed packages to a file.
`pip install -r requirements.txt`	Installs packages from a requirements file.

7.2. Example: Installing `requests`

The requests package is a popular library for making HTTP requests.

pip install requests

Then, in your Python script:

import requests
response = requests.get("https://api.github.com")
print(response.status_code)

7.3. Virtual Environments (Best Practice)

It's strongly recommended to use virtual environments to isolate package dependencies for different projects. This prevents conflicts between projects (e.g., Project A needs django==2.0, Project B needs django==3.0).

# Create a virtual environment
python -m venv myenv

# Activate it
# On Windows:
myenv\Scripts\activate
# On Mac/Linux:
source myenv/bin/activate

# Install packages
pip install requests numpy pandas

8. Distributing Your Own Package (Brief Overview)

Once you have created a package, you might want to share it with the world. Python has a central repository called the Python Package Index (PyPI) where you can upload your packages so others can install them with pip.

8.1. Minimal Structure for Distribution

To prepare a package for distribution, you typically add additional files:

shapes/
├── shapes/                 # Your package code
│   ├── __init__.py
│   ├── circle.py
│   └── rectangle.py
├── setup.py                # Package metadata (name, version, author, etc.)
├── README.html               # Description of your package
└── LICENSE.txt             # License information

8.2. Minimal `setup.py` Example

# setup.py
from setuptools import setup, find_packages

setup(
    name="shapes-package",
    version="1.0.0",
    author="Your Name",
    description="A simple package for shape calculations.",
    packages=find_packages(),
    install_requires=[],  # Dependencies, e.g., "numpy>=1.18.0"
)

8.3. Installation in Development Mode

While developing, you can install your package in "editable" mode, so changes are reflected immediately:

pip install -e .

This makes your package importable from anywhere on your system, just like any installed package.

Note: Publishing packages to PyPI is an advanced topic. For now, focus on creating well-structured packages for your own projects.

9. Quiz: Check Your Understanding

1. What file is required to mark a directory as a Python package?

(A) main.py
(B) __init__.py
(C) setup.py
(D) package.json

Answer

(B) `__init__.py`

2. How do you make a function circle_area from circle.py available as shapes.circle_area() when someone imports the shapes package?

(A) Rename the function to circle_area.
(B) Add from .circle import area as circle_area to __init__.py.
(C) Add a file called __main__.py.
(D) Use __all__ = ['circle_area'] in circle.py.

Answer

(B) Add `from .circle import area as circle_area` to `__init__.py`.

3. Which of the following is a relative import?

(A) import shapes.circle
(B) from shapes import circle
(C) from .circle import area
(D) from shapes.circle import area

Answer

4. What does pip freeze do?

(A) Installs all packages in a requirements file.
(B) Lists all installed packages and their versions.
(C) Removes all installed packages.
(D) Shows the current Python version.

Answer

(B) Lists all installed packages and their versions.

5. What is the correct way to install a package named requests using pip?

(A) pip install requests
(B) pip download requests
(C) install requests
(D) pip get requests

Answer

(A) `pip install requests`

6. True or False: The __init__.py file is executed every time you import any module from the package.

Answer

False. It runs **once** on the first import from the package, then cached.

7. If you define __all__ = ['circle'] in __init__.py, what happens when a user runs from shapes import *?

(A) All modules are imported.
(B) Only the circle module is imported.
(C) A NameError is raised.
(D) Nothing is imported because * is ignored.

Answer

(B) Only the `circle` module is imported (as specified in `__all__`).

8. Can relative imports be used in a script that is run directly (e.g., python circle.py)?

(A) Yes, always.
(B) No, they will raise an ImportError.
(C) Yes, but only if the script is in the current directory.
(D) Only on Linux.

Answer

(B) No, they will raise an `ImportError` because the package context is unknown.

7.10 Exercises

Exercise 1: Create a Simple `shapes` Package

Goal: Follow the instructions from the core content and create a fully functional shapes package.

Instructions:

Create a folder named shapes.
Inside it, create __init__.py, circle.py, and rectangle.py with the code provided above.
In __init__.py, import circle and rectangle so that users can use them directly.
Add __all__ to control wildcard imports.
Create a main.py script in the parent directory that imports and uses the package in all three ways:
- import shapes.circle
- from shapes import circle, rectangle
- from shapes import * (if you set __all__ correctly)

Sample Solution

Refer to the code provided in the tutorial sections; the final __init__.py and main.py are shown in the appendix of the tutorial.

Exercise 2: Add a New Module to the Package

Goal: Extend the shapes package with a new module for triangles.

Instructions:

Create shapes/triangle.py with functions:
- area(base, height) – returns 0.5 * base * height.
- perimeter(side1, side2, side3) – returns the sum of the three sides.
In __init__.py, import and expose area as triangle_area and perimeter as triangle_perimeter.
Update __all__ to include these new names.
Test your package from a script by calculating the area of a triangle with base 10 and height 5.

Sample Solution

triangle.py:

def area(base, height):
    return 0.5 * base * height

def perimeter(s1, s2, s3):
    return s1 + s2 + s3

Update in init.py:

from .triangle import area as triangle_area, perimeter as triangle_perimeter
__all__ = ['circle_area', 'circle_circ', 'rect_area', 'rect_perimeter', 'triangle_area', 'triangle_perimeter']

Exercise 3: Nested Sub-Package

Goal: Add a sub-package for 3D shapes.

Instructions:

Create a subdirectory shapes/3d/.
Inside shapes/3d/, add __init__.py and sphere.py with a function volume(radius) (return 4/3 * math.pi * radius**3).
In shapes/3d/__init__.py, import volume from sphere and expose it as sphere_volume.
From a script, import shapes.3d and calculate the volume of a sphere with radius 3.

Sample Solution

shapes/3d/sphere.py:

import math
def volume(radius):
    return 4/3 * math.pi * radius**3

shapes/3d/init.py:

from .sphere import volume as sphere_volume

Usage:

from shapes import sphere_volume
print(sphere_volume(3))

Exercise 4: Using `pip` to Install a Package

Goal: Practice using pip to install and use a third-party package.

Instructions:

Install the requests package: pip install requests.
Write a simple script that uses requests.get() to fetch the HTML of https://www.python.org and prints the status code.
If you have time, install a virtual environment and repeat the installation inside it.

Sample Solution

import requests
response = requests.get("https://www.python.org")
print(f"Status code: {response.status_code}")

7.11 Common Pitfalls and Troubleshooting

Problem	Likely Cause	Solution
`ImportError: No module named 'my_package'`	The package is not in the current directory or `sys.path`.	Ensure you are in the parent directory of the package. Add the path to `sys.path` if needed.
`ImportError: attempted relative import with no known parent package`	You tried to use a relative import in a script that is run directly (not inside a package).	Use absolute imports, or restructure so the script is part of a package and run with `python -m package.module`.
`ModuleNotFoundError: No module named 'shapes.circle'`	The module file does not exist, or `__init__.py` is missing.	Check the file structure and ensure `circle.py` is in the `shapes/` directory with `__init__.py`.
The `__init__.py` code is not running.	Python may be using a cached version of the package.	Delete the `__pycache__` folder and restart the interpreter.
`pip install` fails with permission errors.	You don't have write permissions to the global site-packages.	Use a virtual environment, or run `pip install --user package_name` to install only for your user.
The package installs but cannot be imported.	The package name might not match the import name (e.g., you installed `shapes-package` but imported `shapes`).	Check the package's documentation for the correct import name. Use `pip show package_name` to see metadata.
Relative imports work in one place but fail in another.	You may have a different entry point for your script (e.g., running from outside the package).	Standardize on absolute imports for clarity and reliability.

7.12 Homework Questions

Answer the following questions in complete sentences. Each response should be 3–5 sentences unless otherwise specified.

Short Answer Questions

1. What is the role of __init__.py in a Python package? Give three distinct uses.

Sample Answer

The `__init__.py` file marks a directory as a Python package. It can run initialization code when the package is imported, define `__all__` to control wildcard imports, and pre‑import key functions from submodules to simplify the public API.

2. Explain the difference between absolute and relative imports within a package. When would you prefer one over the other?

Sample Answer

Absolute imports specify the full path from the top‑level package (e.g., `from package.sub import module`). Relative imports use dots to refer to the current or parent package (e.g., `from . import module`). Absolute imports are more explicit and portable, while relative imports are concise but only work inside a package and break when the script is run directly.

3. What is a virtual environment, and why is it recommended when using pip to install packages?

Sample Answer

A virtual environment is an isolated Python environment with its own site‑packages directory, allowing you to install packages without affecting the global installation. This prevents version conflicts between projects and ensures reproducibility.

4. How does the __all__ variable affect the behavior of from package import *?

Sample Answer

`__all__` is a list of module names that will be imported when `from package import *` is used. It restricts the import to only those names, preventing accidental exposure of internal modules and giving control over the package's public interface.

Essay Questions

Answer the following questions in 300–500 words each.

5. Discuss the design decisions involved in creating a Python package, including directory structure, the use of __init__.py, and the choice between absolute and relative imports. How do these decisions affect the usability and maintainability of the package?

Suggested outline:

Introduction: Importance of package design.
Directory structure: Organizing modules logically.
__init__.py: Initialization, API simplification, __all__.
Imports: Absolute vs. relative – clarity vs. convenience.
User experience: How these decisions affect how others use the package.
Maintainability: Clear structure makes updates easier.
Conclusion: Good design is key to a successful package.

6. Compare and contrast the process of creating a single module versus creating a package. When would you choose one over the other? Provide examples of projects that would benefit from each approach.

Suggested outline:

Introduction: Define module and package.
Single module: Simple, one‑file utility; easy to share.
Package: Multiple related modules; larger projects.
When to use module: For a single functionality (e.g., a date formatter).
When to use package: For a library with many components (e.g., a web framework).
Benefits of package: Organization, namespace, scalability.
Conclusion: Choose based on complexity and future growth.

Research Questions

These questions require additional research beyond the tutorial content.

7. Research the concept of "namespace packages" in Python. How do they differ from regular packages, and in what scenarios would you use them?

Sample Answer

Namespace packages allow a single package to be split across multiple directories, without requiring `__init__.py` in each. They are useful for large frameworks that can be extended by third‑party plugins, or for distributing a package across multiple repositories.

8. Investigate the structure of a popular Python package (e.g., requests, numpy, flask). Look at its __init__.py and how it exports its API. Write a short summary of what you observe and how it affects the user's experience.

Sample Answer

For example, the `requests` package has a minimal `__init__.py` that imports the main classes/functions from submodules, making them available at the top level. This allows users to write `import requests` and directly call `requests.get()` without needing to import submodules. This design simplifies usage and hides internal complexity.

13. Summary of Tutorial 7

A package is a directory that contains modules and a special __init__.py file.
Creating a package involves: creating a directory, adding __init__.py, and placing .py module files inside.
The __init__.py file serves multiple critical roles:
- Marks the directory as a package.
- Executes initialization code on first import.
- Defines __all__ to control wildcard imports.
- Pre-imports key functions from submodules to simplify the public API.
Importing from a package can be done in several ways: import package.module, from package import module, or from package import * (controlled by __all__).
Relative imports (using . and ..) are used within a package to import other modules in the same or parent packages. They do not work in scripts run directly.
Absolute imports are more explicit and generally preferred for clarity.
The pip tool is used to install, manage, and uninstall third-party packages from PyPI.
Virtual environments isolate package dependencies per project, preventing version conflicts.
A well-designed package provides a clean, intuitive, and documented public API while hiding internal complexity.

Quiz Answers (Check Yourself)

Q1: B) __init__.py
Q2: B) Add from .circle import area as circle_area to __init__.py.
Q3: C) from .circle import area (starts with a dot).
Q4: B) Lists all installed packages and their versions.
Q5: A) pip install requests
Q6: False. It runs once on the first import from the package, then cached.
Q7: B) Only the circle module is imported (as specified in __all__).
Q8: B) No, they will raise an ImportError because the package context is unknown.

This concludes the expanded Tutorial 7 and the entire series on Python modules and packages. You now have a comprehensive understanding of how to organize, import, create, and distribute reusable Python code. Congratulations on completing this journey!

Appendix: Complete Package Code Example

For reference, here is the complete shapes package as described in the tutorial:

shapes/__init__.py:

"""
shapes – A package for calculating properties of geometric shapes.

This package provides modules for circles, rectangles, and more.
"""

# Package initialization
print("Initializing shapes package...")
VERSION = "1.0.0"

# Simplify API: pre-import key functions
from .circle import area as circle_area
from .circle import circumference as circle_circ
from .rectangle import area as rect_area
from .rectangle import perimeter as rect_perimeter

# Control wildcard imports
__all__ = ['circle_area', 'circle_circ', 'rect_area', 'rect_perimeter']

shapes/circle.py:

"""Module for circle calculations."""
import math

def area(radius):
    """Calculate the area of a circle."""
    return math.pi * radius ** 2

def circumference(radius):
    """Calculate the circumference of a circle."""
    return 2 * math.pi * radius

shapes/rectangle.py:

"""Module for rectangle calculations."""

def area(length, width):
    """Calculate the area of a rectangle."""
    return length * width

def perimeter(length, width):
    """Calculate the perimeter of a rectangle."""
    return 2 * (length + width)

main.py (demonstration script):

# main.py (in the parent directory)
import shapes

print(f"Using shapes package version {shapes.VERSION}")
print(f"Circle area (r=5): {shapes.circle_area(5)}")
print(f"Rectangle area (4x6): {shapes.rect_area(4, 6)}")
print(f"Rectangle perimeter (4x6): {shapes.rect_perimeter(4, 6)}")

Previous | Tutorial index | Next

Tutorial 7: Creating and Using Your Own Packages

Learning Objectives

1. Introduction: From Modules to Packages – The Next Level of Organization

2. Recap: Module vs. Package

3. Creating a Package – Step by Step

Step 1: Create the Directory

Step 2: Add __init__.py

Step 3: Add Your Modules

Example Structure

Step 4: Write the Module Code

Step 5: Import and Use the Package

4. The Power of __init__.py – Beyond Just Marking a Package

4.1. Package Initialization

4.2. Simplifying the Public API

4.3. Controlling Wildcard Imports with __all__

4.4. Combining All Three Techniques

5. Importing from a Package

5.1. Importing the Entire Module

5.2. Importing Specific Modules

5.3. Importing Specific Functions (if exposed via __init__.py)

5.4. Importing Everything (Using *)

5.5. Importing Nested Sub-Packages

6. Relative vs. Absolute Imports

6.1. Absolute Imports

6.2. Relative Imports

6.3. Which One Should You Use?

7. Installing External Packages with pip

7.1. Basic pip Commands

7.2. Example: Installing requests

7.3. Virtual Environments (Best Practice)

8. Distributing Your Own Package (Brief Overview)

8.1. Minimal Structure for Distribution

8.2. Minimal setup.py Example

8.3. Installation in Development Mode

9. Quiz: Check Your Understanding

7.10 Exercises

Exercise 1: Create a Simple shapes Package

Exercise 2: Add a New Module to the Package

Exercise 3: Nested Sub-Package

Exercise 4: Using pip to Install a Package

7.11 Common Pitfalls and Troubleshooting

7.12 Homework Questions

Short Answer Questions

Essay Questions

Research Questions

13. Summary of Tutorial 7

Quiz Answers (Check Yourself)

Appendix: Complete Package Code Example

Step 2: Add `init.py`

4. The Power of `init.py` – Beyond Just Marking a Package

4.3. Controlling Wildcard Imports with `all`

5.3. Importing Specific Functions (if exposed via `init.py`)

5.4. Importing Everything (Using `*`)

7. Installing External Packages with `pip`

7.1. Basic `pip` Commands

7.2. Example: Installing `requests`

8.2. Minimal `setup.py` Example

Exercise 1: Create a Simple `shapes` Package

Exercise 4: Using `pip` to Install a Package