Previous | Tutorial index | Next

Tutorial 2: Understanding Packages – Organizing Modules into Directories

Learning Objectives

Explain what packages are, what files are required, and how they are structured in a file system.

1. Introduction: From Toolboxes to Workshops

In Tutorial 1, you learned that a module (a single .py file) is like a toolbox – it groups related functions, classes, and variables. But what happens when your project grows to hundreds of toolboxes? You wouldn't throw all your tools into one giant box; you would organize them into categories (e.g., "Electrical Tools", "Plumbing Tools", "Carpentry Tools") and place them on different shelves or in different rooms.

In Python, packages are the "rooms" or "shelves" that hold your modules. They allow you to structure your code hierarchically, making large projects manageable, logically organized, and easy to navigate. Packages are essential for distributing libraries to other developers.

2. What Exactly is a Python Package?

Technical Definition: A package is a directory (folder) that contains a special file named __init__.py and one or more module files (.py files). It is a way of structuring Python's module namespace by using "dotted module names".

For example, the module name A.B designates a submodule named B in a package named A. This hierarchical naming prevents collisions. If two different packages have a module named utils, you can distinguish them via package1.utils and package2.utils.

The Core Purpose of Packages

• Group related modules (e.g., all database-related modules go into a database/ package).
• Provide a hierarchical namespace (e.g., company.hr.employee vs company.finance.account).
• Enable large-scale code distribution (most third-party libraries you install via pip are packages).

3. Module vs. Package: A Clear Comparison

It is crucial to distinguish these two fundamental concepts. Let's break them down side-by-side:

4. The Crucial __init__.py File

The __init__.py file is the most important distinguishing feature of a package. It is executed automatically when the package (or any module inside it) is imported for the first time.

4.1. Marking the Directory as a Package

Before Python 3.3, an __init__.py file was strictly required for a directory to be recognized as a package. Without it, Python would raise a ImportError and treat the directory as just a normal folder. While Python 3.3+ introduced "namespace packages" which don't require this file, it is highly recommended and considered best practice to always include __init__.py for explicit, regular packages. It ensures backward compatibility and provides a clear signal that this folder is part of your code base.

4.2. Executing Package Initialization Code

You can place any Python code inside __init__.py. This code runs the very first time you import anything from the package. It is often used to:

• Set up package-level configuration (e.g., logging, loading environment variables).
• Connect to databases or external APIs at the package level.
• Run checks to ensure the system meets the package's requirements.

Example (shapes/__init__.py):

# shapes/__init__.py
print("Initializing the 'shapes' package...")

# Setup a global package-level variable
__version__ = "1.0.0"

# Code to run automatically on import
import os
if not os.path.exists("data"):
    print("Warning: 'data' folder not found.")

4.3. Defining __all__ for Wildcard Imports

When a user types from shapes import *, Python looks for a list named __all__ inside the __init__.py file to determine which modules should be imported. If __all__ is not defined, from package import * imports nothing (except the package itself) – it does not automatically load all submodules.

Example:

# shapes/__init__.py
__all__ = ['circle', 'rectangle', 'triangle']  # Only these modules will be imported on `from shapes import *`

4.4. Simplifying the API Interface

You can pre-import key functions or classes from submodules inside __init__.py so that users can access them directly from the package level, without having to drill down into submodules.

Instead of:

from shapes.circle import area
from shapes.rectangle import area

You can do this in __init__.py:

# shapes/__init__.py
from .circle import area as circle_area
from .rectangle import area as rect_area

Now the user can simply do:

import shapes
shapes.circle_area(5)   # Clean and simple!

5. Nested Packages (Hierarchical Structure)

Packages can contain sub-packages to create a deep, hierarchical tree. This is extremely useful for large frameworks (e.g., Django or Flask) where features are categorized into several layers.

Example File System Tree:

company/                      # Top-level package
├── __init__.py               # company package init
├── hr/                       # hr sub-package
│   ├── __init__.py           # hr package init
│   ├── employee.py           # Module for employee management
│   └── recruitment.py        # Module for hiring processes
├── finance/                  # finance sub-package
│   ├── __init__.py           # finance package init
│   ├── accounting.py         # Module for accounting
│   └── payroll.py            # Module for payroll processing
└── it/                       # it sub-package
    ├── __init__.py
    ├── hardware.py
    └── software.py

Accessing Nested Modules

To access the payroll module, you use dot notation:

import company.finance.payroll
company.finance.payroll.calculate_salary()

Or, for a cleaner import:

from company.finance import payroll
payroll.calculate_salary()

Or, for a specific function:

from company.finance.payroll import calculate_salary
calculate_salary()

6. How Python Finds Your Package (The Path Mechanism)

Python uses the same sys.path search list to find packages as it does for modules. When you type import company, Python looks for a directory named company that contains an __init__.py file inside each of the directories listed in sys.path.

Important: The parent directory of your package must be in sys.path, not the package directory itself.

• If your package is at /home/user/projects/company/, then /home/user/projects/ must be in sys.path.
• If you are working on a project and your package is in the same folder as your main script, Python will find it automatically (since the current directory is always the first entry in sys.path).

7. Quiz: Check Your Understanding

Test your foundational knowledge before proceeding to the practical exercises.

Q1: Which file is strictly required to mark a standard directory as a Python package?

A) main.py B) __init__.py C) package.json D) __main__.py

Q2: What is the primary difference between a module and a package?

A) A module can contain functions, but a package cannot. B) A module is a single file, while a package is a directory containing modules and an __init__.py file. C) Packages cannot contain sub-packages. D) Modules are built-in, while packages are always third-party.

Q3: If you define __all__ = ['module_a', 'module_b'] in your __init__.py, what happens when a user runs from package import *?

A) Python raises an error. B) Only module_a and module_b are imported. C) All modules inside the package are imported. D) Nothing happens; the * is ignored.

Q4: True or False: Code written inside __init__.py runs every single time you import a module from that package.

Q5: You have a package called animals with a sub-package mammals and a module dog.py. How do you import the bark() function from dog.py using a single line of code, while keeping the import statement concise?

A) import animals.mammals.dog.bark B) from animals.mammals.dog import bark C) from dog import bark D) import animals.mammals.dog

8. Hands-on Exercises (Practical Tasks)

These exercises guide you through creating, structuring, and importing packages on your own machine.

Exercise 1: Building a Basic Package from Scratch

Goal: Create a simple package and verify that Python recognizes it.

Instructions:

1. On your computer, create a new folder called vehicles.

2. Inside the vehicles folder, create an empty file named __init__.py. (On Windows, you can right-click > New > Text Document and rename it carefully; on Mac/Linux, use touch __init__.py).

3. Inside the vehicles folder, create two new Python files: car.py and bike.py.

4. Write the following code inside car.py:

   # car.py
   def start():
       return "Car engine starts with a key."
   
   def honk():
       return "Beep! Beep!"
   

5. Write the following code inside bike.py:

   # bike.py
   def start():
       return "Bike engine starts with a kick."
   
   def ring_bell():
       return "Ring! Ring!"
   

6. Open your terminal, navigate to the parent directory containing the vehicles folder (not inside it).

7. Launch the Python interpreter (python).

8. Type import vehicles. Does it raise an error? (It shouldn't if __init__.py is present).

9. Type import vehicles.car.

10. Type vehicles.car.honk() and observe the output.

11. Type vehicles.bike.start() (Wait! This will raise an AttributeError because you haven't imported bike yet).

12. Type import vehicles.bike and then try vehicles.bike.start() again. It should work now.

Exercise 2: Utilizing __init__.py to Simplify Imports

Goal: Modify __init__.py so that users can call functions directly from the package level.

Instructions:

1. Open the vehicles/__init__.py file in your code editor.

2. Add the following lines to pre-import the start functions:

   # vehicles/__init__.py
   from .car import start as car_start
   from .bike import start as bike_start
   

   (The . before car means "from the same directory/package").

3. Save the file.

4. In the Python interpreter (make sure you restart it, or use import importlib; importlib.reload(vehicles)), try this:

   import vehicles
   print(vehicles.car_start())  # Output: Car engine starts with a key.
   print(vehicles.bike_start()) # Output: Bike engine starts with a kick.
   

   Notice that you didn't have to import car or bike separately! The __init__.py handled it for you, creating a clean, user-friendly interface.

Exercise 3: Creating a Nested Sub-Package

Goal: Add a sub-package to your vehicles package.

Instructions:

1. Inside the vehicles folder, create a new folder named electric.

2. Inside the electric folder, create an empty __init__.py file.

3. Inside the electric folder, create a module named tesla.py with this content:

   # vehicles/electric/tesla.py
   def start():
       return "Tesla starts silently with a push of a button."
   
   def autopilot():
       return "Engaging autopilot mode..."
   

4. In your terminal (Python interpreter), import the nested module:

   from vehicles.electric import tesla
   print(tesla.autopilot())
   

5. Alternatively, import the function directly:

   from vehicles.electric.tesla import start
   print(start())  # Output: Tesla starts silently...
   

9. Common Pitfalls and Troubleshooting

10. Homework Questions (Take-Home Assignment)

These questions require deeper analysis, code writing, and planning.

Homework Question 1: Construct a Music Library Package

Task: You are building a music player application. Create a package called music_library with the following structure:

music_library/
├── __init__.py
├── playlist.py       (contains functions: create_playlist(), add_song(), remove_song())
├── player.py          (contains functions: play(), pause(), skip())
└── metadata.py        (contains a dictionary: SONG_DATA = {"song1": "Artist A", "song2": "Artist B"})

Requirements:

• Write the actual Python code for all three modules (you can use simple print() statements to simulate the functions).
• In music_library/__init__.py, do the following:
  • Define __all__ so that from music_library import * imports playlist and player only (not metadata).
  • Pre-import the play() function from player.py so that a user can call music_library.play() directly.
• Create a separate main.py script in the parent directory that imports the package and demonstrates using all its features.

Submission: Provide the full code for all files.

# playlist.py
def create_playlist(name):
    print(f"Creating playlist: {name}")

def add_song(song, playlist):
    print(f"Adding {song} to {playlist}")

def remove_song(song, playlist):
    print(f"Removing {song} from {playlist}")

# player.py
def play(song):
    print(f"Playing: {song}")

def pause():
    print("Paused")

def skip():
    print("Skipping to next song")

# metadata.py
SONG_DATA = {
    "song1": "Artist A",
    "song2": "Artist B",
    "song3": "Artist C"
}

# __init__.py
from .playlist import create_playlist, add_song, remove_song
from .player import play, pause, skip
from .player import play as play_song  # Pre-import for direct access

__all__ = ['create_playlist', 'add_song', 'remove_song', 'play', 'pause', 'skip']

# main.py
import music_library as ml

ml.create_playlist("My Favorites")
ml.add_song("song1", "My Favorites")
ml.play_song("song1")
ml.pause()
ml.remove_song("song1", "My Favorites")

Homework Question 2: The Magic of __init__.py - Analyzing the Output

Task: Consider the following package structure:

test_pkg/
├── __init__.py
└── module_x.py

__init__.py contains:

print("Initializing test_pkg...")
VERSION = "1.0"

def get_version():
    return VERSION

module_x.py contains:

print("Loading module_x...")
from test_pkg import get_version
print(f"Module X says version: {get_version()}")

Now, a user runs the following script in the parent directory:

import test_pkg
print("-- First import done --")
import test_pkg.module_x
print("-- Second import done --")
import test_pkg.module_x

Assignment:

• Write down, in exact order, the exact output that appears on the console when the script runs.
• Explain why the "Initializing..." and "Loading module_x..." messages appear the number of times they do. Reference the module caching mechanism in your answer.

Initializing test_pkg...
-- First import done --
Loading module_x...
Module X says version: 1.0
-- Second import done --

Homework Question 3: Designing a Project Structure from a Scenario

Task: You are tasked with building a simple E-Commerce system. It will handle:

• User Management: Registration, login, password hashing.
• Inventory: Adding products, updating stock, checking availability.
• Orders: Creating an order, calculating tax, generating an invoice.

Assignment: Design a complete package hierarchy (draw a tree diagram or use text) for this system.

• Your hierarchy must include at least 3 top-level packages or sub-packages, and at least 5 distinct modules.
• For each module, list exactly 2 functions that would logically belong there.
• Briefly defend your design choices: Why did you put certain modules together? How does your structure improve code maintenance for a team of 5 developers?

ecommerce/
├── __init__.py
├── users/
│   ├── __init__.py
│   ├── auth.py          (register_user(), login_user())
│   └── profiles.py      (update_profile(), get_user_history())
├── products/
│   ├── __init__.py
│   ├── catalog.py       (add_product(), list_products())
│   └── inventory.py     (update_stock(), check_availability())
└── orders/
    ├── __init__.py
    ├── checkout.py      (create_order(), calculate_total())
    ├── payment.py       (process_payment(), refund_payment())
    └── invoice.py       (generate_invoice(), send_invoice_email())

Homework Question 4: Debugging a Broken Package

Task: You are given a broken package structure. A developer sends you the following folder:

shopping/
    __init__.py   # Contains: __all__ = ['cart']
    cart.py       # Contains: def total_price(items): return sum(items)
    checkout.py   # Contains: def confirm(): print("Confirmed")

The user tries to run this code and gets errors:

from shopping import *
cart.total_price([10, 20])   # Works fine.
checkout.confirm()            # Raises NameError: name 'checkout' is not defined

Questions:

1. Why does checkout.confirm() raise a NameError even though checkout.py exists in the folder?
2. Fix the issue without changing the user's final line of code (i.e., they must still be able to type checkout.confirm() directly). How would you modify the __init__.py file?
3. If the user instead typed import shopping and then tried shopping.checkout.confirm(), what would need to be changed in the package to make that work?

# shopping/__init__.py
__all__ = ['cart', 'checkout']
from . import cart, checkout

# shopping/__init__.py
from . import cart, checkout

Homework Question 5: Relative vs. Absolute Imports

Task: You have this package:

animals/
├── __init__.py
├── wild.py
└── domestic/
    ├── __init__.py
    ├── dog.py
    └── cat.py

Inside wild.py, you want to import the Dog class from domestic.dog. Inside domestic/dog.py, you want to import a function from wild.py.

Assignment:

1. Write the correct import statement for wild.py to import Dog.
2. Write the correct import statement for domestic/dog.py to import a function named roar() from wild.py.
3. Which import is an "absolute" import, and which is a "relative" import? Explain the difference and when you would prefer one over the other.

from animals.domestic.dog import Dog

from ..wild import roar

This concludes the Tutorial 2. Once you have completed the exercises and homework, you will have a solid practical and theoretical grasp of Python packages. You are now ready to move on to Tutorial 3, where we will learn how to import these packages and modules using various import statements!

Previous | Tutorial index | Next