Previous | Tutorial index | Next

Tutorial 5: The Python Standard Library – Common Modules and Their Uses

Learning Objectives

Explain the functionalities of some standard and widely used modules, and use them comfortably in programming.

1. Introduction: The "Batteries Included" Philosophy

Python is famous for its philosophy of being "batteries included." This means that when you install Python, you get a vast, powerful, and comprehensive Standard Library – a collection of over 200 modules that handle everything from file compression to web services, from mathematical computations to system administration.

Why is this important? Because you don't need to reinvent the wheel for common programming tasks. Need to parse a JSON response from an API? There's a module for that. Need to generate a random password? There's a module for that. Need to work with dates and times across time zones? There's a module for that too.

In this tutorial, we will explore 10 of the most widely used standard library modules. Mastering these will dramatically increase your productivity and allow you to write professional-grade Python code without relying on third-party dependencies for basic tasks.

2. Module 1: math – Mathematical Operations

The math module provides access to mathematical functions defined by the C standard. It is the go-to module for pure mathematical computations.

Key Features:

Practical Examples:

import math

# Constants
print(f"π = {math.pi}")                # 3.141592653589793
print(f"e = {math.e}")                 # 2.718281828459045

# Trigonometry (angles in radians)
angle_deg = 45
angle_rad = math.radians(angle_deg)
print(f"sin(45°) = {math.sin(angle_rad):.4f}")  # 0.7071

# Square root and powers
print(f"sqrt(144) = {math.sqrt(144)}")          # 12.0
print(f"2^10 = {math.pow(2, 10)}")              # 1024.0

# Rounding
print(f"ceil(4.2) = {math.ceil(4.2)}")          # 5
print(f"floor(4.9) = {math.floor(4.9)}")        # 4

# Factorials and combinations
print(f"5! = {math.factorial(5)}")              # 120
print(f"C(10, 3) = {math.comb(10, 3)}")         # 120

Important Note: The math module works with floats. If you need arbitrary precision or complex numbers, look at the decimal and cmath modules.

3. Module 2: random – Generating Pseudo-Random Numbers

The random module is used for generating pseudo-random numbers, selecting random elements, and shuffling sequences. It is essential for simulations, games, sampling, and security (though not for cryptographic purposes – use secrets for that).

Key Features:

Practical Examples:

import random

# Reproducibility (useful for debugging)
random.seed(42)
print(random.random())  # Always 0.6394267984578837 with seed 42

# Integers
print(random.randint(1, 6))    # Simulate a die roll
print(random.randrange(0, 100, 10))  # Random multiple of 10

# Sequences
colors = ['red', 'green', 'blue', 'yellow']
print(random.choice(colors))        # Random color
print(random.sample(colors, 2))     # 2 unique colors
print(random.choices(colors, k=5))  # 5 random colors (with repeats)

# Shuffling
cards = list(range(1, 53))
random.shuffle(cards)
print(cards[:5])  # First 5 shuffled cards

Key Point: The randomness is pseudo-random – it's deterministic based on the seed. For true randomness (e.g., for security tokens), use the secrets module instead.

4. Module 3: datetime – Dates and Times

The datetime module provides classes for manipulating dates and times in both simple and complex ways. It handles leap years, time zones, and time arithmetic.

Key Classes:

Key Functions:

Practical Examples:

from datetime import datetime, date, time, timedelta

# Current date and time
now = datetime.now()
print(f"Now: {now}")                      # 2026-08-11 14:30:25.123456
print(f"Date: {now.date()}")              # 2026-08-11
print(f"Time: {now.time()}")              # 14:30:25.123456

# Creating specific dates
birthday = date(1990, 5, 15)
print(birthday)

# Date arithmetic (timedelta)
one_week = timedelta(days=7)
next_week = now + one_week
print(f"Next week: {next_week}")

# Formatting (strftime) - converting datetime to string
formatted = now.strftime("%A, %B %d, %Y at %I:%M %p")
print(formatted)  # Tuesday, August 11, 2026 at 02:30 PM

# Parsing (strptime) - converting string to datetime
date_str = "2025-12-25 18:30:00"
parsed = datetime.strptime(date_str, "%Y-%m-%d %H:%M:%S")
print(parsed)  # 2025-12-25 18:30:00

# Calculating age
birth_date = date(1990, 5, 15)
today = date.today()
age = today.year - birth_date.year - ((today.month, today.day) < (birth_date.month, birth_date.day))
print(f"Age: {age}")

Format Codes Cheat Sheet:

• %Y - Year (4 digits)
• %m - Month (01-12)
• %d - Day (01-31)
• %H - Hour (00-23)
• %I - Hour (01-12)
• %M - Minute (00-59)
• %S - Second (00-59)
• %A - Full weekday name
• %B - Full month name
• %p - AM/PM

5. Module 4: os – Operating System Interface

The os module provides a portable way to use operating system-dependent functionality. It allows you to interact with the file system, environment variables, and processes.

Key Features:

Practical Examples:

import os

# Current directory and listing files
cwd = os.getcwd()
print(f"Current directory: {cwd}")
print(f"Files in current dir: {os.listdir('.')}")

# Creating directories
os.makedirs("my_project/data", exist_ok=True)  # exist_ok prevents error if exists

# Path manipulation (cross-platform safe)
path = os.path.join("my_project", "data", "file.txt")
print(f"Path: {path}")  # On Windows: my_project\data\file.txt; on Linux: my_project/data/file.txt

# Checking if a file or directory exists
if os.path.exists("my_project"):
    print("my_project exists")
    
# Environment variables
home = os.environ.get("HOME")  # or "USERPROFILE" on Windows
print(f"Home directory: {home}")

# Executing a shell command (be careful!)
os.system("echo 'Hello from Python'")

Best Practice: For modern, object-oriented path manipulation, consider using pathlib (covered later in this tutorial) instead of os.path. However, os is still essential for environment variables, process management, and OS-level functions.

6. Module 5: sys – Python Interpreter Interaction

The sys module provides access to variables and functions that interact with the Python interpreter itself. It's crucial for command-line scripting, debugging, and runtime configuration.

Key Features:

Practical Examples:

import sys

# Getting Python version
print(f"Python version: {sys.version}")
print(f"Platform: {sys.platform}")

# Command-line arguments (save this as script.py and run: python script.py arg1 arg2)
if len(sys.argv) > 1:
    print(f"Arguments received: {sys.argv[1:]}")
else:
    print("No arguments provided.")

# Module search path
print(f"First search path: {sys.path[0]}")  # Usually the current directory

# Exiting with an error code
def validate_input(value):
    if value < 0:
        print("Error: Value cannot be negative.")
        sys.exit(1)  # Exit with error code 1

# Memory usage
my_list = [1, 2, 3, 4, 5]
print(f"Size of list: {sys.getsizeof(my_list)} bytes")

Use Case: sys.argv is the foundation of building command-line tools in Python. Combine it with argparse for more sophisticated argument parsing.

7. Module 6: json – Working with JSON Data

JSON (JavaScript Object Notation) is the most ubiquitous data interchange format on the web. The json module allows you to convert Python objects to JSON strings and vice versa.

Key Functions:

Type Mapping (Python ↔ JSON):

Practical Examples:

import json

# Python dictionary to JSON string
data = {
    "name": "Alice",
    "age": 30,
    "is_student": False,
    "courses": ["Math", "Physics"],
    "address": {
        "city": "New York",
        "zip": 10001
    }
}

json_string = json.dumps(data, indent=4)  # indent for pretty printing
print(json_string)

# JSON string to Python dictionary
json_input = '{"name": "Bob", "age": 25, "active": true, "score": null}'
parsed_data = json.loads(json_input)
print(parsed_data["name"])        # Bob
print(parsed_data.get("score"))   # None

# Reading and writing JSON files
with open("data.json", "w") as f:
    json.dump(data, f, indent=4)

with open("data.json", "r") as f:
    loaded_data = json.load(f)
    print(loaded_data["name"])    # Alice

Common Pitfall: JSON keys must be strings. Python dictionaries with non-string keys will raise a TypeError when serialized. Use the default parameter to handle custom objects.

8. Module 7: csv – Reading and Writing CSV Files

CSV (Comma-Separated Values) is a simple file format used for spreadsheets and data exchange. The csv module provides functionality to read and write CSV files easily.

Key Features:

Practical Examples:

import csv

# Writing to a CSV file
data = [
    ["Name", "Age", "City"],
    ["Alice", 30, "New York"],
    ["Bob", 25, "Los Angeles"],
    ["Charlie", 35, "Chicago"]
]

with open("people.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerows(data)  # Write all rows at once

# Reading from a CSV file
with open("people.csv", "r") as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)  # Each row is a list

# Using DictReader (assumes first row is header)
with open("people.csv", "r") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(f"{row['Name']} is {row['Age']} years old and lives in {row['City']}.")

# Writing with DictWriter
data_dicts = [
    {"Name": "David", "Age": "40", "City": "Boston"},
    {"Name": "Eve", "Age": "28", "City": "Seattle"}
]

with open("people_dict.csv", "w", newline="") as f:
    fieldnames = ["Name", "Age", "City"]
    writer = csv.DictWriter(f, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerows(data_dicts)

Important: Always use newline="" when opening CSV files to prevent issues with line endings on different operating systems.

9. Module 8: re – Regular Expressions (Pattern Matching)

The re module provides regular expression matching operations. Regular expressions are a powerful language for pattern matching in text. They are essential for searching, validating, and manipulating strings.

Key Functions:

Common Regex Patterns (Metacharacters):

Practical Examples:

import re

text = "My email is alice@example.com and my phone is 123-456-7890."

# Searching for a pattern
email_pattern = r"\w+@\w+\.\w+"  # Simple email pattern
match = re.search(email_pattern, text)
if match:
    print(f"Found email: {match.group()}")

# Find all matches
phone_pattern = r"\d{3}-\d{3}-\d{4}"
phones = re.findall(phone_pattern, text)
print(f"Phone numbers: {phones}")  # ['123-456-7890']

# Replacing text
new_text = re.sub(phone_pattern, "[REDACTED]", text)
print(new_text)  # My email is alice@example.com and my phone is [REDACTED].

# Compiling for performance
pattern = re.compile(r"\b[A-Z][a-z]+\b")  # Capitalized words
cap_words = pattern.findall("Hello World! This is Python.")
print(cap_words)  # ['Hello', 'World', 'This', 'Python']

# Splitting strings
parts = re.split(r"\s+", "Split   this  text    by spaces")
print(parts)  # ['Split', 'this', 'text', 'by', 'spaces']

# Validation
def is_valid_email(email):
    pattern = r"^[\w\.-]+@[\w\.-]+\.\w+$"  # More robust pattern
    return re.match(pattern, email) is not None

print(is_valid_email("test@example.com"))   # True
print(is_valid_email("invalid-email"))      # False

Pro Tip: Regular expressions can get complex. Use online tools like regex101.com to test and debug your patterns. Remember that re is powerful but can be slow for massive text processing; for simple string operations, str methods are often sufficient.

10. Module 9: collections – Specialized Container Data Types

The collections module provides alternatives to Python's built-in containers (list, dict, set, tuple) with specialized functionality.

Key Data Types:

Practical Examples:

from collections import Counter, defaultdict, deque, namedtuple

# 1. Counter - Counting elements
text = "mississippi"
count = Counter(text)
print(count)                     # Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})
print(count.most_common(2))      # [('i', 4), ('s', 4)]

# 2. defaultdict - Auto-handle missing keys
my_dict = defaultdict(list)      # Default value is an empty list
my_dict['fruits'].append('apple')
my_dict['fruits'].append('banana')
my_dict['vegetables'].append('carrot')
print(dict(my_dict))             # {'fruits': ['apple', 'banana'], 'vegetables': ['carrot']}

# 3. deque - Fast queues and stacks
queue = deque(['a', 'b', 'c'])
queue.append('d')
queue.appendleft('z')
print(queue)                     # deque(['z', 'a', 'b', 'c', 'd'])
print(queue.pop())               # 'd'
print(queue.popleft())           # 'z'

# 4. namedtuple - Lightweight immutable data objects
Point = namedtuple('Point', ['x', 'y'])
p1 = Point(10, 20)
print(p1.x, p1.y)                # 10 20
# Access by index as well (like a tuple)
print(p1[0])                     # 10

11. Module 10: pathlib – Modern Path Manipulation

The pathlib module offers an object-oriented approach to file system paths. It is designed to be more intuitive and less error-prone than os.path. Introduced in Python 3.4, it is now the recommended way to handle paths.

Key Features:

Practical Examples:

from pathlib import Path

# Creating paths (cross-platform safe)
home = Path.home()
project_dir = home / "my_project" / "data"
file_path = project_dir / "config.json"
print(file_path)  # /home/user/my_project/data/config.json (Linux) or C:\Users\User\my_project\data\config.json (Windows)

# Checking existence and creating directories
if not project_dir.exists():
    project_dir.mkdir(parents=True)  # Creates all intermediate directories

# Reading and writing files
file_path.write_text('{"key": "value"}')
content = file_path.read_text()
print(content)  # {"key": "value"}

# Working with directory contents
for item in Path.home().iterdir():
    if item.is_file():
        print(f"File: {item.name}")
    elif item.is_dir():
        print(f"Directory: {item.name}")

# Accessing path components
p = Path("/home/user/docs/report.pdf")
print(p.name)          # report.pdf
print(p.stem)          # report
print(p.suffix)        # .pdf
print(p.parent)        # /home/user/docs
print(p.parent.parent) # /home/user

# Glob pattern matching (advanced)
for py_file in Path.home().glob("**/*.py"):  # Recursive search for .py files
    print(py_file)

Why Use pathlib? It's more readable, more concise, and works uniformly across Windows, Linux, and macOS without needing os.path.join or conditionals. It is now the preferred standard library module for path operations.

12. Quiz: Check Your Understanding

1. Which module would you use to generate a random integer between 1 and 100?

• (A) math
• (B) random
• (C) sys
• (D) os

2. You have a string "2025-12-25 10:30:00". Which function correctly parses this into a datetime object?

• (A) datetime.strftime("2025-12-25 10:30:00", "%Y-%m-%d %H:%M:%S")
• (B) datetime.strptime("2025-12-25 10:30:00", "%Y-%m-%d %H:%M:%S")
• (C) datetime.parse("2025-12-25 10:30:00")
• (D) datetime.convert("2025-12-25 10:30:00")

3. Which module would you use to get the command-line arguments passed to your Python script?

• (A) os
• (B) argparse (this is valid, but from the standard library list, sys is the direct answer)
• (C) sys
• (D) pathlib

4. What does the json.dumps() function do?

• (A) Converts a JSON string to a Python dictionary.
• (B) Converts a Python dictionary to a JSON string.
• (C) Writes a Python object to a JSON file.
• (D) Reads a JSON file into a Python object.

5. Which collections data type is specifically designed for fast appends and pops at both ends of a sequence?

• (A) Counter
• (B) defaultdict
• (C) deque
• (D) namedtuple

6. Which module provides an object-oriented and cross-platform way to handle file paths?

• (A) os
• (B) os.path
• (C) pathlib
• (D) fileio

7. True or False: The re module is used for mathematical regular expressions (like solving equations).

8. You need to count the frequency of each word in a large text. Which collections class is most suitable?

• (A) deque
• (B) namedtuple
• (C) Counter
• (D) defaultdict

5.13 Exercises

Exercise 1: Random Password Generator

Goal: Generate a strong random password of a given length.

Instructions:

1. Import random and string.
2. Define a function generate_password(length=12) that:
   • Creates a string of allowed characters: uppercase, lowercase, digits, and punctuation (string.ascii_letters + string.digits + string.punctuation).
   • Uses random.choices() to select length random characters.
   • Joins them into a single string and returns it.
3. Generate and print three passwords of length 16.

import random
import string

def generate_password(length=12):
    chars = string.ascii_letters + string.digits + string.punctuation
    return ''.join(random.choices(chars, k=length))

for i in range(3):
    print(f"Password {i+1}: {generate_password(16)}")

Exercise 2: File System Explorer

Goal: Write a script that lists all files and directories in a given path using both os and pathlib (for comparison).

Instructions:

1. Import os and pathlib.
2. Ask the user to input a directory path (use input()).
3. Using os.listdir() and os.path.isdir()/os.path.isfile(), print all items, categorizing them as [DIR] or [FILE].
4. Using pathlib.Path(path).iterdir() and checking .is_dir() and .is_file(), do the same thing.
5. Print the total number of items found.

import os
from pathlib import Path

path = input("Enter directory path: ")

# Using os
print("Using os:")
os_items = os.listdir(path)
for item in os_items:
    full = os.path.join(path, item)
    if os.path.isdir(full):
        print(f"[DIR] {item}")
    else:
        print(f"[FILE] {item}")
print(f"Total: {len(os_items)}")

# Using pathlib
print("\nUsing pathlib:")
p = Path(path)
items = list(p.iterdir())
for item in items:
    if item.is_dir():
        print(f"[DIR] {item.name}")
    else:
        print(f"[FILE] {item.name}")
print(f"Total: {len(items)}")

Exercise 3: JSON Configuration Reader/Writer

Goal: Create a simple configuration manager that loads settings from a JSON file and allows updating them.

Instructions:

1. Create a JSON file config.json with the following content:

   {
       "username": "admin",
       "theme": "dark",
       "notifications": true,
       "language": "en"
   }
   

2. Write a Python script that:
   • Loads the JSON file using json.load().
   • Prints all current settings.
   • Asks the user for a new theme (e.g., "light" or "dark").
   • Updates the theme key with the user's input.
   • Saves the updated dictionary back to the JSON file using json.dump() with indent=4.
3. Reload the file and print the updated settings to confirm.

import json

# Load
with open("config.json", "r") as f:
    config = json.load(f)
print("Current config:", config)

# Update
new_theme = input("Enter new theme (light/dark): ")
config["theme"] = new_theme

# Save
with open("config.json", "w") as f:
    json.dump(config, f, indent=4)

# Reload and verify
with open("config.json", "r") as f:
    updated = json.load(f)
print("Updated config:", updated)

Exercise 4: Log File Analyzer with Counter

Goal: Analyze a server log file and count the frequency of different status codes.

Instructions:

1. Create a text file server.log with the following lines (simplified log format):

   192.168.1.1 - GET /index 200
   10.0.0.1 - POST /login 401
   192.168.1.1 - GET /about 200
   10.0.0.2 - GET /contact 404
   192.168.1.1 - POST /data 500
   10.0.0.1 - GET /home 200
   10.0.0.2 - GET /index 200
   

2. Write a Python script that:
   • Reads the file line by line.
   • Extracts the last token (the status code) from each line using line.split()[-1].
   • Uses collections.Counter to count the frequency of each status code.
   • Prints the counts in descending order using .most_common().

from collections import Counter

status_codes = []
with open("server.log", "r") as f:
    for line in f:
        parts = line.split()
        if parts:
            status = parts[-1]
            status_codes.append(status)

counter = Counter(status_codes)
for code, count in counter.most_common():
    print(f"{code}: {count}")

Exercise 5: Regex Email Validator

Goal: Build a function that validates email addresses using regular expressions.

Instructions:

1. Import the re module.
2. Define a function validate_email(email) that:
   • Uses a regular expression pattern to check if the email is valid. The pattern should ensure:
     • It starts with alphanumeric characters, dots, hyphens, or underscores.
     • Followed by an @ symbol.
     • Followed by a domain (alphanumeric, dots, hyphens).
     • Followed by a dot and a top-level domain (2-6 letters).
   • Returns True if valid, False otherwise.
3. Test your function with at least 5 valid and 5 invalid email addresses.
4. Bonus: Use re.findall() to extract all valid emails from a block of text.

import re

def validate_email(email):
    pattern = r"^[\w\.-]+@[\w\.-]+\.\w{2,6}$"
    return re.match(pattern, email) is not None

emails = [
    "user@example.com",
    "first.last@domain.co.uk",
    "invalid-email",
    "missing@dotcom",
    "user@.com",
    "user@domain.c"
]
for e in emails:
    print(f"{e}: {validate_email(e)}")

# Bonus extraction
text = "Contact us at support@company.com or sales@company.net for more info."
valid_emails = re.findall(r"[\w\.-]+@[\w\.-]+\.\w{2,6}", text)
print(valid_emails)

5.14 Common Pitfalls and Troubleshooting

5.15 Homework Questions

Answer the following questions in complete sentences. Each response should be 3–5 sentences unless otherwise specified.

Short Answer Questions

1. Why is the Python standard library often described as "batteries included"? Give two examples of modules that demonstrate this philosophy.

2. What is the difference between random.random() and random.randint(a, b)?

3. Explain the purpose of the if __name__ == "__main__": guard when using modules. How does it relate to the sys module?

4. How does pathlib improve upon os.path for path manipulation? Give an example.

5. When would you use re.compile() instead of calling re.search() directly?

Essay Questions

Answer the following questions in 300–500 words each.

6. Compare and contrast the os module and the pathlib module for file system operations. Discuss their respective strengths and weaknesses, and explain when you would choose one over the other.

Suggested outline:

• Introduction: Briefly describe both modules.
• Historical context: os was the traditional way; pathlib was introduced in Python 3.4.
• os: Strengths – familiar to many, extensive low‑level functions (e.g., os.environ, os.system). Weaknesses – verbose, uses strings for paths, lacks object‑oriented design.
• pathlib: Strengths – more readable, object‑oriented, cross‑platform, many methods (.read_text, .mkdir). Weaknesses – still relatively new, some functionality (like os.environ) is not covered.
• Recommendations: Use pathlib for most path‑related tasks; use os when you need process management, environment variables, or low‑level OS calls.
• Conclusion: Both have their place, but pathlib is the modern recommended approach for file system paths.

7. Discuss the importance of regular expressions in text processing. Provide two real‑world scenarios where re would be the ideal tool, and explain the potential pitfalls of using regular expressions for complex tasks.

Suggested outline:

• Introduction: Define regular expressions and their power.
• Scenario 1: Email validation – pattern matching ensures format correctness.
• Scenario 2: Log file parsing – extracting IP addresses, status codes, etc.
• Pitfalls: Regex can become unreadable and hard to maintain; over‑use for tasks that simple string methods can handle; performance issues on large texts.
• Conclusion: Use re wisely, but know its limitations.

Research Questions

These questions require additional research beyond the tutorial content.

8. Research the difference between datetime and pytz (or zoneinfo in Python 3.9+). Why is timezone handling important, and how does the standard library address it?

9. Investigate the subprocess module. How does it improve upon os.system()? Provide an example of using subprocess to capture the output of a system command.

16. Summary of Tutorial 5

• The Python Standard Library is vast and "batteries included," providing solutions for many common programming tasks without requiring third-party installs.
• math provides mathematical functions and constants for scientific and numerical work.
• random generates pseudo-random numbers and supports random selections and shuffling.
• datetime handles dates, times, and time intervals with powerful parsing and formatting capabilities.
• os interfaces with the operating system for file/directory operations and environment variables.
• sys provides access to interpreter-level information, command-line arguments, and runtime control.
• json enables seamless serialization and deserialization of JSON data, essential for web APIs.
• csv simplifies reading and writing tabular data in spreadsheet formats.
• re offers regular expression support for advanced text pattern matching and manipulation.
• collections provides specialized container data types like Counter, defaultdict, and deque for common data structures.
• pathlib is the modern, object-oriented, and cross-platform way to handle file system paths.

This concludes the Tutorial 5. Once you have completed the exercises and homework, you will be comfortable with the most essential modules in the Python Standard Library. You are now ready to move on to Tutorial 6, where we will learn how to write and use your own custom modules!

Previous | Tutorial index | Next