Building Your Own Utilities

Practical Python Scripting — Building Your Own Utilities
Practical Python Scripting
Course 1 · Chapter 6 · Building Your Own Utilities

🛠️ Building Your Own Utilities

You've learned the core patterns: paths, configuration, databases, and automation. Now it's time to take these patterns and apply them to build reusable utilities for any project. This final chapter teaches best practices for structure, documentation, testing, and sharing your utilities.

📋 The Utility Template

Every utility you build should follow this structure:

my-utility/ ├── .env.example ← Configuration template ├── requirements.txt ← Python dependencies ├── README.md ← Usage instructions ├── utils.py ← Core functionality ├── main.py ← Entry point └── tests/ └── test_utils.py ← Unit tests

✅ Best Practices

1. Separate Concerns

# ❌ BAD: Everything in one file main.py: - Load config - Connect to database - Process files - Insert into database - Display results # ✅ GOOD: Separate modules config.py → Configuration loading db_utils.py → Database operations file_utils.py → File operations main.py → Orchestration

2. Configuration Management

# ✅ GOOD: All config in one place config.py: - Load from .env - Set defaults - Validate - Return structured config # ✅ GOOD: .env.example for documentation .env.example: DB_HOST=localhost DB_USER=your_username DB_PASS=your_password

3. Error Handling

# ✅ GOOD: Graceful failure try: # Main operation except SpecificError as e: print(f"❌ {e}") return False finally: # Cleanup

4. Type Hints

def process_file(filepath: str, config: dict) -> bool: """Process a file and return success status.""" pass def find_files(directory: str) -> list[str]: """Find all files in directory.""" pass

5. Documentation

# README.md should explain: 1. What it does (1 paragraph) 2. Installation (pip install -r requirements.txt) 3. Setup (.env configuration) 4. Usage (how to run it) 5. Examples (real-world usage) 6. Troubleshooting (common issues)

🔄 Adapting Patterns to New Projects

From TypeScript Course to Any Course

Component TypeScript Example Generic Pattern
Find Files glob("ts1-*.html") glob("{prefix}-*.{ext}")
Parse Metadata Extract from <h2> Extract key data from file
Database Entity TypeScript Fundamentals Any course name
Insert Operation insert_chapter_with_content() insert_<item>()

Real-World Examples

Example 1: Import Blog Posts

  • Find: blog-*.md files
  • Parse: Extract title, date, content
  • Database: Insert into posts table

Example 2: Sync User Data

  • Find: users.csv file
  • Parse: Read CSV rows
  • Database: Update users table

Example 3: Process Images

  • Find: *.jpg files in directory
  • Parse: Extract metadata (size, dimensions)
  • Database: Insert into images table

🧪 Testing Your Utility

Unit Tests

import pytest from utils import parse_metadata, find_files def test_parse_metadata(): """Test metadata parsing.""" result = parse_metadata("chapter-1.html") assert result['number'] == 1 assert result['title'] is not None def test_find_files(tmp_path): """Test file discovery.""" # Create test files (tmp_path / "chapter-1.html").touch() (tmp_path / "chapter-2.html").touch() # Test finding them files = find_files(tmp_path) assert len(files) == 2

Integration Tests

def test_full_workflow(tmp_path, test_db): """Test complete workflow.""" # Create test files # Run main workflow # Verify database pass

🎯 Common Utility Patterns

Pattern 1: File Discovery + Database Insert

Find files → Parse → Insert into database

Pattern 2: Data Transformation

Read source → Transform → Write result

Pattern 3: API Sync

Fetch from API → Parse → Store in database

Pattern 4: CSV/Excel Import

Read file → Parse rows → Insert records

Pattern 5: Batch Processing

Find items → Process each → Report results

📝 Documentation Checklist

✅ README.md with overview and usage
✅ .env.example showing all config options
✅ requirements.txt with all dependencies
✅ Type hints on all functions
✅ Docstrings on modules and functions
✅ Error messages that explain what went wrong
✅ Examples of how to use the utility
✅ Troubleshooting section for common issues

💻 Final Challenges

Challenge 1: Create a Blog Post Importer

Build a utility that:

  1. Finds all blog-*.md files
  2. Parses title and date from front matter
  3. Inserts into database
  4. Reports count

Goal: Apply patterns to a new domain (blog posts vs. courses).

→ Solution

Challenge 2: Data Transformation Utility

Build a utility that:

  1. Reads a CSV file
  2. Transforms data (cleanup, validation)
  3. Writes to database
  4. Generates report

Goal: Practice the data transformation pattern.

→ Solution

Challenge 3: Reusable Utility Package

Build a complete utility package with:

  1. Main script with proper structure
  2. Configuration management
  3. Error handling and logging
  4. Unit tests
  5. Complete documentation

Goal: Build a production-ready utility you could share.

→ Solution

🎓 Course Conclusion

🏆 You've Mastered
  • ✅ File system navigation with Path and glob
  • ✅ Configuration management with .env
  • ✅ Database connections and operations
  • ✅ Building reusable utility modules
  • ✅ Complete automation workflows
  • ✅ Best practices for production code

What You Can Build Now

With these patterns, you can build utilities for:

  • 📚 Course content management systems
  • 📝 Blog post importers
  • 📊 Data migration tools
  • 🔄 API sync utilities
  • 📁 Batch file processors
  • 🗄️ Database administration tools
  • 🚀 CI/CD automation scripts

Next Steps

  1. Pick a project: What do you want to automate?
  2. Follow the template: Use the structure from this course
  3. Test thoroughly: Write unit and integration tests
  4. Document well: Help future users understand it
  5. Share it: Put it on GitHub so others can use it
💡 Remember

The patterns you've learned work for any project. The details change (different file types, different databases, different APIs), but the structure stays the same. Master the patterns, adapt them to your needs, and you can build anything!