Python for Data and Files: Files, Databases, APIs, and Production Pipelines
Turn Your Python Skills into Practical Data Engineering
Knowing Python fundamentals is one thing. Using Python to process real-world data, connect to databases, retrieve information from APIs, and build reliable pipelines is another.
Python for Data and Files takes the next step. Designed for readers who already understand basic Python, this practical guide shows you how to transform everyday data-handling tasks into structured, repeatable workflows.
You will follow a continuous project built around Verzio, a fictional platform that consolidates information from multiple agencies. What begins as a collection of inconsistent files gradually becomes a production-ready data pipeline capable of collecting, validating, transforming, storing, and reporting on data.
WHAT YOU WILL LEARN
Read, write, organise, and process files and directories.
Work with CSV and Excel files using Python, openpyxl, and pandas.
Read and manage JSON, XML, and configuration files.
Introduce databases with SQLite and query data using SQL.
Connect Python applications to databases with SQLAlchemy.
Retrieve information from REST APIs and handle authentication, pagination, and rate limits.
Clean and transform datasets with pandas.
Validate data quality and enforce data contracts with pandera.
Create charts and reports with matplotlib and openpyxl.
Package and distribute Python projects.
Design ETL pipelines and production-oriented data workflows.
Schedule and orchestrate recurring jobs.
Improve performance through chunked processing, connection pooling, and parallel execution.
Explore the foundations of real-time data processing.
GO BEYOND MANUAL DATA PROCESSING
Real datasets rarely arrive perfectly organised. Files may have inconsistent column names, different formats, missing values, or unexpected records. APIs can fail, databases need careful handling, and recurring workflows must remain reliable when the person who created them is not watching.
This book addresses those challenges through practical implementation. Each chapter contributes another component to the Verzio project, helping you understand how file handling, validation, storage, reporting, and orchestration fit together.
By the end, you will have built a complete pipeline that combines multiple data sources, validates and transforms information, writes results to a database, generates reports, and runs as a scheduled process.
WHO IS THIS BOOK FOR?
Python programmers ready to work with real-world datasets.
Data analysts who want to automate repetitive data preparation.
Developers learning data engineering concepts.
Excel users moving toward programmatic data processing.
Professionals who need more reliable, repeatable reporting workflows.
You should know Python basics, including variables, functions, loops, classes, and exception handling. Previous experience with pandas, SQLAlchemy, or workflow orchestration tools is not required; the book develops these concepts through concrete examples.
BUILD PIPELINES YOU CAN REUSE, TEST, AND TRUST.
If you are ready to move beyond isolated scripts and start building reliable data workflows, Python for Data and Filesgives you the practical skills to take that step.
Book 2 of the Python Groundwork Series by Waskey Press.
Customer Reviews
ThriftBooks ® and the ThriftBooks ® logo are registered trademarks of Thrift Books Global, LLC
