Modern Databases & Data Pipelines: Hands-On Data Engineering with Kafka, Airflow, and dbt

By: Denning, Hollis

Modern Databases & Data Pipelines: Hands-On Data Engineering with Kafka, Airflow, and dbt

Build, automate, and scale end-to-end data pipelines with the tools defining modern data engineering in 2025.

Databases are no longer static repositories-they are the beating heart of every digital system.
In today's data-driven world, engineers must design ecosystems that move, transform, and validate data continuously across cloud, hybrid, and on-prem environments.
This book takes you beyond theory into the hands-on reality of building, deploying, and operating modern, production-grade data systems.

What You'll LearnMaster the modern database trio: PostgreSQL for OLTP, DuckDB for fast local analytics, and ClickHouse for distributed OLAP queries.Stream data in real time with Apache Kafka - implement change data capture (CDC), manage offsets, and build fault-tolerant ingestion layers.Orchestrate with Apache Airflow - create production DAGs, manage scheduling, automate recovery, and integrate transformations seamlessly.Transform and model data with dbt - apply modular SQL development, Jinja macros, version control, and automated testing.Implement data quality & observability - integrate Great Expectations, Prometheus, and Grafana for continuous validation and live monitoring.Automate deployment using GitHub Actions, Terraform, Docker, and Helm - build reproducible environments and scale effortlessly.Secure and govern your pipeline - enforce encryption, RBAC, audit logs, and compliance-ready lineage tracking.Hands-On Practice Labs

Every chapter includes a fully tested lab with real-world configurations, SQL scripts, and reproducible Docker-based environments.
You'll deploy working pipelines that move data from PostgreSQL → Kafka → Airflow → dbt → ClickHouse → Grafana, complete with monitoring and automated recovery.

Who This Book Is ForData Engineers modernizing legacy ETL into ELT and real-time pipelinesDevelopers & Analysts looking to understand orchestration, automation, and transformation toolsDevOps & Platform Engineers expanding into DataOps, CI/CD, and infrastructure automationStudents and Career Switchers building a portfolio-ready full-stack data projectArchitects & Technical Leads designing end-to-end, resilient data ecosystemsWhy This Book Stands Out

Unlike generic tutorials, this book is purely practical and up-to-date - written for 2025 and beyond.
You won't just read about data pipelines - you'll build them, monitor them, secure them, and automate them using real tools trusted by global data teams.

No screenshots. No fluff. Just code, flowcharts, and structured logic - engineered for professionals who build systems that move data at scale.

By the End of This Book

You'll have deployed a complete, production-style data platform, capable of streaming, transforming, validating, and visualizing data in real time - from raw ingestion to actionable analytics.

Turn your data into a living, automated system. Build the pipelines that power tomorrow's intelligence - today.

• Format: Paperback
• Language: English
• ISBN: B0G1JVVF6G
• ISBN13: 9798273653337
• Release Date: November 2025
• Publisher: Independently Published
• Length: 414 Pages
• Weight: 2.10 lbs.
• Dimensions: 0.8" x 8.5" x 11.0"