Category: Airflow | All Categories

Customising Airflow: Beyond Boilerplate Settings

I walk through setting up Apache Airflow to use Dask.distributed, PostgreSQL, logging to AWS S3 as well as create User accounts and Plugins.


Python & Big Data: Airflow & Jupyter Notebook with Hadoop 3, Spark & Presto

A guide to running Airflow and Jupyter Notebook with Hadoop 3, Spark & Presto.


Building a Data Pipeline with Airflow

I walk through setting up a data pipeline for currency exchange rates using Airflow, PostgreSQL and Redis.

Copyright © 2014 - 2017 Mark Litwintschik. This site's template is based off a template by Giulio Fidente.