You find Python easier than SQL? User-Defined Functions in PySpark might be what you're looking for. This guide will teach you everything you need to know about UDFs.
Spark SQL - From basics to Regular Expressions and User-Defined Functions (UDF) in 10 minutes - Everything you need to know.
Learn to count words of a book and address the common stop word issue - implemented in PySpark
Spark is based on Resilient Distributed Datasets (RDD) - Make sure you know how to use them
Want to learn Apache Spark for Data Science? This guide will help you get started. Learn how to install PySpark and load your first dataset with Python.
Hardcoding values in your Airflow DAGs is a bad practice. Learn how to use Airflow variables instead.
Learn how to download files from Amazon S3 (AWS) to your local machine with Apache Airflow and Python
Learn how to setup an Amazon S3 (AWS) Bucket and how to upload files from local disk with Apache Airflow.
Learn to work with REST APIs in Apache Airflow by utilizing HttpSensor and HttpOperator Airflow classes.
Learn to send and receive data between Airflow tasks with XComs, and when you shouldn't use it.
%20Explained){kind=link}
