English | MP4 | AVC 1280×720 | AAC 44KHz 2ch | 7.5 Hours | 3.48 GB
Now with Spark 3.0: Learn practical Big Data with Spark DataFrames, Datasets, RDDs and Spark SQL, hands on, in Scala
UPDATED FOR SPARK 3.0
In this course, we will learn how to write big data applications with Apache Spark 3 and Scala. You’ll write 2000+ lines of Spark code yourself, with guidance, and you will become a rockstar.
This course is for Scala programmers who are getting started with Apache Spark and big data. The course is not for advanced Spark engineers.
Why Spark in Scala:
- it’s blazing fast for big data
- its demand has exploded
- it’s a highly marketable skill
- it’s well maintained, with dozens of high-quality extensions
- it’s a foundation for a data scientist
I like to get to the point and get things done. This course
- deconstructs all concepts into the critical pieces you need
- selects the most important ideas and separates them into what’s simple but critical and what’s powerful
- sequences ideas in a way that “clicks” and makes sense throughout the process of learning
- applies everything in live code
The end benefits are still much greater:
- a completely new mental model around data processing
- significantly more marketable resume
- more enjoyable work – Spark is fun!
This course is for established programmers with experience with Scala and with functional programming at the level of the Rock the JVM Scala beginners course. I already assume a solid understanding of general programming fundamentals.
This course is NOT for you if
- you’ve never written Scala code before
- you don’t have some essential parallel programming background (e.g. what’s a process/a thread)
The course is comprehensive, but you’ll always see me get straight to the point. So make sure you have a good level of focus and commitment to become a badass programmer.
I believe both theory and practice are important. That’s why you’ll get lectures with code examples, real life code demos and assignments, plus additional resources, instructions, exercises and solutions. At the end of the course, you’ll have written thousands of lines of Spark.
I’ve seen that my students are most successful – and my best students work at Google-class companies – when they’re guided, but not being told what to do. I have exercises waiting for you, where I offer my (opinionated) guidance but otherwise freedom to experiment and improve upon your code.
Definitely not least, my students are most successful when they have fun along the way!
What you’ll learn
- apply Spark big data principles
- practice Spark DataFrames operations with 100+ examples and exercises
- practice type-safe data processing with Spark Datasets
- work with low-level Spark APIs with RDDs
- use Spark SQL for data processing
- migrate data from various data sources, including databases
2 How to Make the Best out of This Course
3 Scala Recap
4 Spark First Principles
Spark Structured API DataFrames
5 DataFrames Basics
6 DataFrames Basics Exercises
7 How DataFrames Work
8 Data Sources
9 Data Sources, Part 2 + Exercises
10 DataFrames Columns & Expressions
11 Columns & Expressions Exercise
12 DataFrame Aggregations
13 DataFrame Joins
14 DataFrame Joins Exercise
Spark Types and Datasets
15 Working with Common Spark Data Types
16 Working with Complex Spark Data Types
17 Managing Nulls in Data
18 Type-Safe Data Processing Datasets
19 Datasets, Part 2 + Exercise
20 Spark as a Database with Spark SQL Shell
21 Spark SQL
22 Spark SQL Exercises
24 RDDs, Part 2 + Exercise
25 You Rock!