Master Azure Databricks | Udemy

Module 1 :

• What is Data Pipeline
• What is Azure databricks
• Azure Databricks Architecture
• Azure Account Setup
• WorkSpace Setup

Module 2:

• Navigate the Workspace
• Runtimes
• Clusters
• Notebooks
• Libraries
• Repos
• Databricks File System (DBFS)
• DBUTILS
• Widgets
• Workflows
• Metastore – Setup external Metastore

Module 3 :

• What is RDD
• Creating RDD
• RDD transformations
• RDD Actions
• RDD Joins
• Pair RDD
• Broadcast Variables
• Accumulators
• Convert RDD to DataFrame
• Import & Read data
• Create a table using the UI
• Create a table in a Notebook

Module 4 :

• Create DataFrames
• Define Schema
• Functions
• Casting Operations
• Filter Transformation
• Update, Update ALL & UpdateByName
• OrderBy & SortBY
• GroupBy
• Remove Duplicates
• Window Functions
• Date and Timestamp Functions
• UDF (User Defined Function)
• JOIN
• Handle corrupt records using the badRecordsPath
• File metadata column

Module 5 :

• Read Parquet File
• Read CSV Files
• Read JSON Files
• Read XML Files
• Read Excel file
• SQL databases using JDBC
• Azure blob storage

Module 6 :

• What is Spark Structure Streaming
• Data Source & Sink
• Rate & File Source
• Kafka Source
• Sink : Console, Memory, File & Custom
• Build Streaming ETL
• Stream ETL 1 : Setup Event Hub
• Streaming ETL 2 : Event Hub Producer
• Streaming ETL 3 : Integrate Event Hubs with Data Bricks
• Streaming ETL 4 : Transformation
• Streaming ETL 5 : Ingest into Azure Data storage
• Twitter Sentiment Analysis – Introduction
• Setup Twitter Developer Account
• Twitter Sentiment Analysis – II
• Twitter Sentiment Analysis – III

Module 7 :

• Components in Databricks SQL
• Configuring a SQL Endpoint
• Creating a Table from a CSV File
• Create Queries
• Parameterized Query
• Query Profile
• Building Visualization (Table, BAR & PIE )
• Building Line Chart & Counter Chart
• Adding Charts to Dashboards
• Defining a Query Alert
• Access Control on Databricks SQL Objects
• Lab: Data Object Access Control
• Transfer Ownership
• Access SQL Endpoint from Python
• Databricks SQL CLI
• Databricks SQL CLI