Advanced Data Engineering
Development Academy of the Philippines
Enrollment is Closed
Course Overview
This course deals with the tools available in the big data ecosystem for extraction, transformation, and loading (ETL), data pipelines, and data storage. This enables participants to use more sophisticated ways to deal with data that has outgrown custom-made Python and SQL scripts. Using videos and hands-on exercises, participants will be exposed to different methods for managing and streaming data, in preparation for business intelligence reporting, analytics, and modeling.
What You Will Learn
Upon completion of this course, the learners are expected to:
- identify the different popular ETL frameworks and how they are used;
- recognize the advantages and disadvantages of using different types of storage and database solutions;
- identify the key differences of relational databases and the data warehouses;
- design a normalized data model based on a business scenario;
- transform normalized models into dimensional models; and
- design a data warehouse based on a business setting.
Course Requirements
You will need a computer or laptop with the following tools:
- Any open-source RDBMS
- Any visual diagramming tool (e.g., MS Visio, lucidchart)
- Jupyter Notebook/Spyder
- Any spreadsheet application
Note to SPARTA scholars: Upon enrollment, you will have 6 months to finish a SPARTA course. Failure to complete the course in 6 months and/or inactivity for 3 months will result in course access revocation.
Course Instructors
Raymond Freth Lagria
Subject Matter Expert
Simon Lorenzo
Subject Matter Expert
Pierre Allan Villena
Subject Matter Expert
Course Content
Week 1 Recipes for Data Analysis in SQL
7 Videos | 5 Activities
7 Videos
- Welcome to the course!
- Introduction to Databases and Database Management Software
- Data Models and the Entity Relationship Diagram
- Creating an Entity Relationship Diagram
- Entity Relationship Diagram to the Relational Model
- Relational Databases and Anomalies
- Normalization in Databases
5 Activities
- Decoding Data Engineering: Word Grid and Brainstorming
- Pop Quiz: Multiple Choice and Drag and Drop
- Peer Graded Assignment: Entity Relationship Diagram
Week 2 Review: SQL Fundamentals
7 Videos | 4 Activities
7 Videos
- CREATE Statement and Data Types
- Keys in SQL and Relating Relations
- INSERT INTO Statement
- SELECT Statement
- UPDATE Statement
- DELETE Statement
- JOINS Statement
4 Activities
- Exercise: Missing Word and Labelled Diagram
- Exercise: Classifying Data Types
- Peer Graded Assignment: Normalizing Data
Week 3 Dimensional Modeling and Differences between OLTP and OLAP Databases
12 Videos | 5 Activities
12 Videos
- Introduction to Data Warehousing
- Data Warehousing Components
- Introduction to the Dimensional Model
- Designing the Dimensional Model
- Dimension Tables
- Dimension Table 1 Types 1
- Dimension Table 1 Types 2
- Dimensional Normal Form
- Fact Tables and Granularity
- Fact Table Contents
- From Normalized to Dimensional Model
- A Star Schema in an RDBMS
5 Activities
- Pop Quiz: Fact or Bluff and Labelled Diagram
- Mind Check: Steps Arrangement and Hungman
- Quiz: Dimensional Modeling
Week 4 From OLTP to OLAP Non-RDBMS Sources
9 Videos |4 Activities
9 Videos
- What is ETL?
- Extract and Transform Subsystems
- Loading Subsystems
- Managing Subsystems
- Notes on Transformation
- Design and Development
- The ETL Roadmap for Facts
- Introduction to Non-RDBMS Sources
- Extracting and Manipulating Data from Non-RDBMS Sources
4 Activities
- Decoding ETL: Anagram
- Recall: Steps Arrangement and Multiple Choice
- Peer-Graded Assignment: SQL Scripts and Coding in Python
Week 5 SQL and ETL Challenges
3 Videos | 1 Activity
3 Videos
- SQL and ETL Challenges
- Review on Feature Engineering
- What is Big Data?
1 Activity
- Peer Graded Assignment: SQL and ETL Challenges
Week 6 Capstone Project
1 Video | 1 Activity
1 Video
- Key Takeaways
1 Activity
- Capstone Project