Skip to main content

Advanced Data Engineering

Development Academy of the Philippines
Enrollment is Closed

Course Overview

This course deals with the tools available in the big data ecosystem for extraction, transformation, and loading (ETL), data pipelines, and data storage. This enables participants to use more sophisticated ways to deal with data that has outgrown custom-made Python and SQL scripts. Using videos and hands-on exercises, participants will be exposed to different methods for managing and streaming data, in preparation for business intelligence reporting, analytics, and modeling.

What You Will Learn

Upon completion of this course, the learners are expected to:

  • identify the different popular ETL frameworks and how they are used;
  • recognize the advantages and disadvantages of using different types of storage and database solutions;
  • identify the key differences of relational databases and the data warehouses;
  • design a normalized data model based on a business scenario;
  • transform normalized models into dimensional models; and
  • design a data warehouse based on a business setting.

Course Requirements

You will need a computer or laptop with the following tools:

  • Any open-source RDBMS
  • Any visual diagramming tool (e.g., MS Visio, lucidchart)
  • Jupyter Notebook/Spyder
  • Any spreadsheet application

Note to SPARTA scholars: Upon enrollment, you will have 6 months to finish a SPARTA course. Failure to complete the course in 6 months and/or inactivity for 3 months will result in course access revocation.

Course Instructors

Course Staff Image #1

Raymond Freth Lagria

Subject Matter Expert

Course Staff Image #2

Simon Lorenzo

Subject Matter Expert

Course Staff Image #3

Pierre Allan Villena

Subject Matter Expert

Course Content

Week 1 Recipes for Data Analysis in SQL

7 Videos | 5 Activities

7 Videos

  • Welcome to the course!
  • Introduction to Databases and Database Management Software
  • Data Models and the Entity Relationship Diagram
  • Creating an Entity Relationship Diagram
  • Entity Relationship Diagram to the Relational Model
  • Relational Databases and Anomalies
  • Normalization in Databases

5 Activities

  • Decoding Data Engineering: Word Grid and Brainstorming
  • Pop Quiz: Multiple Choice and Drag and Drop
  • Peer Graded Assignment: Entity Relationship Diagram

Week 2 Review: SQL Fundamentals

7 Videos | 4 Activities

7 Videos

  • CREATE Statement and Data Types
  • Keys in SQL and Relating Relations
  • INSERT INTO Statement
  • SELECT Statement
  • UPDATE Statement
  • DELETE Statement
  • JOINS Statement

4 Activities

  • Exercise: Missing Word and Labelled Diagram
  • Exercise: Classifying Data Types
  • Peer Graded Assignment: Normalizing Data

Week 3 Dimensional Modeling and Differences between OLTP and OLAP Databases

12 Videos | 5 Activities

12 Videos

  • Introduction to Data Warehousing
  • Data Warehousing Components
  • Introduction to the Dimensional Model
  • Designing the Dimensional Model
  • Dimension Tables
  • Dimension Table 1 Types 1
  • Dimension Table 1 Types 2
  • Dimensional Normal Form
  • Fact Tables and Granularity
  • Fact Table Contents
  • From Normalized to Dimensional Model
  • A Star Schema in an RDBMS

5 Activities

  • Pop Quiz: Fact or Bluff and Labelled Diagram
  • Mind Check: Steps Arrangement and Hungman
  • Quiz: Dimensional Modeling

Week 4 From OLTP to OLAP Non-RDBMS Sources

9 Videos |4 Activities

9 Videos

  • What is ETL?
  • Extract and Transform Subsystems
  • Loading Subsystems
  • Managing Subsystems
  • Notes on Transformation
  • Design and Development
  • The ETL Roadmap for Facts
  • Introduction to Non-RDBMS Sources
  • Extracting and Manipulating Data from Non-RDBMS Sources

4 Activities

  • Decoding ETL: Anagram
  • Recall: Steps Arrangement and Multiple Choice
  • Peer-Graded Assignment: SQL Scripts and Coding in Python

Week 5 SQL and ETL Challenges

3 Videos | 1 Activity

3 Videos

  • SQL and ETL Challenges
  • Review on Feature Engineering
  • What is Big Data?

1 Activity

  • Peer Graded Assignment: SQL and ETL Challenges

Week 6 Capstone Project

1 Video | 1 Activity

1 Video

  • Key Takeaways

1 Activity

  • Capstone Project

  1. Course Number

  2. Classes Start

  3. Classes End

  4. Estimated Effort

    1-2 hours/week (12 hours)
  5. Price