Incremental Loads (Delta)

Problem

Instead of truncating and reloading the massive 500 million row employees table every night, you want to build a pipeline that only fetches what changed today. Write a query to select all employees hired after '2005-01-01' to simulate an incremental fetch based on a high-water mark.

Database Schema

Table: employees

• employee_id (INT): Unique identifier for the employee.
• first_name (VARCHAR): First name of the employee.
• last_name (VARCHAR): Last name of the employee.
• salary (INT): Employee's salary.
• department_id (INT): Foreign key referencing the department.
• manager_id (INT): Foreign key referencing the manager (another employee).
• hire_date (DATE): Date the employee was hired.

Table: departments

• department_id (INT): Unique identifier for the department.
• department_name (VARCHAR): Name of the department.

Try It

Hint

Solution