Incremental Loads & High Watermarks

Problem

You are writing an ETL pipeline that syncs billing logs. Since scanning the entire source database every hour is too slow, you want to load only new records. Write a query that outputs strategy with the value: "Watermarking tracks the maximum timestamp or ID from the last successful execution and queries only rows larger than that watermark."

Database Schema

This problem acts as a scratchpad. You will write your own DDL/DML or conceptual queries, so there is no predefined schema.

Try It

Hint

Solution