ETL, pipelines, architecture concepts
Build an incremental data loading pipeline in Azure Synapse using the watermark pattern. Copy only changed rows using config tables, delta queries, and stored procedures. Covers ID-based and date-based watermarks with real examples.
Incremental Data Loading in Azure Synapse: The Delta Copy Pattern with Watermark Tables Read More »
Build a metadata-driven pipeline in Azure Synapse Analytics with Parquet output and stored procedure audit logging. Track rows copied, duration, and errors for every table. Step-by-step guide with real-world examples.
Building a Metadata-Driven Pipeline in Azure Synapse Analytics with Audit Logging Read More »
Learn how to build a metadata-driven pipeline in Azure Data Factory using the Lookup, ForEach, and Copy pattern. Step-by-step guide with parameterized datasets, dynamic expressions, troubleshooting tips, and production best practices.
Building a Metadata-Driven Pipeline in Azure Data Factory: The Complete Guide Read More »
