Advancing Spark - Building Delta Live Table Frameworks

In previous videos we've worked with Delta Live Tables to make repeatable, reusable templates, and that's cool... but we had to create pipelines for each table we wanted to process, and that's no good when we're working at scale.

Thanks to some handy pointers from the Databricks team, we can now encapsulate our template inside iterative functions, meaning we can create a whole bunch of delta live tables, with dependencies & logic, all within a single notebook.

In this video Simon takes an example templated delta live table notebook and builds list-driven workflow that generates multiple-stage live tables all within a single pipeline.

As always, don't forget to hit like & subscribe, and get in touch if Advancing Analytics can help you achieve your Data Lakehouse objectives!