How to dynamically create SQL Serverless views from Azure Synapse Pipelines

Azure Synapse SQL Serverless allows you to add structured schemas over standard file formats in your data lake storage. For all the benefits this brings, one of the trade offs is that your schema is only enforced "on read", rather than "on write". So if it's your Synapse Pipelines that are responsible for writing/updating the data that is stored in your data lake, then they should also be responsible for updating the schema of the SQL Serverless views that you use to query it. In this session, James will show a simple way to dynamically update SQL Serverless views from with a Synapse Pipeline so that they're always up to date with the data in your data lake.

#azuresynapse #sqlserverless #datalake #lakehouse #datamesh #dataengineering #datapipeline