Data Automation (CI/CD) with a Real Life Example

Get my Modern Data Essentials training (for free) & start building more reliable data architectures
https://www.ModernDataCommunity.com

-----

One of the most fun aspects of being a data engineer is creating different automations.

And in particular, one area that's really important is CI/CD , which stands for Continuous Integration and Continuous Deployment.

This is where you can automate your testing and release strategy.

But I also understand that this concept can be a little vague or unclear if you haven't seen it in action.

So in today's video I'll show you a real-life example of how to use Github to make this happen.

This is just one example of why people really like code-based tools because of the ability to automate and do things like this.

This can be applied not only to your deployments but as we'll mostly cover in this video the idea of automating your data quality checks.

Thank you for watching!


Timestamps:
0:00 - Intro
0:43 - Create Workflow File
1:18 - Review File Layout
2:55 - Use Pre-build Actions
4:06 - Trigger Workflow

Title & Tags:
Data Automation (CI/CD) with a Real Life Example
#kahandatasolutions #dataengineering #automation