Disaster Recovery Strategies for Structured Streams

In recent years, many businesses have adopted real-time streaming applications to enable faster decision making, quicker predictions, and improved customer experiences. Few of these applications are driving critical business use cases like financial fraud detection, loan application processing, personalized offers, etc. These business critical applications need robust disaster recovery strategies to recover from the catastrophic events to reduce the lost uptime. However, most organizations find it hard to set up disaster recovery for streaming applications as it involves continuous data flow. Streaming state and temporal behavior of data brings add complexities to the DR strategy. A reliable disaster recovery strategy includes backup, failover and failback approaches for the streaming application. Unlike the batch applications, these steps include many moving elements and need a very sophisticated approach to ensure that the services are failing over the DR region and meet the set RTO and RPO requirements.

In this session, we will cover following topics with a FINSERV use case demo:
- Backup strategy: backup of delta tables, message bus services and checkpoint including offsets
- Failover strategy: failover strategy to disable services in the primary region and start the services in the secondary region with minimum data loss
- Failback strategy: failback strategy to restart the services in the primary region once all the services are restored
- Common challenges and best practices for backup

Talk by: Shasidhar Eranti and Sachin Balgonda Patil

Connect with us: Website: https://databricks.com
Twitter:   / databricks  
LinkedIn:   / databricks  
Instagram:   / databricksinc  
Facebook:   / databricksinc