How to add Surrogate key in dataframe to uniquely identity the records using Spark-Databricks

How to Create Surrogate key in dataframes
How to Add columns in dataframes
How to filter null values from dataframes
How to compare two dataframes
How to validate data from dataframes
how to compare parquet file with delta table


----------------------------------------------------------------------------------------------------------------------------------------
These videos serve both as a learning tool for myself and as a source of information for others interested in the role and responsibilities of an SDET. While I have done my best to ensure accuracy, I acknowledge that there may be inaccuracies in the information presented. If you notice any mistakes, please feel free to leave a comment and help me improve my understanding
----------------------------------------------------------------------------------------------------------------------------------------
#databricks
#Surrogatekey
#AzureDevOps
#ExecuteTestCases in ADO
#AzureTestPlan
#AzureDevOps
#azuredevops
#dataframes
#dataframes
#surrogatekey
#DevOps
#ADO
#TestCaseExecution
#TestPlan
#TestSuite
#BugLogging in AzureDevOps
#AzureDevOps
#SoftwareTesting
#ExecuteTestCases in ADO
#AzureTestPlan
#AzureDevOps
#Azure
#DevOps
#ADO
#TestCaseExecution
#TestPlan
#TestSuite
#BugLogging in AzureDevOps
#AzureRepos
#AzureDevOpsrepos
#LearnAzure
#AzureLearning
#AzureDevOpsBasics
#LearnAzureDevOps
#LearnDevOps
#AzureRepos
#AzureDevOpsrepos
#LearnAzure
#AzureLearning
#AzureDevOpsBasics
#LearnAzureDevOps
#Learn Azure DevOps
#Azure DevOps Pull changes
#Commit changes in Azure DevOps
#Staged changes in Azure DevOps
#Pull changes in Azure DevOps
#Push changes in Azure DevOps
#Approve changes in Azure DevOps
#Approve PR In Azure DevOps
#ETLTesting
#ETL
#Datatbricks
#ADF
#AzureDataFactory
#DataEngineers
#Data warehouse
#ETLTesting
#DatabricksTesting
#ADFTesting
#dataEngineer
#Deltabricks
#Transformation
#DeltaLake
#DataMesh
#ApprovePRInAzureDevOps