Secured Kerberos based Spark Notebook for Data Science: Spark Summit East talk by Joy Chakraborty

This presentation will provide technical design and development insights in order to set up a Kerberosied (secured) JupyterHub notebook using Spark. Joy will show how Bloomberg set up the Kerberos-based Spark-notebook-integrating JupyterHub, Sparkmagic, and Levy. Sparkmagic provides the Spark kernel for Scala and Python. Livy is one of the most promising open source software to allow to submit Spark jobs over http-based REST interfaces. In this presentation, Joy will highlight the capabilities of Sparkmagic and Livy, along with the gap or development required in order to integrate the software seamlessly to work with your secured Spark cluster. The Kerberos integration techniques that he’ll discuss can be applied to other types of authenticators, such as OAuth. No prior knowledge of any of these technologies is required in order to understand this presentation.