Designing a Multitenancy File System for Cloud Environment | Ceph Days NYC 2024

In the dynamic landscape of cloud technology, creating multitenant file systems to meet industry-specific needs presents a unique set of challenges. This talk details a project by 45Drives, aimed at developing a multitenant filesystem for a Fortune 100 client within the Media & Entertainment industry. The project faced considerable design challenges, primarily due to the traditional file systems' lack of inherent multitenancy and the client's requirement for a filesystem workflow accessible via SMB or NFS over object storage.

To overcome these hurdles, the 45Drives team employed RADOS Namespaces & Cephx Keyrings for effective data segregation among tenants in a single cluster, and virtualized NFS/SMB gateways to separate storage protocols for each tenant. The solution was further enhanced by integrating advanced automation and deployment tools, including Proxmox, Ansible, Terraform, and Cloud-init. A notable requirement was the use of Self-Encrypting Drives (SEDs), necessitating the development of custom software to unlock drives on boot, as dmcrypt proved unsuitable from customers perspective.

An unexpected requirement for network micro-segmentation to meet the client's security standards introduced additional complexity towards the project's completion. Nevertheless, the project succeeded in delivering scalable performance results. Future directions include efforts to automate the scaling of Multi Daemon Servers (MDS) and the transition from Virtual Machines on Proxmox to Containers on Kubernetes, aiming to enhance efficiency and flexibility in cloud infrastructure deployments. This presentation will share insights from the experience of navigating the complexities of developing a multitenant filesystem for the Media & Entertainment industry, reflecting on both the achieved outcomes and prospective future improvements in cloud infrastructure.