KubeCon + CloudNativeCon North America 2019

Machine learning workflows within Spotify have been migrated to Kubernetes by adopting Kubeflow and Kubeflow Pipelines. It helps teams increase model development speed and reduce the time to productionize a machine learning model.

In this talk, we will demonstrate some best practices Spotify has learned from managing Kubernetes for backend services and apply them to building a centralized Kubeflow platform. We treat infrastructure as code. We establish customizable and repeatable deployment process. Even with a handful of machine learning/data engineers, we are successfully able to manage multiple Kubernetes clusters and machine learning workloads at scale.

We will also show how teams at Spotify use Kubeflow platform as a one-stop shop for their machine learning development, which helps them build better products to improve user listening experience.