Loading…
Wednesday, November 20 • 11:50am - 12:25pm
Building and Managing a Centralized Kubeflow Platform at Spotify - Keshi Dai & Ryan Clough, Spotify

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
Machine learning workflows within Spotify have been migrated to Kubernetes by adopting Kubeflow and Kubeflow Pipelines. It helps teams increase model development speed and reduce the time to productionize a machine learning model.

In this talk, we will demonstrate some best practices Spotify has learned from managing Kubernetes for backend services and apply them to building a centralized Kubeflow platform. We treat infrastructure as code. We establish customizable and repeatable deployment process. Even with a handful of machine learning/data engineers, we are successfully able to manage multiple Kubernetes clusters and machine learning workloads at scale.

We will also show how teams at Spotify use Kubeflow platform as a one-stop shop for their machine learning development, which helps them build better products to improve user listening experience.

Speakers
avatar for Keshi Dai

Keshi Dai

Senior ML Infra Engineer, Spotify
Keshi Dai is a Senior Engineer at Spotify who works on Machine Learning platform. His team is building a centralized Kubeflow platform to help Machine Learning engineers at Spotify to adopt Kubernetes. Previously, Keshi also worked on Discover Weekly and Release Radar at Spotify... Read More →
avatar for Ryan Clough

Ryan Clough

Senior ML Engineer, Spotify
Ryan Clough is a Senior Engineer on Spotify's Machine Learning Infrastructure team. Alongside his colleagues, he is responsible for designing and building the platform and tools that ML practitioners all across Spotify use to bring ML solutions from an idea all the way to production... Read More →



Wednesday November 20, 2019 11:50am - 12:25pm
Pacific Ballroom, Salon 20-22 - Marriott Marquis San Diego Marina Hotel