Tuesday, November 19 • 2:25pm - 3:00pm
Introducing KFServing: Serverless Model Serving on Kubernetes - Ellis Bigelow, Google & Dan Sun, Bloomberg

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
Production-grade serving of ML models is a challenging task for data scientists. In this talk, we'll discuss how KFServing powers some real-world examples of inference in production at Bloomberg, which supports the business domains of NLP, computer vision, and time-series analysis. KFServing (https://github.com/kubeflow/kfserving) provides a Kubernetes CRD for serving ML models on arbitrary frameworks. It aims to solve 80% of model serving use cases by providing performant, high abstraction interfaces for common ML frameworks. It provides a consistent and richly featured abstraction that supports bleeding-edge serving features like CPU/GPU auto-scaling, scale to and from 0, and canary rollouts. KFServing's charter includes a rich roadmap to fulfill a complete story for mission critical ML, including inference graphs, model explainability, outlier detection, and payload logging.


Dan Sun

Senior Software Engineer, Bloomberg
Dan Sun is a Senior Software Engineer of the Data Science Infrastructure team at Bloomberg, focusing on designing and building mission critical production ML inference managed solution. He strives to understand and tackle data scientists' complex problems. He also has many years of... Read More →

Ellis Bigelow

Software Engineer, Google
Ellis Bigelow is a software engineer at Google Cloud developing next generation systems for the AI Platform Prediction Service. In addition to his efforts on Google's managed product, he leads the open source project, Kubeflow/KFServing, a kubernetes-based serverless inferencing platform... Read More →

Tuesday November 19, 2019 2:25pm - 3:00pm
Room 15AB - San Diego Convention Center Mezzanine Level