Advanced (Expert level experience) [clear filter]
Tuesday, November 19

10:55am PST

Blazin’ Fast PromQL - Tom Wilkie, Grafana Labs
PromQL, the Prometheus Query Language, is a concise, powerful and increasingly popular language for querying time series data. But PromQL queries can take a long time when they have to consider >100k series and months of data. Even with Prometheus’ compression, a 90 day query over 200k series can touch ~100GB of data.

In this talk we will present a series of techniques employed by Cortex (a CNCF project for clustered Prometheus) for accelerating PromQL queries -- namely query results caching, time slice parallelisation, aggregation sharding and automatic recoding rule substitutions.

But there’s more: we will show how you can use this technology to get these improvements with Thanos and Prometheus. Finally, we will cover optimisations to the PromQL engine by the Cortex team, and how these have already been merged upstream to benefit the whole community.


Tom Wilkie

VP Product, Grafana Labs
Tom is VP Product at Grafana Labs, but really he is a software engineer. Tom is a maintainer on the Prometheus project and a maintainer and the original author of Cortex, both CNCF projects. Previously Tom founded Kausal, a company working on Prometheus, and worked at companies such... Read More →

Tuesday November 19, 2019 10:55am - 11:30am PST
Room 11AB - San Diego Convention Center Upper Level

11:50am PST

CAP_NET_RAW and ARP Spoofing in Your Cluster: It's Going Downhill From Here - Liz Rice, Aqua Security
Did you know that by default, your applications running in Kubernetes can open raw network sockets? This talk demonstrates how, in the right circumstances, the CAP_NET_RAW capability that allows this can be abused by a compromised application.

* ARP spoofing: pretending to represent the wrong IP address
* If the app can ARP spoof the IP address of the DNS service, this potentially lets it spoof DNS addresses: pretending to represent the wrong domain name

Sounds bad, doesn't it?

These attacks, and their consequences, will be demonstrated live, along with preventative measures that you can take to ensure they aren't happening on your cluster.

This talk explains CAP_NET_RAW and spoofing, but the audience is expected to be comfortable with Kubernetes concepts like pod specs and admission controllers.

avatar for Liz Rice

Liz Rice

Chief Open Source Officer, Isovalent
Liz Rice is Chief Open Source Officer with eBPF specialists Isovalent, creators of the Cilium cloud native networking, security and observability project. She is the author of Container Security, and Learning eBPF, both published by O'Reilly, and she sits on the CNCF Governing Board... Read More →

Tuesday November 19, 2019 11:50am - 12:25pm PST
Room 11AB - San Diego Convention Center Upper Level

3:20pm PST

Superpowers for Windows Containers - Deep Debroy & Jean Rouge, Docker
The Windows Operating System does not support privileged operations from inside a container today. Daemon-sets on Windows nodes in Kubernetes clusters that need to perform configuration actions on the node are significantly impacted by the absence of privileged mode support on Windows. In this talk we:
1. Explore the pros and cons of the options the SIG Windows community brainstormed to provide containers running on Windows the ability to perform privileged operations while being managed by Kubernetes.
2. Delve into the specific characteristics of the privileged proxy approach we decided to adopt.
3. Demonstrate how the privileged proxy approach is used to support privileged operations that need to be executed by daemon-sets associated with CSI plugins running on Windows nodes in a Kubernetes cluster.

avatar for Deep Debroy

Deep Debroy

Software Engineering Manager, Docker
Deep Debroy is a software Engineering Manager at Docker Inc. focussing on different aspects of enabling Windows workloads on Kubernetes as well as Persistent Storage in general. He is an active contributor to Kubernetes projects under SIG Windows and SIG Storage.

Jean Rouge

Senior Software Engineer, Docker
Jean is a Senior Software Engineer at Docker and an active contributor in kubernetes and various Docker open-source projects. Most recently he has led the work around GMSA support in Windows on Kubernetes. He's been passionate about DevOps since the beginning of his career: he's worked... Read More →

Tuesday November 19, 2019 3:20pm - 3:55pm PST
Room 7AB - San Diego Convention Center Upper Level

4:25pm PST

Polymorphic Reconcilers in Kubernetes - Advanced DuckTyping - Scott Nichols & Matt Moore, Google
The explosion of Custom Resources in Kubernetes has lead to the development of new techniques to reference and reconcile objects in Kubernetes. Come learn how we are leveraging some simple patterns to produce complex systems within Kubernetes in the Knative project, and how you can adapt these methods to your applications.

avatar for Matthew Moore

Matthew Moore

Software Engineer, VMware
Matt is a member of the Technical Oversight Committee for Knative, leads Knative Serving, and started Knative Build. Previously as Google, Matt was Uber TL of container tools, and was the original TL for Google's Container Registry (gcr.io).
avatar for Scott Nichols

Scott Nichols

Founder Chainguard, Chainguard, Inc
Scott Nichols is a focused on making it easy to create and understand portable event driven serverless workloads. This work is done through Kubernetes, Knative and CloudEvents.

Tuesday November 19, 2019 4:25pm - 5:00pm PST
Room 16AB - San Diego Convention Center Mezzanine Level
Wednesday, November 20

3:20pm PST

Writing a Kubernetes Operator: the Hard Parts - Sebastien Guilloux, Elastic
Building a custom controller or operator to manage your Kubernetes applications is becoming easier, with the help of libraries and tools such as controller-runtime and Kubebuilder. Putting together an initial working prototype is fairly straightforward, but devil is in the details.

This talk focuses on lessons learned while writing Kubernetes controllers for stateful workloads with the help of controller-runtime. It covers some of the "hard parts".

The operator lives in the past: how to deal with resources cache inconsistencies? Why does idempotency matter? What can you do when StatefulSets are not good enough for the orchestration you need? How to empower advanced users but still provide good defaults? What namespace(s) should the operator have access to? How to test that monster you ended up building? These are questions engineers at Elastic had to answer.

avatar for Sebastien Guilloux

Sebastien Guilloux

Principal Software Engineer, Elastic
Sébastien is a software engineer at Elastic. He has spent most of his career working with distributed systems, building resilient applications and orchestrating Apache Kafka and Elasticsearch nodes around the world. He currently works on writing a Kubernetes operator for Elasticsearch... Read More →

Wednesday November 20, 2019 3:20pm - 3:55pm PST
Ballroom Sec 20CD - San Diego Convention Center Upper Level
  Application + Development

5:20pm PST

Knative - The Security Platypus? - Ariel Shuper, Aqua Security
Knative provides a way to extend Kubernetes to run serverless workloads. Although it runs as pods, given the nature of those workloads it requires an approach to security that is distinct from standard Kubernetes security practices. As 18th century explorers were wondering when they first encountered the platypus, is it a duck? an otter? or something else?
In this talk Ariel reviews the serverless threat landscape, which is quite differentiated from the container equivalent, using examples of how coding mistakes may expose applications despite the extremely ephemeral workloads.
This talk will show how combining preventative methods and more "offensive" methods such as tripwires can provide much better visibility and reduce the risk of Knative workloads being used as attack vehicles to reach other areas of the cluster or application.
Finally, the platypus question will be resolved.

avatar for Ariel Shuper

Ariel Shuper

VP, Product Management, Portshift
Ariel Shuper is Vice President of proudct management at Portshift Security, specializing in cloud native identity based security for micro services. He specialized in serverless environments as an entrepreneur prior to joining Aqua. He also focuses on other innovative cloud native... Read More →

Wednesday November 20, 2019 5:20pm - 5:55pm PST
Pacific Ballroom, Salon 20-22 - Marriott Marquis San Diego Marina Hotel
Thursday, November 21

10:55am PST

Handling Risky Business: Cluster Upgrades - Puneet Pruthi, Lyft
Have you ever had to upgrade your Kubernetes clusters to update to a new release version, push new features or patch critical security vulnerabilities? Did it ever feel daunting to live update API masters or etcds? Can you automate such an operation?

We hope to share our musings at Lyft in solving the complexity of automating cluster upgrades and how that is incorporated into the design for - k8srotator - a Kubernetes custom controller.

As multiple components operating in cohesion make a cluster healthy, there are numerous points of failure that can occur during an upgrade cycle. Although there are varied ways of operating a Kubernetes cluster, the issues encountered during the process are common.

Attendees will walk away with knowledge about different cluster upgrade failures scenarios and ways to automate such operations without being in constant fear of losing the cluster state.

avatar for Puneet Pruthi

Puneet Pruthi

Engineering Manager, Lyft
Puneet is the Engineering Manager for Cloud Orchestration Team at Lyft which maintains the platform for microservices to interact with cloud providers. Previously he was a Senior Software Engineer on the Compute Team where he worked on supporting the Kubernetes Infrastructure and... Read More →

Thursday November 21, 2019 10:55am - 11:30am PST
Ballroom Sec 20CD - San Diego Convention Center Upper Level

11:50am PST

Kubernetizing Big Data and ML Workloads at Uber - Mayank Bansal & Min Cai, Uber
Uber relies on Big Data and ML to make business critical decisions such as pricing, trip ETA, etc. Today, those workloads such as Hive and Spark are running on YARN. To save millions of dollars by efficient use of cluster resources, Uber is planning to use Kubernetes to co-locate BigData/ML and micro-service workloads.

Kubernetes is the de-facto standard for running micro-services. However, in comparison to YARN, it still lacks many features like hierarchical resource pools, elastic resource sharing, gang scheduling etc. To bridge this gap, we have re-architected Peloton to be a set of Kubernetes scheduler and controller plugins so that we can provide feature parity with YARN.

This talk will cover:
- Learnings of running large-scale BigData/ML on Kubernetes with Peloton
- Colocation of mixed workloads
- Federation across zones
- Feature and API parity with YARN

avatar for Min Cai

Min Cai

Sr. Staff Engineer, Uber
Min Cai is a Sr. Staff Engineer in Compute Platform team at Uber working on all-active datacenters, cluster management and micro-service deployment systems. He received his Ph.D. degree in Computer Science from Univ. of Southern California. Before joining Uber, he was a Sr. Staff... Read More →
avatar for Mayank Bansal

Mayank Bansal

Staff Engineer, Uber
Mayank Bansal is currently working as a Staff engineer at Uber in data infrastructure team. He is co-author of Peloton. He is Apache Hadoop Committer and Oozie PMC and Committer. Previously he was working at ebay in hadoop platform team leading YARN and MapReduce effort. Prior to... Read More →

Thursday November 21, 2019 11:50am - 12:25pm PST
Room 15AB - San Diego Convention Center Mezzanine Level
  Machine Learning + Data

11:50am PST

Walk-through: Debugging an RBAC Problem in Istio (But Without the Swearing) - Matt Turner, Native Wave
A few months ago, I lost a whole day to debugging a problem with RBAC in Istio. I swore a lot, but I also learned a lot. I learned new tools, new interfaces, and the rabbit hole took me past most major parts of Istio.

Today I'll recreate that debugging session live, to show you the mental models and techniques I used to methodically follow this issue through a complex distributed system. We'll learn about systems debugging techniques in general, and operating Istio in particular.

avatar for Matt Turner

Matt Turner

Software Engineer, Tetrate
Matt is a software engineer at Tetrate, working on Istio-related products, and loves sharing the latest tech and trends with everyone. He's been doing Dev, sometimes with added Ops, for over a decade. His idea of "full-stack" is Linux, Kubernetes, and now Istio too. He's given many... Read More →

Thursday November 21, 2019 11:50am - 12:25pm PST
Room 6F - San Diego Convention Center Upper Level
  Service Mesh

Filter sessions
Apply filters to sessions.