Video details

Kafka, Devops...And Resilience for all • James Bowkett • GOTO 2021


This presentation was recorded at GOTOpia Chicago 2021. #GOTOcon #GOTOpia
James Bowkett - Lead Consultant at OpenCredo
ABSTRACT So you’ve made the enlightened choice to use Kafka in your architecture, and you know you need to provision your cluster using infrastructure as code (IAC), but why stop at just provisioning brokers? In this talk we will explore the available options to make deploying your Kafka-based applications more repeatable, resilient and observable. We’ll look at specific examples and techniques for applying IAC to the following areas : • Topic and partition creation • Secrets management • Configuration options
I’ll also share some lessons I’ve learned related to tooling, monitoring and backup management. And finally we’ll look at some resilience do’s and don’ts with some specific examples of where these tools have helped, and where their absence has hindered [...]
TIMECODES 00:00 Intro 02:16 Kafka - what's the difference? 03:49 Key concepts 07:30 What can I DevOpsify? 10:40 What are the options? 11:21 Bash 12:17 Terraform 13:33 Kafka config manager 14:47 Secrets management 16:48 In an ideal world 18:33 In a k8s-native world 20:27 Backups 21:54 Backup options 25:37 What about application parameters? 31:20 Monitoring 35:18 Monitoring tooling 36:33 Tooling - commercial 37:14 Tooling - open source 38:01 Key takeaways 40:12 Outro
Download slides and read the full abstract here:
RECOMMENDED BOOKS Liz Rice • Container Security • Liz Rice • Kubernetes Security • Brendan Burns, Joe Beda & Kelsey Hightower • Kubernetes: Up and Running • Chris Richardson • Microservices Patterns • John Arundel & Justin Domingus • Cloud Native DevOps with Kubernetes • Michael Hausenblas & Stefan Schimanski • Programming Kubernetes • Sam Newman • Monolith to Microservices • Sam Newman • Building Microservices • Ronnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• Mitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • #ApacheKafka #Kafka #DevOps #Resilience #IAC #Bash #Terraform #Kubernetese #k8s #MirrorMaker
Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.